$500 million is a big sum by anyone’s measure. Now imagine using email to deliver most of that money in just 18 months. Toby Fallsgraff and his team raised a sum not far off that amount for the Obama campaign last year. In fact, the digital department on the Obama campaign accounted for the bulk of the election funds raised from donors.
The strategy behind this incredible success involved something the email team called “Frankenstein testing”. I was fortunate enough to catch-up with Toby and he shared details of his strategy.
Before getting to the Frankenstein’s let me first give the bigger picture. Three things that were key to success:
- List size – the campaign did have a big quality database to leverage
- Email frequency – OK, so a lot of email was sent
- Test and optimization – up to x6 more revenue was produced per campaign by testing
I’m going to share with you their approach to the test and optimization piece of the puzzle. In Toby’s own words “we did as much testing as humanly possible”.
The fundraising emails were all about getting donations and so copy had to do most of the work persuading donors to open their wallets. The process used is as follows:
- A team of 18 writers created 4 to 6 different body copy messages for every campaign.
- Each body copy variation was sent out to test audiences with three different subject line choices
- Large test cells of 50,000 contacts were used to determine winners quickly
- The most important evaluation metric was simply the number of donations.
- The test cells were usually sent around 7am and a winner picked about 60 to 90 minutes later
- The winner went to the remaining database, but some segments received slight variations on the winner depending on their donation history.
You may be wondering how 18 writers could be afforded. Simple, the uplifts seen more than paid for them. On the ‘I will be outspent campaign’ of June 2012 the winning campaign raised $2.5m and a projected $2.1m more than the worst performing cell. That pays for the copy writers and the 10 analytics engineers checking over the results.
Occasionally although there was a winner, none of the emails performed to the level expected and in these cases the whole send was canned. The mentality was not one of send unconditionally.
Now let’s return to Frankenstein. This is the idea of taking the best bits of messages from different test cells to make a super email with the hope of creating monstrous results (cough, sorry).
In some cases a generic subject line was used to allow an evaluation of the body copy alone, such as use of the subject line ‘Hey’. This could be matched and make sense with any other body message.
A Frankenstein test might then take body copy that gave good results and match with a subject line that gave good results, expecting to see the two together being a best combination. Obviously this could only be done when the subject line was not so specific to the body that it would not have sense.
What was interesting was that the Frankenstein’s’ mostly didn’t turn out to be winners. Sometimes yes, but mostly not.
The learning was that the combination of subject line and body copy was found to be important.
The subject line impact is much more than just getting the open. How the subject line frames the subscribers thoughts and how they perceive the whole message as a result of the subject line plays a big role.
Subject lines must be optimized for the action and not just the open.
For completeness let me point out it wasn’t just subject lines and body copy that was tested, other items tested included; calls to action, donation amounts, text formatting and emphasis.
Acknowledgement: My sincere thanks to Toby for taking the time to share this information with me.