Subject Line Data: Choose Your Words Wisely
Most people quickly scan the subject lines in their inbox before deciding which messages are worth their time and attention. With so much pressure on the subject line to entice the potential reader, we thought it would be interesting to see how much of a difference a single word can make in a campaign’s open rate.
To get some answers, we studied approximately 24 billion delivered emails with subject lines composed of approximately 22,000 distinct words. If you think that sounds like a lot of data, you’re right. We looked at subject lines both in general and within specific industries. Here’s a quick rundown of our criteria and approach:
- Investigate campaigns sent by users from the United States with tracking turned on in the past year. Only consider campaigns that were sent to 500 recipients or more, and only consider campaigns sent by users who have sent 10 or more campaigns before.
- For each campaign, calculate the open rate and standardize it using the user/list average open rate and standard deviation.
- Remove special symbols and convert subject lines to lowercase. For any given word, average the performance of all related subject lines and perform t-tests to identify high-impact words.
- For every subject line being tested, create flags for the presence of high impact words. Perform a correlation analysis on word presence to determine which words are frequently used together. Create additional flags for frequently used word combinations.
- Perform a linear regression analysis to estimate the impact each word has on standardized campaign open rates when accounting for all other tested words. Repeat this process on industry-specific data sets.
The numbers presented below are standard deviations from the mean open rate for a user/list. Words with positive impacts resulted in increased open rates, and words with negative impacts hurt those same rates.
Results for comparable word groups
The output of the regression analysis suggests that similar words often have similar impacts on open rates. Makes sense. Still, choosing the right words can result in higher open rates without altering the bottom line of your message. Again, to interpret these results, it’s important to know that a standard deviation is a standardized measurement of how much something deviates from the average value. One standard deviation for a user who tends to see large swings in open rates will be a higher percentage than it will be for someone with consistent open rates. That means choosing words wisely will have a larger impact on open rates for people with a higher standard deviation, while users with very consistent open rates can expect to see smaller changes.
Mailchimp’s merge tags let senders include first and last names provided by the recipients in campaign subjects or bodies. The impact this has on open rates has been debated before, but the consensus is that it’s positive. Our analysis found that personalization does indeed increase open rates. One of the most interesting findings is that, though the use of both first and last names in a subject is less common, it has the largest positive impact on open rates. Many of these campaigns seemed to contain highly personalized content.
Congratulations, *|FNAME|* *|LNAME|*
TED2014: Invitation to register for *|FNAME|* *|LNAME|*
Hi *|TITLE:FNAME|* *|TITLE:LNAME|*, please update your email preferences