Effects of List Segmentation on Email Marketing Stats
Does list segmentation really help email marketing stats?
It’s common knowledge that segmenting your email-marketing lists helps you get better open and click rates. By narrowing your focus and sending messages to targeted groups within your lists, your recipients will find your campaigns more relevant—and relevant campaigns get better results.
We scanned our system for MailChimp users who use our list segmentation feature and sampled about 2,000 MailChimp users who sent about 11,000 segmented campaigns to almost 9 million recipients. We compared the results of those segmented campaigns to the results of the same customers’ non-segmented campaigns. The results are interesting—some are even counter-intuitive.
When we measured stats “across all segmented campaigns,” segmented campaigns distinctly improved email-newsletter performance almost across the board, except for one metric: unsubscribes. Why would more people unsubscribe from targeted messages that are supposedly more relevant?
|Opens:||14.444% better than the list average|
|Clicks:||14.994% better than the list average|
|Bounces:||0.803% better than the list average|
|Abuse Reports:||0.010% better than the list average|
|Unsubs:||0.632% worse than the list average|
When MailChimp users segmented their lists based on some field in their recipient database (examples might include "customer_type" or "ZIP code" or "job_title"), results went way up compared to non-segmented lists. This turned out to be the most popular way to segment lists.
|Opens:||18.852% better than the list average|
|Clicks:||21.976% better than the list average|
|Bounces:||1.429% better than the list average|
|Abuse Reports:||0.015% better than the list average|
|Unsubs:||0.986% worse than the list average|
|Total campaigns of this type: 6,040|
MailChimp has a feature that lets you segment your lists based on "those who signed up within the last x days" or "those who signed up since I sent my last campaign." One common use for this segment is to send a campaign to your most recent subscribers.
|Opens:||11.641% better than the list average|
|Clicks:||10.501% better than the list average|
|Bounces:||0.492% worse than the list average|
|Abuse Reports:||0.002% better than the list average|
|Unsubs:||0.126% worse than the list average|
|Total campaigns of this type: 2,761|
Email marketers can create signup forms with checkboxes so subscribers can indicate their interests. For example, a music website might have an email signup form with options for favorite musical genre. When segmenting campaigns based on these interest groups, MailChimp users got slightly better results than average.
|Opens:||1.659% better than the list average|
|Clicks:||1.712% better than the list average|
|Bounces:||0.206% better than the list average|
|Abuse Reports:||0.002% better than the list average|
|Unsubs:||0.071% worse than the list average|
|Total campaigns of this type: 851|
Here's the shocker. MailChimp offers an add-on module called Subscriber Activity Reports that allows our users to track user-specific stats in their email campaigns. For example, instead of just seeing your campaign's total number of opens and clicks, this add-on allows you to drill down and see who opened and clicked. This allows for some extremely focused segmentation, which you might think would improve your results dramatically. For example, you can use this to segment a list based on "those who opened a previous campaign" or "those who opened my three most recent campaigns" or "those who did not open my last campaign." But it turns out almost all metrics went down when users segmented by "Subscriber Activity Reports."
|Opens:||2.072% worse than the list average|
|Clicks:||2.524% worse than the list average|
|Bounces:||1.842% better than the list average|
|Abuse Reports:||0.003% worse than the list average|
|Unsubs:||0.262% better than the list average|
|Total campaigns of this type: 297|
For the most part, segmenting your email-marketing lists improves your open and click rates, and reduces the number of bounces from each campaign you send. We’re baffled as to why more people would unsubscribe from seemingly more relevant campaigns. We have some theories: Maybe the segmented campaigns were sent in addition to normal batch-and-blast campaigns, which resulted in annoying duplicate messages, or maybe the content was just too specific. We’ll save that for another study.
Most disturbing was the discovery that Subscriber Activity Reports generated worse results than non-segmented campaigns We had to look more closely at what was causing this, because the Subscriber Activity Reports add-on is so wonderful, there's no way it could possibly do any harm (scientifically speaking, that is). Turns out the majority of users who had Subscriber Activity Reports installed weren't using it to send special emails to loyal subscribers ("segment based on those who opened my recent three campaigns"), but they were using it to send follow-up campaigns to “those who did not open my last message.” This has been documented on email-marketing sites as an extremely effective tactic to generate more bookings by hotels and event organizers (Marketingsherpa: Should You Re-Send Your Email Newsletter to Non-Openers?). But when you factor in how inherently inaccurate open-rate tracking is, it’s understandable that some of these follow-up campaigns are perceived as pesky duplicates to some recipients.