Create an A/B Testing Campaign

Each time you create an email campaign, you may wonder whether your content is presented in the best way. Use an A/B testing campaign to test how small changes to your campaigns impact your results.

In this article, you'll learn how to create and edit A/B testing campaigns.

Before You Start

  • Be sure you're familiar with how A/B testing campaigns work.
  • Depending on your plan, you may not have access to A/B Testing Campaigns. To find out what features are included in each plan, check out our pricing page.
  • We recommend you send combinations to at least 5,000 subscribed contacts to get the most useful data from your test. If the size of your audience or segment doesn't allow for 5,000 subscribers in each combination, consider testing your entire audience and use the campaign results to inform future campaign content decisions.
  • Subscribers may not interact with your campaign right away, so test data may take some time to populate. We recommend you wait at least four hours after you send an A/B testing campaign to send the winning campaign. Winner notification emails are sent to any user on your account with a user level of Manager or higher.
  • When you test content as the variable in an A/B testing campaign, choose the winner by click rate, as opposed to open rate, since opens occur before a subscriber views your content.
  • If you change your mind during the test phase, you can convert to manual selection to choose the winner from the campaign report.

Create an A/B Testing Campaign

A/B testing campaigns are built in a series of steps, just like other campaign types. If you haven't started building an A/B testing campaign yet, start at the beginning of these instructions. If you already started and saved your progress, skip to the step you want to work on.

To create a new A/B testing campaign, follow these steps.

  1. Navigate to the Campaigns page.
  2. Click Create Campaign.
    click create campaign
  3. In the pop-up modal, click Email.
  4. Click A/B test.
  5. Enter a campaign name and click Begin.
    click begin

Recipients Step

On the first step of the Campaign Builder, choose the audience or segment you want to send your campaign to. To send to a segment, configure a new segment on this screen, or choose a saved segment.

Variables Step

To choose the elements you want to test, follow these steps.

  1. On the Variables step, click the Plus icon for the variable you want to test.
    Screen of the four variables, subject line, From name, Content, and Send time.
  2. Click the Plus or Minus icons to add or remove variations of the variable. You can test one variable with up to three variations.
    Cursor hovers over plus icon to choose 3 subject lines.
    Each variation will generate a campaign combination as shown in the Summary. For example, to test three subject lines, we'll create three combinations of your campaign.
    Screen of 3 subject lines totalling up to 3 combinations.
  3. Under How should we split your recipients?, click and drag the slider to set the percentage of your recipients you want to send your test combinations to. Review the number of recipients that will receive each combination in the Summary.
    Screen of cursor dragging slider to 80%.
    If you're testing Send time, you'll always send to 100% of the subscribed contacts in your chosen audience or segment.
  4. Choose whether to determine a winner based on open rate, click rate, total revenue, or manually. Then, choose how long you want the test to run.
    Screen showing how winner will be chosen, with By Open Rate after 4 hours chosen.
    At the end of the test phase, we'll send the winning combination to your remaining recipients.

Setup Step

Depending on which variable you test, the Setup step will present multiple Email subject, From email address and name, and Subject line fields.

  1. Type in your variation content in the field provided. Send time variations settings also appear on this step.
    Screen of three different subject line options.
  2. Check the boxes next to your preferred settings or tracking options to include. Track opens and Track clicks can't be disabled, since they're used to determine the winner of the test.

Content Step

The Content step is where you add content for your campaign. What you see on this step will vary if you choose to test content.

Test Subject Line, From Name, or Send Time

  1. Choose your layout on the Template step of the Content Builder.
  2. On the Design step of the Content Builder, design and test your campaign.
  3. When everything is how you want, click Confirm before sending.

Test Content

The Content setup page shows a preview for each variation of your campaign content. To add content to each variation, follow these steps.

  1. Click Add Content for a variation of your campaign.
    Screen of content setup page with cursor hovering over to add content to the first variation.
  2. Choose your layout on the Template step of the Content Builder.
  3. On the Design step of the Content Builder, design and test your campaign. When everything is how you want, click Next.
  4. On the Description step of the Content Builder, type in text to describe the variation of the content. Your subscribers won't see this description, but it will help you remember the differences between each variation.
    Screen of Content description step, with Administration Building Photo to describe the content.
  5. Click Continue to content to return to your content variations.
  6. Repeat steps 1-5 for each content variation. On the Template step, use a different template or select a variation you already designed in the Previous Variations tab.
    Screen of Previous Variations tab with one variation option.
  7. When everything is how you want, click Next.

Confirm Step

The Confirm step is where you'll review the combinations that make up your A/B testing campaign.

  1. Review the Pre-Delivery Checklist. Click Resolve or Edit to make changes to any section of the campaign.
  2. After you're finished, Send your campaign now or Schedule it to go out later.
    Screen of Schedule and Send buttons.

Account users with a Manager user level or higher will receive an email when a winner has been automatically chosen, or after four hours if you haven't yet picked a winner manually.


  • If a subscriber opens the campaign in their browser, they'll always see the campaign combination they received, even if the winning combination is different.
  • The winning combination of your A/B test will appear in the public campaign archive for your audience, unless you tested send time.

Manually Choose a Winner

If you chose Manual Selection from the drop-down menu on the Variables step of the Campaign Builder, you'll be able to view the campaign report after tests are sent, and choose a winner.
Screen of Manual Selection choose from the What metric should determine the winner drop-down menu.

To manually choose a winner, follow these steps.

  1. Navigate to the Reports page.
  2. Click View report for the A/B testing campaign you want to work with.
  3. An alert will tell you how long into the testing phase you are and recommend that you wait at least four hours.
    Feedback block showing information about how long the test has run and the recommendation to wait 4 hours before choosing a winner.
  4. Review the report for each combination. Under Finalize your test, click the radio button next to the combination you want to send to your remaining subscribers.
    Screen of cursor choosing an option from the combination results table.
  5. Click Send To Remaining Recipients.
    Cursor clicks the Send To Remaining Recipients button.
  6. Click Send Now in the confirmation pop-up modal to send the winning campaign.

Convert to Manual Selection

After you send your test with the choice to have the winner chosen by open rate, click rate, or total revenue, you may decide that you want to manually choose the winning combination instead.

To convert to manual selection after you sent your campaign, follow these steps.

  1. Navigate to the Reports page.
  2. Click View Report for the A/B testing campaign you want to work with.
  3. Click Convert to manual selection.
    Feedback block showing the cursor clicking the option to convert to manual selection.
  4. In the pop-up modal, type CONVERT in all caps and click Convert.
    Screen of modal with CONVERT typed in the field and the cursor clicking Convert.
  5. When you're ready, choose a combination to immediately send to your remaining subscribers.


In the event that none of your recipients opens or clicks in any of your combinations, the final set of recipients will get the last combination by default. This could happen if your test sends to very few recipients.

If the same number of people click and open your combinations, the first combination will send to the remaining recipients.

Was this article helpful?

Anything else we can do to improve our site?