Creating an A/B split test email activity

Last Updated: 22/3/2024     Tags: howto, how, to, ab, a
  • Switch Version
  • V5
  • V4

A/B split testing is a way of working out which of two (or more) content options is the most effective (in regards to opens/clicks). Read more about what A/B split testing is here.

To begin, simply create a new activity or open an existing activity. You will need to determine which content you wish to split test before continuing.

Testing multiple subject lines

To add a subject line variation, simply click the 'add' link underneath the subject line field.

A/B Test on Subject Lines

This will display a second subject line field which you can then populate. If you wish to add more subject lines, simply keep clicking the 'add' link until the desired amount of subject line fields appear. There is no limit to how many subject lines you may use, but we would recommend using five or less depending on subscriber numbers.

Testing multiple content blocks

To test multiple content block variations, simply click the 'optimize' link in one of your content blocks.

A/B Test on Content Block

This will add an alternative block which you are able to then expand and populate. If you wish to add more variations, simply keep clicking the 'add' link until the desired amount of variations appear. There is no limit to how many variations you may use, but we would recommend using five or less depending on subscriber numbers. With A/B split testing, you will need to have the same number of content blocks as you have subject lines.

Understanding combinations

Since A/B split testing forms combinations and tests them against other combinations, it is important to understand how combinations are made when adding subject line and content block variations.

Here's an example: If you had three subject lines, three hero image variations and three banner block variations, this would form three combinations consisting of:

  • Subject Line 1 + Hero Image 1 + Banner Block 1
  • Subject Line 2 + Hero Image 2 + Banner Block 2
  • Subject Line 3 + Hero Image 3 + Banner Block 3

Each variation is paired with other variations in numerical order to form a combination.

If you had two subject lines, three hero images and three banner blocks, only two combinations will show as the third combination requires a subject line to be paired with it.

Proofing an A/B split test activity

When sending proofs, no further options need to be enabled. Our system will detect any A/B split test combinations and will send proofs for each combination.

You can also see combinations via the UI by selecting a combination from the dropdown box at the top of the activity preview window:

Combinations

Broadcasting an A/B split test activity

Broadcasting an A/B split test activity is the same as broadcasting a standard activity, simply approve the activity and click the 'Deploy' button. Ensure you adjust the throttle to a reasonable rate to allow our system to track subscriber interactions during the send and perform the split test. For more information on how our system works in regards to split testing and subscriber interaction tracking, click here.

Combinations