A guide to email split testing

Email split testing is an important tool for working out which version of an email campaign will perform the best, giving you the chance to make sure you get the biggest impact.

Also known as email A/B testing and multi-variant testing, split testing involves sending two or more variants of an email to a small portion of subscribers. The one which performs the best after a certain amount of time will win the split test, and this winning campaign will then be sent to the remaining subscribers.

Why is this important? Even if you are currently happy with your engagement rate on your email marketing automation campaigns, if you’re not testing, there’s no way to know if you could still improve upon it. And if you’re starting to see a decline, testing allows you to experiment with new tactics with minimal risk.

When you run split tests on your emails, there’s no doubting whether coincidence was at play or whether causation was the real reason for your sudden increase in click-through rates. You know that the single change you made is actually the thing that led to the change in performance.

Email split testing gives you the opportunity to find out what your subscribers respond to and engage with. You can then use this knowledge to fine-tune details in your email campaigns and ultimately make better, more data-driven marketing decisions.

Speaking of data-driven decisions, here are some stats you need to know:

• 39% of brands still don’t test their broadcast or segmented emails
• 60% of organisations that use split-testing find it highly valuable for optimising their conversion rates
• Split testing once helped HubSpot gain an extra 131 leads from a single email just by tweaking the subject line

In this article, we’ll explain everything you need to know about email split testing and how you should be doing it.

What parts of your email are worth split testing?

So, you’ve decided to start split testing, but what should you focus on first? Will it be replacing an image with another? A different coloured call-to-action? Reducing the amount of text?

The most important rule here to make sure your tests are worthwhile is this:

Only ever test one element of your email campaign at a time.

Split testing your emails is only effective if you isolate each element, otherwise, you won’t be able to identify which change it was that impacted your performance. Let’s take a look at which aspects of your email you can test:

Subject lines

Your email’s subject line is the first thing your subscribers will see, making it one of the most crucial elements of your campaign because it helps them decide whether they want to bother looking at your email or not.

So what can you test with your subject lines?

Length is certainly one. According to Invescpro, the performance of subject lines vary according to length, and subject lines of 6-10 words perform best, with an open rate of 21%. This dropped to 9% for longer subject lines of 21-25 words. So be concise. Shorter subject lines often work better because they get the message across more succinctly.

You could even save space by experimenting with emojis, which have been shown to increase email open rates in some cases. They have to be right for your brand and target audience, of course, but split testing gives you the chance to try it out.

This example from Rockett St George shows an effective use of emojis in their subject line, with the leaves fitting naturally into the theme:

Preview header text

The preview header text of your email will also have an impact on your open rate and split testing different preview text copy could help you identify what makes your subscribers choose to open your email campaign.

The preview text needs to support your subject line, working as a team to drive open rates upwards.

The format

The way that you present information in an email can impact the way people interact with it.

The F-shaped reading pattern shows how people scan information on a screen and is a good guide for email marketers to follow.

Would a standardised design allow you to get your main message across with the level of creativity you need, or do you need to take an alternative approach?

Images

The images and positioning of your images can help your recipients scan your emails quickly. This may be as simple as moving your image from above to below a paragraph.

Have you carefully considered where your images are placed in your email campaigns? Does their positioning complement the copy you use?

CTAs

The position and wording of your calls-to-action can have an impact on your click-through and conversion rates.

Writing different call-to-action copy for a split test will improve the overall click-through rate of your email campaign when it’s distributed to the full mailing list.

How to effectively analyse your results - Top tips

With so many elements to test, there are several options for measuring the success of email variants. Your task is to decide which metrics are most important to your performance.

Everybody wants to achieve higher revenue from their emails, but what specifically do you want to achieve with your email split test? Most email marketing platforms provide a wealth of data that will help you gauge the performance of your email marketing campaigns.

The key is to think back to your email marketing goals. Your goals will tell you what email marketing metrics you should focus on - such as delivery rate, open rate, or click rate – and what you should work on improving.

You also need to plan your time carefully. If you rush the split testing process, this will have an impact on the accuracy of your results.

You need to send your email split test to a small portion of your email list, and then you must wait. Give your subscribers enough time to receive and react to your split test before judging the results.

So don’t decide to do a split test at the last minute. You should plan your split test before you start the creation process, saving you a lot of stress and potentially wasting a lot of effort.

Getting the sample size right is also crucial. The larger the split test, the more reliable your results will be. However, increasing the size of your split test also means that more people will receive an unoptimised campaign.

Use your common sense to identify whether your split test takes into account a large enough cross-section of your list.

If you only want to send your split test to a small number of subscribers, you have to remember that your split test is statistically viable. As your email marketing list gets smaller, the value in the results of a split test decline. If you need to send an email to a very small segment of your list, take a look at previous campaign data and trust your instincts.

We suggest that, for a split test to work, your minimum list size should be at least 1000. However, even then, your results won’t be as reliable as they would with a much larger list.

Getting started with email split testing

As you have seen, split testing your email marketing campaign is a great way to fine-tune your emails and improve campaign performance.

Marketers have a huge number of decisions to make when creating email campaigns, and it’s easy to become overwhelmed. Split tests can help by providing invaluable insights into what kind of emails resonate with your subscribers, so you can feel confident that you’re giving them what they want.

You can eliminate any guesswork and make better marketing decisions by running split tests on any elements of your emails. Try out different subject lines, copy, images, and templates, and we’ll help you see what works best. Contact us to discuss more.