Testing two versions of something is a great way to 1) lift response and 2) gain a greater understanding of your customers.
Plus, the tools for testing your web pages and emails keep getting better, cheaper, and easier to implement.
How does A/B testing work? For a website, you would first build a variation of an important page. Then you would set your website to show that variation to some of your website visitors, and measure what happens afterwards. For a fuller definition, check out this glossary entry at Anne Holland’s Which Test Won.
What to Test?
So, what parts of your web pages and emails can you change?
- Copy. Start here, by tweaking word choice. You can also test how people respond to different articles, which can give you insight into what buyers care about.
- Formatting. Is a larger font better?
- Layout. This is harder to do, but appropriate for high-volume response pages.
- Button colors and size, especially anything that’s a call to action.
In an upcoming blog post I’ll review the web page testing tools. Today we’ll look at email.
My favorite service for blasting out emails, Campaign Monitor, has a built-in testing tool. It’s not exactly new, but oh is it easy. If you’re sending out a promotional email, there’s really no reason not to do this kind of test.
Here’s how it works.
Say you have a list of 2,000 addresses. You take the email you’re about to send, and think of two different subject lines. Version A gets sent to 500 people, Version B gets sent to 500 people, and the service tracks how many people open and clicks. After a preset number of hours, the remaining 1,000 get sent the winning version.
You can also set Version A and B to have differing email bodies, or designs, or From addresses.
Below is the money shot from the Campaign Monitor help page.
The sizes of A and B are set with the grey slider.
If your mailing list has thousands of people, testing 30% of the list (as shown in the image) would be adequate. The people at Visual Website Optimizer have a nice tool for determining statistical significance.