Easy Email A/B Testing

Testing two versions of something is a great way to 1) lift response and 2) gain a greater understanding of your customers.

Plus, the tools for testing your web pages and emails keep getting better, cheaper, and easier to implement.

How does A/B testing work? For a website, you would first build a variation of an important page. Then you would set your website to show that variation to some of your website visitors, and measure what happens afterwards. For a fuller definition, check out this glossary entry at Anne Holland’s Which Test Won.

What to Test?

So, what parts of your web pages and emails can you change?

  • Copy. Start here, by tweaking word choice. You can also test how people respond to different articles, which can give you insight into what buyers care about.
  • Formatting. Is a larger font better?
  • Layout. This is harder to do, but appropriate for high-volume response pages.
  • Images.
  • Button colors and size, especially anything that’s a call to action.

In an upcoming blog post I’ll review the web page testing tools. Today we’ll look at email.

My favorite service for blasting out emails, Campaign Monitor, has a built-in testing tool. It’s not exactly new, but oh is it easy. If you’re sending out a promotional email, there’s really no reason not to do this kind of test.

Dead Simple

Here’s how it works.

Say you have a list of 2,000 addresses. You take the email you’re about to send, and think of two different subject lines. Version A gets sent to 500 people, Version B gets sent to 500 people, and the service tracks how many people open and clicks. After a preset number of hours, the remaining 1,000 get sent the winning version.

You can also set Version A and B to have differing email bodies, or designs, or From addresses.

Below is the money shot from the Campaign Monitor help page.


The sizes of A and B are set with the grey slider.

If your mailing list has thousands of people, testing 30% of the list (as shown in the image) would be adequate. The people at Visual Website Optimizer have a nice tool for determining statistical significance.

Tools for Video Marketing: Flimp and VisibleGains

If you’re a business that has decided to make a promotional video, the easiest thing to do is to post it to your website. That’s simple, and hosting the video is also easy whether you use Vimeo or YouTube (my take here).

Yet, if you’ve got 1) lots of website traffic; and/or 2) a large email list; and/or 3) a mental commitment to using video to grow your sales, there are at least two interesting tools for going beyond the simple embed-video-on-website approach.

For those of you who don’t have a video (i.e. finished footage), understand now that it takes thousands of dollars to make video look professional. The lighting, the audio, the postproduction… It’s a lot harder than a brochure, and comparable in effort to building a simple website. There are a handful of exceptions: “Business casual talking head” video can be done effectively in house, and the cinema verite approach can work for some people like Jim the Realtor.


Flimp is a paid service that you use to embed your video into a custom-made “microsite”, i.e. a standalone website with just a little content. You get to lay out and add your own text and images to the microsite, using an online click-drag-and-type editor. Most important, you add links from this microsite to your company’s website. Then, you upload a list of prospect (or customer) email addresses into their system, and they send out an email blast. You then see who has clicked into the site, and who has clicked over to your main website.
flimp-logoSee a sample here (opens new window). Note that the video is set to auto-play once opened, which may lift conversion but will annoy a few people along the way.

The sample microsite, incidentally, is cluttered with copy that competes with the video.

The Flimp team is positioning their tool as an quick and less-technical way to get video out in front of your prospects and customers. Yet you need someone with design talent to avoid having a homemade look — which usually degrades your conversion rate. Moreover, they use the term “video brochure” to describe their microsites, which is an overstatement since their microsites are apparently a single page.

Where does it fit into the sales cyles? Flimp is a lead development tool, aiming to convert cold leads into warm. It’s not a lead generation tool, since without email addresses you won’t get any additional benefit over simply placing the video on your website. Moreover, there is no incremental SEO benefit to Flimp.

Do temper your expectations when you hear “video” and “email” in the same sentence. Your prospects won’t see video right in their email program — not with Flimp or anyone else. Email clients, from Outlook to web-based email programs to mobile, hardly show images in their default view. Below are screen shots of 1) a Flimp email as it appears in Gmail with the default display options, and then 2) after permissioning Gmail to show images. No video, just a link to the microsite.

Flip’s email blast – not too exciting…

Now images are displayed, which invites the click.


Flimp is about $1800 a year, with addon usage fees kicking in at a high volume. It makes the most sense for businesses with 1) a large email list; 2) low video production costs; and 3) a marketer with moderate technical and design talents, and an appetite for testing. Here’s their website.


The VisibleGains product is different than Flimp, technically and in the role it plays in the sales cycle.
vg-logoTo explain how, let’s take a step back. When someone comes to your website, they have a task — usually to learn more about who you are and what your products do. Typically, people navigate through your site’s text and images to get to their answer. Along the way, they might lose interest, get frustrated, or get lost.

VisibleGains makes your video part of the user’s site navigation experience, and counts on the more engaging nature of video to retain more people on the site as they seek their answer.

The VisibleGains innovation is addition of built-in calls to action after the video stops (which is comparable to the “related video suggestions” you see after a YouTube video).

Take a look at their site now. They really really want you to play that video in the middle of the age, right? And then they steer you in several directions.

Knowledge of the customer’s needs is critical with this tool. You need to script the video and the calls to action to answer the most questions, and best develop leads.

That last point is a big difference between our two tools. Flimp helps your conversion from email to website, and VisibleGains helps your conversion on the website. Thus, companies with a lot of offline, PR, and SEO in their marketing mix would be better off with the VisibleGains product.

VisibleGains runs about $3600 per year, with the price stepping up with the number of custom videos running on your site. Logically, then, this is a tool best suited for companies with few big-ticket products, instead of many low-margin products. Also, service companies benefit more from the VisibleGains product since people per se are part of the product.

Does it Make Sense?

Before you commit to these tools, think about:

  • Is your web traffic high enough, or your email list large enough, to offset the large fixed costs of these tools?
  • Can you sell more by improving your conversion from email, or converstion of onsite visitors? (Or is it something else?)
  • Do you know well enough which video creative will really engage your customers? That is, can you make something they’ll watch?
  • Is your house email list straightened up enough, and segmented?
  • Is the person who would run these programs hungry to test different online promotions? It’s uncommon to get things totally right on the first try.


All About Online Content, for the Business Owner

Most of the material on this blog is about improving the online reach of your company’s selling message. Or getting the message out in less time, or for less money, or higher conversion effectiveness.

Of equal importance to the distribution of the message, though is the message itself.

The content.

Herewith, some basics about online content…

Is Online Content Important For My Business?

If your customers are online when they do any part of their research or buying, YES.

If you have a national footprint, YES.

If you care about your position in the search engines, YES.


A Definition

Content is anything you produce other than your product that your customers choose to spend time with, and tells them at least indirectly how your company can fix their problem.

Let’s break that definition down into three parts.

“anything you produce other than your product”

The most obvious example is a description of your product or service – which can be written and/or photographic and/or video. More examples include customer case studies; testimonials; FAQ; user forums; press releases (sometimes); the About Us page on your website; a newsletter; your company blog, your LinkedIn/Facebook page; etc.

“that your customers choose to spend time with”

This bit is here to indicate the context: a person is actively evaluating your product or service. They have a need, they are looking for a solution, and their mind is open. This is your sales and marketing opportunity.

“tells them at least indirectly how your company can fix their problem”

Here is where you convey your own differentiating selling message, conveying the truth about your products in an engaging and relevant way.

Another way to look at content is thus:

Content Not Content
Graphic design
Direct Mail
Forms and other response mechanisms
Event planning


What’s The Role Of Content In The Buying Process?

  1. Advertising attracts.
  2. Content answers questions, engages and educates.
  3. Systems qualify, move prospects (i.e. develop leads) , and measure.
  4. People close.


Two Types of Content

The content your marketers churn out is one of two kinds.

  • The static, evergreen sort like brochures, FAQ, executive bios, and so on.
  • Another sort, which I’ll call the “Steady Stream” of content. This is the rest: the engaging, the promotional, the conversation, and the personality. Steady Stream comprises blogs, bylined articles, PR, newsletter articles, photos, videos, tweets, Facebook page updates, and so on.

The time and money commitment for Steady Stream content is an order of magnitude greater than that for static content, but the upside is equally great.

So Why is Online Content Important?

First off, content is the best way you can develop your leads, without talking to a prospect. And it’s scalable. Content usually doesn’t generate leads on its own. But better content helps revenue by improving your conversion to sale.

Second, content is the most important factor in search engine optimization. If you want a long-term high position in Google for high- and medium-traffic search terms, you need good content.

Third, the effectiveness of online content can be measured, and thus optimized through A/B testing.

Last, Not all companies produce online “steady stream” content effectively. (As compared to producing basic websites, business cards, voicemail etc.) Since your company’s peer group will have “haves” and “have-nots”, there is a competitive opportunity for the firm that creates online content consistently well.


Video hosting: The YouTube vs. Vimeo choice, about control and mobile distribution

What makes a marketing video successful? Most of the big factors have little to do with the technology, and instead are editorial and promotional in nature: The script, production, talent, and promotion/distribution.

My focus here is on a more mundane issue: where to host the video. That is, where does the video file “live”. A video can be embedded into your website, but be hosted by another company like YouTube or Vimeo. (Nearly all of your customers won’t notice the difference, and those that do won’t care.)

First off, don’t host video yourself. The difference in functionality between your own homegrown video player and service level versus Vimeo vastly exceeds the minimal cost of hosting.

There are some key differences between the two best hosting services, YouTube and Vimeo. I’m a fan of the latter for most B2B situations.

YouTube vs. Vimeo

Branding. If you embed a YouTube video on your site, you’ll see the YouTube watermark in the lower-right hand corner. That’s a bit of a distraction for your customers, and can cheapen the look somewhat. Vimeo offers a paid tier of service permitting you to remove their “V” logo.


vimeoViewership Control. All YouTube videos, if clicked when playing, sends the user to the video’s page at YouTube.com. That’s another potential distraction. If you’re using video on a landing page, where conversion is essential, don’t give prospects this kind of side door. Vimeo’s paid tier doesn’t have this click-away feature.

Privacy. Sometimes you want only a selection of people to see a video, if you’re sharing proprietary or competitive information. Examples include videos for an intranet, or an outside sales force extranet. Vimeo is the way to go here; you can prevent anyone from seeing a video except if they’re on a certain site. YouTube’s methods for limiting access are imperfect for business use.

Virality. Generally, YouTube is the choice for consumer-facing videos, since their reach is 10X that of Vimeo. Your video is much more likely to be encountered by people watching related videos, which means more traffic.

Cost. YouTube is free but has one set of features for everyone. Vimeo’s paid tier is $60 per year.

Mobile. Until last week, this was a major competitive difference between the two hosts. YouTube’s market power earned it perfect integration in the iPhone from Day 1. Vimeo, whose video player was Flash-based and thus nonfunctional on the iPhone, was left out in the cold. Yet on August 17, Vimeo finally upgraded their player code; now their videos work on iPhones and Android devices.

youtubeVideo Duration
. YouTube videos are limited to 10 minutes (July 2010 UPDATE: It’s now 15 minutes. Dec 2010 UPDATE - Now even longer…), which won’t work for long presentations or training material. Vimeo has a rather commodious filesize limit, so you can do very long videos.

Measurement. Both hosts have a solid set of metrics. Vimeo counts the number of times a video is watched in its entirety.


At the risk of oversimplifying, B2B companies are generally better off hosting their videos at Vimeo, and firms selling to consumers should probably start with YouTube.