July 21, 2006

In past articles, such as Profitable Content network bidding in Google AdWords using the new AdWords Analysis report, we discuss how to use Google Analytics to conduct A/B split tests.

Google Analytics defines A/B split testing as "Testing the relative effectiveness of multiple versions of the same advertisement, or other content, in referring visitors to a site."

An excellent article was recently written by Matthew Roche in Conversion Chronicles, "Do Your Home Page Tests Flop? We Know Why...", which outlines the four major obstacles to getting your A/B split tests on your homepage to work correctly.

The fact is that most companies make their first foray into live testing by showing two versions of a home page (often one for a week, then another). And sadly, many of these folks find that both versions perform equally well (or equally poorly).

His first point is especially helpful: don't change too much for one test! Remember all the way back to high school science experiments, and you had one 'test variable' and one 'control variable'? This is basically the same premise – you change just one variable (such as a headline, a promotion, etc) and test that against a 'control' (your normal home page, with no changes at all).

It is understandably tempting to test multiple variables at once, but the real value of A/B split tests depends on the correct procedure.

To read the rest of his four points, be sure to check out his article. If you need help with your A/B split tests, drop us a line and we'll be glad to help you convert more browsers into buyers!

Posted by Meredith Smith at 10:52 AM

Social Bookmarks:

Permalink | Comments ( 2 ) | TrackBacks ( 0 )

Filed under: Website Optimization
Tagged as: Improving Conversion

Comments

Scott Miller said:

The home page is almost always the single most difficult page to test on any site. Another issue I have seen come up is people testing a very different home page (obvious no-no) with a different template and everything, and then having it feed into the main site which still uses the original template. This might work if they were using customer engagement as their success metric, but instead they use sales. This will result in almost no learning whatsoever.

Regards,

Scott Miller
Author of "The Conversionlab.com"

July 21, 2006 12:30 PM

Meredith Smith said:

Great point, Scott! There are so many reasons that the home page is a bigger challenge to test then other pages within a site - some of which are inherent (the home page is usually the longest distance from a conversion within the site, for example), and others that are hurdles in the testing process itself.

July 21, 2006 2:24 PM