The ROI Revolution Blog

Nagging Problems with Home Page A/B Testing

July 21, 2006

In past articles, such as Profitable Content network bidding in Google AdWords using the new AdWords Analysis report, we discuss how to use Google Analytics to conduct A/B split tests.

Google Analytics defines A/B split testing as "Testing the relative effectiveness of multiple versions of the same advertisement, or other content, in referring visitors to a site."
Blindfold.jpg
An excellent article was recently written by Matthew Roche in Conversion Chronicles, "Do Your Home Page Tests Flop? We Know Why...", which outlines the four major obstacles to getting your A/B split tests on your homepage to work correctly.

The fact is that most companies make their first foray into live testing by showing two versions of a home page (often one for a week, then another). And sadly, many of these folks find that both versions perform equally well (or equally poorly).

His first point is especially helpful: don't change too much for one test! Remember all the way back to high school science experiments, and you had one 'test variable' and one 'control variable'? This is basically the same premise – you change just one variable (such as a headline, a promotion, etc) and test that against a 'control' (your normal home page, with no changes at all).

It is understandably tempting to test multiple variables at once, but the real value of A/B split tests depends on the correct procedure.

To read the rest of his four points, be sure to check out his article. If you need help with your A/B split tests, drop us a line and we'll be glad to help you convert more browsers into buyers!

If our experiences with paid search have taught us anything, it's that deliberate keyword lists, tightly structured accounts, and proven ads are only half of the equation. Intuitive landing pages tested with Google Website Optimizer experiments are just as essential for profitable PPC results. Learn more about how we maximize ROI from the search to the finalized sale.

Comments

Scott Miller said:

The home page is almost always the single most difficult page to test on any site. Another issue I have seen come up is people testing a very different home page (obvious no-no) with a different template and everything, and then having it feed into the main site which still uses the original template. This might work if they were using customer engagement as their success metric, but instead they use sales. This will result in almost no learning whatsoever.


Regards,

Scott Miller
Author of "The Conversionlab.com"

July 21, 2006 12:30 PM

Meredith Smith said:

Great point, Scott! There are so many reasons that the home page is a bigger challenge to test then other pages within a site - some of which are inherent (the home page is usually the longest distance from a conversion within the site, for example), and others that are hurdles in the testing process itself.

July 21, 2006 2:24 PM

Scott Miller said:

The home page is almost always the single most difficult page to test on any site. Another issue I have seen come up is people testing a very different home page (obvious no-no) with a different template and everything, and then having it feed into the main site which still uses the original template. This might work if they were using customer engagement as their success metric, but instead they use sales. This will result in almost no learning whatsoever.


Regards,

Scott Miller
Author of "The Conversionlab.com"

July 21, 2006 12:30 PM

Meredith Smith said:

Great point, Scott! There are so many reasons that the home page is a bigger challenge to test then other pages within a site - some of which are inherent (the home page is usually the longest distance from a conversion within the site, for example), and others that are hurdles in the testing process itself.

July 21, 2006 2:24 PM

Post Your Comments

© 2002-2014 ROI Revolution, Inc. All rights reserved.