Blog Navigation

Blog Updates Via RSS

Main

March 30, 2010

Google Website Optimizer is a very capable testing platform allowing you to setup complex multivariate experiments using a simple, intuitive interface. So what's the problem?

AVERAGES LIE

Simplicity, while being Website Optimizer's greatest strength, is also a tremendous weakness when it comes to data segmentation. Google Website Optimizer doesn't offer the data granularity that Google Analytics does.

In Website Optimizer all of your traffic, no matter how different the traffic source, is lumped together into a single bin. You might be sending traffic from paid search, organic, and content to the same page. When you view data for an experiment on the page, you'll see aggregate data across all traffic sources...

TEST RESULTS WILL DIFFER ACROSS TRAFFIC SOURCES

When we run a Website Optimizer test results at ROI, we manage our experiments just as you would expect - running a test until we have a winner within the Website Optimizer interface. However, when the test is completed, that's when things get interesting.

WHAT DOES A WIN REALLY LOOK LIKE?

We use Google Analytics to dive deep into our Website Optimizer test results, breaking up the test data by traffic source, campaign, ad group, and any other significant way we can segment traffic. When you view the results on campaign by campaign basis, things start to look a little different.

Here's an A/B test we recently ran for one of our clients testing a revamped headline on their paid search landing page:

Sweet - a 60% win overall! But our job isn't over yet...

When we break the test down campaign by campaign in Google Analytics, we can see that the new headline actually led to as much as a 40% decrease in conversion rate in some of our campaigns (Seeing A/B Experiments in Google Analytics). So what should you do about it?

ELIMINATE THE LOSSES

Well, the short answer is to segment your traffic. Typically, when people have a win in an A/B test they send all traffic to the winning page. When you do this, you understate your win.

Take the following example. We run a Website Optimizer test and find after a week of testing that Page B gets 100 more conversions then Page A. Accordingly, we send all traffic to Page B for a 6.45% improvement.

However, if you look closer at the data from our test, you'll see that Page A actually performed better than Page B in Campaign 2. If we optimize where we send our traffic, we can get an extra 50 conversions bumping our actual improvement up to 9.68%!

I GET IT, HOW SHOULD I SEGMENT MY TRAFFIC?

There are a number of different ways to send visitors from different traffic sources to different pages. By far the easiest method is simply update your destination URLs to point to an optimized page for each traffic source.

Really, it doesn't what method you choose to split your traffic to the appropriate page. Just make sure whatever system you go with is robust/manageable enough to handle ongoing testing.

Interested in learning more about Google Analytics?
Get the latest tips and tricks sent to you via our free, twice-monthly ROI Revolution newsletter. Or for customized help with Google Analytics implementation, check out our Support Plans. Here at ROI Revolution, we consider Google Analytics tracking essential for paid search, so it's included in our PPC Campaign Management service.

Comments

Jacob Lucky, Website Optimization Specialist said:

One of my favorite ways to segment traffic on the landing page is to use the Google Analytics manual tagging parameters to allow the page to 'know' what traffic source the visitor came from.

Let's say that you run an A/B experiment and find that Page A works better for some traffic, while Page B works better for other traffic. You can use server-side code (or JavaScript) to read the manual tagging parameters and pull in the winning content for that specific traffic source.

I keep an XML file that stores all of my clients sources, campaigns and ad groups that send traffic to a specific page with a key for what content should be displayed for each specific traffic source. The landing page uses server-side code to quickly look up the traffic source and then uses the keys to find the specific content that should be displayed.

April 5, 2010 3:22 PM

Post Your Comments

© Copyright 2009 ROI Revolution. All Rights Reserved.