Receive new blog posts immediately direct to your email inbox!
The ROI Revolution Blog
« LAST CHANCE: To Attend the Washington, DC Google Website Optimizer Seminar For Success! | Main | UPDATE: Urchin 6.5 Is Released For Download! »
Excluding Robot Traffic in Urchin 6
December 19, 2008
If you are using the IP+User+Agent method to track visitors in Urchin 6, you've probably noticed that quite a bit of your traffic is actually robot traffic:
While it's interesting to see this activity broken out, most of the time you're better off excluding it from your reports.
Urchin 6 does offers a pre-configured filter to "Filter out robot traffic":
Unfortunately, this filter does not nearly exclude all the robot traffic.
Now we can, of course, update this filter so that it matches more of the robots we're seeing in this report. There are two obvious problems with this:
- You have to keep updating your filter every time a new robot pops up that doesn't match your filter.
- There are both bots and genuine browsers that show up under the Mozilla Compatible Agent entry, so you can't exclude this outright.
Fortunately, there is a solution:
As you can see, this is an extremely simple filter that makes use of the robots_agent (AUTO) field and will require no maintenance whatsoever. Robots will be excluded using the same criteria that got them into the Robots reports in the first place.
Now that's a thing of beauty!
The robots_agent field is new to Urchin 6. If you're still using Urchin 5 and having trouble excluding robots, now might be a good time to upgrade.
Also, Urchin 6.5 has just been released. This release will be of particular interest to those of you using the UTM method and tracking your AdWords Campaigns through Urchin. With Urchin 6.5, you now have the ability to directly download AdWords CPC data to Urchin using the AdWords API.
Leave a comment if you've found this filter useful, or if you have any other questions about Urchin.
ROI Revolution is an Urchin 6 reseller and so if you are ready for Urchin 6, go to www.roirevolution.com/urchin/urchin-six.htm to purchase or upgrade to Urchin 6 today! If you'd like to evaluate Urchin 6, download a FREE 30-day trial.
Posted by Jeremy Aube, Analytics and Optimization Specialist at 3:21 PM
Permalink | Comments ( 5 ) | TrackBacks ( 0 )
Filed under: Urchin Software
TrackBack
TrackBack URL for this entry:
/mt/mt-tb.cgi/335.
Comments
anything like this in google analytics?
Posted by: steve at December 23, 2008 1:36 PM
@steve: Because bots tend to not process javascript, bots will be excluding from your Google Analytics reports by default.
Similarly, properly configuring the UTM method of tracking for Urchin will also exclude bot traffic, but this isn't always an option for sites with strict privacy policies or for processing historic data.
Posted by: Jeremy Aube, Google Analytics Support Tech
at
December 23, 2008 1:52 PM
Unfortunately, this only works with robots that declare themselves as robots. There are a few that mask themselves as typical users, though you can tell by the amount of data they download that they are not a typical user.
To identify these users, look at the IT Reports > Domains & Users > IP Address report, and sort it by the Bytes column.
There will be a clear separation between those robot users and your typical users, with the robot users taking - at minimum - twice as much data as other users. for the same time period.
Posted by: Sean at December 30, 2008 11:44 AM
@Sean: Good point. The additional robots that you identify through the report you referenced could easily be added to the default exclude robots filter that Urchin provides and used in conjunction with the filter described in this post. Thanks for the heads up on this!
Posted by: Jeremy Aube, Google Analytics Support Tech
at
December 30, 2008 11:57 AM
Thanks for the heads up on the new robot_agent field - it does indeed simplify filtering out robot data.
Mich�le
