Enhanced Google Website Optimiser tracking using setVar and Event Tracking

Moneyspyder has been using Google Analytics and Website Optimiser for some considerable time now. As Google Analytics Authorised Consultants and Website Optimiser Authorised Consultants we are constantly looking for new opportunities for clients to improve the quality of their sites across all their KPIs.

When analysing Website Optimiser test data, we've identified a serious need for more fine grained data to get really crisp results. For example, Website Optimiser will tell you how many visits have occurred to each of your experiment variations and how many visits have converted. The nature of the conversion may change of course as might the secondary and tertiary effects of tests. How can we see whether a test of the on-site search functionality has increased ecommerce conversion? Can we see whether customers engage better with the site if they see more featured products? Website Optimiser data won't necessarily answer these questions out of the box. We've identified a couple of techniques that will help though.

The test(s)

Babyetc.co.uk was about to conduct a couple of tests. I'll gloss over the tests here because this post is more about the how rather than the what'. In a nut shell - are the featured products adding value and can search or newsletter signup perform better sitting next to the logo?

Newsletter signup in the header with no featured products:


Search in the header with featured products:


The tests ran for a couple of weeks and the first result comes in:



No great surprise - search usage is improved by nearly 40%. The other test is still running though. So, were searchers buying more? Are featured products engaging customers? Are test subjects spending longer on the site? Are these changes adding value to the bottom line? We'll consider event tracking first of all:

Google Analytics Event Tracking

Event tracking is currently a beta feature. You'll need to request it to be added to your Google Analytics account. It's a much sweeter mechanism for recording single events than virtual page views especially if you don't want your content report data soiled.

First of all, get your test set up and running normally. Now, you want to record an event firing for each variation and potentially for each conversion. We'll cover variation tracking - goal tracking is a lot easier!

You'll need to find out your experiment number - you can find this in the WO control script provided (it's the variable k):

function utmx_section(){}function utmx(){}
(function(){var k='123456789123',......

Now, before you track your event, let the WO script do the first part of it's magic, then using the experiment number, you can interrogate the __utmx cookie to find out what variation is being served:

 variation = '';
 x=find_utmx('__utmx');
 if(typeof(x) != 'undefined'){
   variation = x.substring(x.length,x.length-1);
 }

Before any further WO scripting - track the event (uacct is the GA account ref used for your given Ga profile):

  var tracker = _gat._getTracker(uacct);
  tracker._initData();
  if(label != ''){
      if(value != null){
        tracker._trackEvent(category,action,label,value);
      }else{
        tracker._trackEvent(category,action,label);
      }      
  }else{
    tracker._trackEvent(category,action);
  }

Bingo - WO variation views are tracked as events, simples! So, how does this look in GA? Head over to your reports, go into Content -> Event Tracking:



In the overview you'll see something like this:



Not that interesting...drill into actions though and you start to see real data treasure. Here's the ecommerce tab:



Row 0 refers to the original, row 1 is the search box in the header and row 2 is the newsletter signup.

We can see that AOV is down - no big deal - not statistically significant. We can also see that conversion rate and overall revenue are up and highly significant. So, the test confirmed more searches were happening but beyond that, we know that searchers convert 2-3 times better than non-searchers so we would hypothesise that revenue and conversion would increase with search usage. This data confirms the hypothesis and tells us by how much!

Now we can drop this data out and run it through SPSS (enterprise scale stats tool) to run more numbers through at our leisure to gain further insights: not something you can do with WO reports.

superSetVar

User defined variables are a really neat and powerful segmentation mechanism that we can use to provide a facility that allows segmentation of data based on partiular events firing. For many cases, segmentation based on events will not be useful or interesting but coupling event tracking with WO tests does make segmentation worth while. Say I wanted to see on-site search usage stats, ecommerce performance and site engagement data in one custom report based on a user seeing a certain test variation - now segmentation becomes useful. You don't have to use event tracking with user defined variables - I did in order to explore the technique. Say for example you've extracted the variation number as in the event tracking example. Now we can use superSetVar from Lunamaterics to set a user defined value using a similar naming convention to the event tracking:

    unSetVar('/featured-test','<%= ENV['ga_account'] %>');
    superSetVar('/featured-test='+variation,'<%= ENV['ga_account'] %>');

As you can see, we are a Rails shop - feel free to substitute the syntax of your choice to get the right uAcct value in place.

Right now, Website Optimiser has not suggested as clear winner for the featured products test. I wonder how things are shaping up?

If we were to head over to Visitors -> User Defined and take a look at the ecommerce tab we would see the following interesting report:



'featured-test=0' is the original where feature products are shown. 'featured-test=1' is the test where featured products are not shown.

Now, create a custom segment:



And apply this to the report then go look at the ecommerce tab:



You can probably see how you can now look at any report in GA to see how your tests in WO are affecting metrics across your site.

Now, I have to say that these techniques are still in development - if you see any potential issues, it would be great to get your feedback. Likewise, if you use any techniques here, do so at your own risk. When trying out new techniques, do so in safe profiles - not your main data repositories!

I implemented an event tracker and a superSetvar call in a rather dumb way and ended up destroying a days worth of bounce rate data! Hopefully you have been interested by this missive, perhaps even inspired but certainly warned!