Friday, 20 November 2009

First/last click campaign attribution and onsite purchase trigger analysis techniques.

Overview
Understanding which traffic sources contribute to successful outcomes on your website is crucial to maximising Return On Investment. By default Google Analytics supports last click attribution. This means a customer who starts a session with a click on the Adwords Campaign 1 (see below) and then starts a new session with a click on Referrer 1 and ends in the purchase of four products will result in Referrer 1 being attributed with the 'credit' for all four sales.



See the Google Conversion University for more details.

We can add the utm_nooverride=1 parameter to the links in our campaigns to ensure the first campaign that is clicked is credited for sales and goal outcomes:



So, we can measure which marketing initiatives incentivise users to visit our sites but we can't so easily see what motivated a user to complete a goal or purchase while on the site – what is driving positive outcomes from user journeys during their visit? We can't easily see this with default techniques.

You can have your cake or eat it...but not both.

Moneyspyder has developed a technique that couples first or last click attribution data to measure brand engagement with last-click-before-purchase triggers on a 'per basket item' basis to reveal purchase or goal completion triggers.

We say you should have your cake, eat it and have extra sprinkles too!

Technique
We modified the Moneyspyder ecommerce engine such that the purchase trigger (last click before adding to basket/cart) is recorded in the Google Analytics clickstream data as well as being recorded against each order item in the basket such that the value can be used in the 'Category' field on the Google Analytics ecommerce tracking code.

Purchase triggers may include clicking on a feature product on the homepage, clicking on products in on-site search results, related product clicks, products in category listings, product clicks in email campaigns, organic search results and of course 'direct' visits from bookmarks. The limit here is your imagination!

Recording product clicks from on-site or external search engine results generates great purchase trigger data. Using the search term to supplement the data is a golden opportunity not to be missed. Likewise, related product clicks should record the product that was related to the purchased product and category list clicks should record the category.

This modification to our ecommerce engine was straightforward – it should simple be on your software too.

By now you will get a clear picture as to what extra data is being recorded. Now, what can you do with it?

Insights
First of all, let's take a look at the pure clickstream data:



From the standard content report in analytics we can see how clicks on the 'sky-lantern' product came from a multitude of different sources:


  • search for lights

  • email campaign

  • linked from other products

  • category links

  • direct



We can see unique page views required for conversion metrics, average time on page, bounce rate, exit % and the super insightful $index. If the scope of this technique stopped here, we'd be pretty happy already with the extra insight we have on customer journeys. The extra sprinkles arrive when we consider the magical 'outcomes'.

As described above, of vital importance to getting maximum value from this technique is to record the last click before 'add to basket' in the clickstream data AND the transaction 'category' data. These sets of related data enable amazingly fine grained conversion metrics to be retrieved. For example, we can dig into the search listed above for 'lights' (the bottom row in the table). That's a pretty handy $index! Looking deeper we can see this search generated some pretty handy revenue over a short period of time:



Applying classic analytical techniques yields further insight:



If you're not convinced as to the merits of this technique by now, and indeed, you can get a lot of this data from the onsite search report then we shall take a mighty leap and look at category conversion metrics for the 'Best Sellers' category based on the landing page.

In the Google Analytics report below (Ecommerce → Product Performance → Categories), it looks at first glance that using the homepage as a landing page work pretty well. Sure, lots of revenue but that's not all!



When we take unique pageviews into account we can see the conversion rate for the best Sellers category when the homepage is the landing page is a respectable 4.8%. However, the Sale and Festive Season (Christmas) categories both out perform the homepage as landing pages at 5.4% conversion and the Best Seller category page as a landing page converts at 5.6%.

It's worth bearing in mind at this point exactly what this data means. A customer entered the site on a particular landing page, found their way to the Best Sellers category and put a product in their basket that they bought – the Best Sellers category page was a trigger to purchase. They liked the page and the products so much that they bought – just what site owners want to see and hear.

Conclusion
Through simple modification to our ecommerce engine, Moneyspyder has revealed finer grained, segmentable insights into the aspects of customer journeys that trigger positive outcomes – goal completions or purchases that include first and last click attribution.

This functionality is a great facility for conversion professionals to identify optimal customer journey paths and focus optimisation efforts with greater accuracy and effectiveness.

The modifications required for this technique are entirely portable and are in no way specific to the Moneyspyder ecommerce platform – we really encourage you to explore this technique.

Moneyspyder Biog:
We are a Google Conversion Professional and develop and host state-of-the-art ecommerce solutions using Ruby on Rails. We continuously improve customer experience using web analytics, split-testing and regular site enhancements based on web analytics data.

Monday, 2 November 2009

Google Analytics: Expanded Mobile Tracking for Mobile Rails sites

Expanded Mobile Reporting - Introduction
Google recently announced expanded tracking for mobile apps using server side code. PHP, Perl, ASPX and JSP are supported as standard. As a dedicated 'Rails house' Moneyspyder has published a sample application for Mobile sites using Rails to broaden the usage of the product.

The need for this sample app is driven by the current use of javascript for tracking clickstream data. There are still a fair number of mobile devices out there that do not support client side javascript which can leave a sizable hole in your clickstream data. Using serverside code to reproduce the behaviour caters for these devices.

Preamble and cavets
The sample app is available for download. This is just a skeletal sample app and requires more thorough testing if deployed in the wild. We'd be keen to hear feedback of course as this is just a starting point....it ain't production ready!

Moneyspyder accepts no responsibility for loss or harm to data caused by using this sample app yadda yadda...

With that out of the way, some details.

Details

Download the app

Files

  • routes.rb

  • environment.rb

  • ga_helper.rb

  • ga_controller.rb

  • index.html



Two routes are used, only one is really required. The utm_gif action (served by the ga_controller.rb controller) is responsible for sending basic request info to the action and the action builds the remainder of the request details and sends them to ga, returning a 1px transparent gif.

The GA Account idientifer is saved in environment.rb. You might want to store it in a configuration yml file, the db or whatever your favourite place is...

The utm_gif_url helper method builds the utm_gif request. This is used in index.html for demonstration purposes.

ga_controller.rb rquires cgi, digest and open-uri. CGI is used for urlencoding strings. digest is used for the MD5 stuff. open-uri is used to send the request to Google analytics.

utm_gif is the only public method needed. response headers are set appropriately and send_data is used to return the transparent gif image. Requesting the gif through the private track_pageview method is where the magic happens.

In summary, all the usual name value pairs that are appended to the _utm.gif request to Google Analytics by ga.js are collected and then sent to Google Analytics using the private send_request_to_ga method. open-uri allows us to send this request programmatically. This might well be a nice plave to build in some error handling...

Once the request is done, the gif data is returned and the page is rendered - sweet.

You should be able to spin upthis little rails app easily enough using webrick on your local machine i the standard rails way and hit http://localhost:3000. (no support is offered by Moneyspyder for this setup - it's pretty standard stuff)

You'll see something like:



Gotcha and thoughts
If bots are hitting your mobile site or you use this tracking code technique insteaqd of ga.js (Hmmm?!) then you'll likely get a lot of traffic from bots - better filter the requests that get tracked.

Monday, 26 October 2009

Easier Website Optimiser tagging

Overview
In my previous posts I introduced the idea of externalised javascript libraries to abstract over Google Analytics page tracking and ecommerce transaction tracking.

This idea also transfers nicely into Google Website Optimiser where you can experience some interesting scripts on your first visit!

Caveat
These articles assume you are at least familiar with the concept of multivariate testing, javascript, Google Analytics tracking and the Live HTTP Headers plugin in FireFox.

As with the last posts - use these scripts at your own risk. Don't link to the script libraries directly, copy and tune to your own needs!

A Simple MV Test made simpler

A normal website optimiser multivariate (mv) test requires a control script, tracking script and goal script to be placed on your pages. The control script decides which experiment variation to show. The tracking script records which test variation was shown and the goal script records the experiment conversion.

Two cookies are also used in combination with the standard GA cookies.

Standard cookies
utma
utmb
utmc
utmv
.
.
.
etc (there are quite a few!)

Website Optimiser cookies
umtx
utmxx

The nature of the standard cookies is not the subject of this article. In summary, the utmx and utmxx cookies are used to remember which test variation a users sees.

If you inspect the cookies you have in your browser right now you probably won't have any utmx cookies for the moneyspyder.co.uk domain. If you go to the example page for this article and recheck your cookies, you should now see a utmx and utmxx cookie for the moneyspyder.co.uk domain.

You've just been tested!

You will have equal chances of seeing a very simple page with one of three different pieces of text and two links:

Original page
Test 1
Test 2

(and two links to goal and purchase)

Delete the utmx and utmxx cookies and reload the page to see different variations. This is standard Website Optimiser type development/testing/debugging.

You should also inspect the HTTP Headers output. Notice the request for siteopt.js? Notice the utmx and utmxx values? If you don't see these - it's broken. Simple. Check for javascript errors.

The source of the html page will reveal the how the test is rendered. As with the page tracking example we are including Moneyspyder javascript libraries:

<script src="/javascripts/ms_ga.js" type="text/javascript"></script>
<script src="/javascripts/ms_wo.js" type="text/javascript"></script>

ms_wo.js gives us three methods to use:

ms_wo_ctrl
ms_wo_tracking
ms_wo_goal

All three methods take the experiment id as a parameter 'k'. You can get this value from the supplied script snippets when you set up your test.

The control script normally sits at the top of your content - ms_wo_ctrl replaces this script and so, sits in the same place. Feed it the test id to setup the test. Directly beneath the ms_wo_ctrl script, call the ms_wo_tracking method. Feed it the test id and Website Optimiser ua account value to track the test view.

Now, normal test scripting applies for a moment. Wrap the content you are testing with your utmx section and noscript tags as normal. At Moneyspyder, we are considering implementing later versions of this script library with the ability to 'document.write' the utmx sections based on matching dom elements by id or class. We'd be interested to hear your thoughts on this. Less code is better, right?

So, assuming your test is set up you should now have a page that mirrors the format of http://moneyspyder.co.uk/ms_ga.html:


Viewing this page and inspecting the HTTP Headers output will reveal the following requests:

ga.js (Google analytics script library from Google)
siteopt.js (Website Optimiser script library from Google)
utm.gif requests

The request to siteopt.js will pass in values of your utmx and utmxx cookies if you have them so that you can see the same test variation you saw last time or, if you don't have the cookies, your test variation and utmx values will be set.

Having displayed the test variation content, your page will track the test view with a utm.gif request to the Website Optimiser UA account and the utmp value of the test id follwed by the string 'test':

utmp=%2F0102111525%2Ftest&utmac=UA-11-56683-1


The goal page follows a very similar format but just calls ms_wo_goal in the same way as ms_wo_tracking with the UA and k values:

ms_wo_goal(['UA-11056683-1'],0102111525);


And so the page is tracked as you will see by inspecting the HTTP Headers output for the utmp and utmac values.

Wrap up
So, we have seen how the javascript supplied by Google for page tracking, transaction tracking and website optimiser testing is totally 100% fit for purpose and great to use. But with a little thought we can afford ourselves a little more power, maintainability and ease of use. We have cleaner pages and can handle change a little better.

If these articles help you implement good quality Google product integrations then we have a success. If you want to take this work further for your own sites, great but remember it's good to share! If you want to challenge this work, feel free to leave comments or drop us a line at info@moneyspyder.co.uk - we'd be happy to exchange ideas!

Google analytics: easier transaction tracking

Overview
In my previous posts I introduced the idea of externalised javascript libraries to abstract over Google Analytics page tracking.

This idea also transfers nicely into Google Analytics transaction tracking where, it seems, most people seem to come unstuck.

We'll try and get you stuck back together!

Caveat
These articles assume you are at least familiar with the concept of javascript, Google Analytics tracking and the Live HTTP Headers plugin in FireFox.

As with the last posts - use these scripts at your own risk. Don't link to the script libraries directly, copy and tune to your own needs!

Simple transaction tracking made simpler

Take a peek at the transaction tracking example

Thanks for buying stuff! No, seriously, even though the page looks as unspectacular as the Page Tracking Example you have actually registered a (fake) transaction with Google Analytics for a t-shirt and an indoor frisbee.

Take a look at the page source. We're using ms_ga.js again. The method we are interested here is ms_ga_trans.

Notice the source does not track a page view! That'll be taken care of for us by the method.

ms_ga_trans takes three parameters: an array of UA accounts, a URL and a data structure representing the transaction.

The array of UA accounts enables us to track the transaction in any number of Google Analytics accounts. The URL is nullable so we can track the transaction page as a specific URI or just use the default. The transaction data structure mirrors the structure set out in the Google Analytics Transaction tracking documentation.

A transaction requires:

an order id
an affiliation
a shipping value
a tax value
a total value
a country, city and state

Clearly an order also requires one or more order items. Each item consists of:

the order id
a sku code
a name
optionally, a category (very useful)
a unit price (how much for one?!)
a quantity

Now, I'll stress this point about numerical values here. Do not include currency symbols. It is really important that when specifying a price or tax value or shipping value that you specify the value as a number to two decimal places. That is all. Not:

$45.00c

or

£345.567

or

[£9.99p]

or similar.

Just go for the simplest option which is (for example),

9.99

and stick it in quotes - everything is a string. Simple. No fuss. No complexity. If something is free, list it as 0. Don't miss out fields. You'll miss the wrong field and the script will fall over. With abstracted/externalised scripts and method calls you can build in more error handling and checking - handy!

So, inside the guts os ms_ga_trans we see a similar format to ms_ga_pagetrack where we iterate through the array of UA account values to track against:

ms_ga_trans(['UA-7862117-1'],'skeleton/thankyou',
{"order_id": "666666", "shipping": "5.95", "web": "moneyspyder.co.uk", "tax": "1.79", "total": "21.74", "country": "UK", "items": [
{"price": "6.00", "category": null, "name": "t-shirt", "sku": "t-1", "quantity": 1},
{"price": "8.00", "category": "home", "name": "indoor frisbee", "sku": "f-1", "quantity": 1}
]
}
);

We see that the page view is tracked here to - hence, we do not track the page separately. It is important to track the page view before recording the transaction.

The transaction is then added using the ga.js addTrans method. Each item in the transaction is then added and finally the transaction is tracked.

So, there isn't that much that is simplified here other than the order in which the steps happen and the opportunity to build in more error checking and handling. If you've got these basics right - well done. You want to know the transaction has registered though, right? Real time? Right, thought so.

Debugging/Checking/Testing
Reload the page with our new best friend Live HTTP Headers looking over our shoulder so we can review what happened under the hood.

First of all, no surprises, ga.js loaded.

Then we track the pageview with utm.gif

Take note of the third section in the HTTP Headers output - there is a new utm value called utmtid and it has our transaction id, '666666'. Cool - transaction being tracked with expected values! Okay, so we can see the total in utmtto, the tax in utmtx, shipping in utmtsp etc.

Each follwing section details each order item. utmut=item, utmipn for the item name, utmiqt for the quantity. Real self explanatroy stuff when you know where to look and what to look for.

Never have a transaction fail to track ever again!

Thats should do for this article - next time we Optimiser our website with Google Website Optimiser - but simpler. ;-)

Google Analytics: easier page tracking

Overview
Google Analytics is great. It's free and super powerful but... there is always a but!

Sometimes the tracking tags can be tough to understand. They can be tricky to implement if your CMS doesn't directly support them. Debugging can be 'gnarly'. Mix in Google Website Optimiser tests and you might end up biting of more than you can chew.

Moneyspyder has written two small skeletal javascript libraries that you can copy and use to make Google Analytics page tracking, transaction tracking and Website Optimiser implementation a bit easier.

We're trying to lower the bar to entry. We're not saying YOU MUST DO IT THIS WAY but we think this is one good way to start doing simple tracking and testing and we want to share some techniques with you. We're not saying the existing Google code is wrong or bad but if you want slightly cleaner pages and to get some extra value by abstracting out javascript code then this might be useful and it might even make managing your site easier.

The next few articles will cover smarter:

Page Tracking
Transaction Tracking
Website Optimiser tagging
All three together

Caveat
Feel free to take a copy of the scripts used in this article. Don't link to them - they will change over time and your tracking might break. Moneyspyder accepts no responsibility for harm to or loss of data if these scripts are used 'as is' or linked directly from the Moneyspyder server. You have been warned - so now let's inspire!

Simple smarter page tracking

Take a look at our first page - Page tracking example

Spectacular it is not...at least at face value. View the source to see the interesting bits.

1. In the
head
section you will notice a javascript include

<script src="/javascripts/ms_ga.js" type="text/javascript"></script>

Take a look at the ms_ga.js source. The section that we are interested in is:

ms_ga_pagetrack(['ua-1','UA-7862117-1'],'/skeleton/tracking');

ms_ga_pagetrack(['ua-1']);

2. In the html, notice we are making two calls to ms_ga_pagetrack
The parameters that are expected are:

  • An array of one or more strings - 'UA-XXXXXXX-X'

  • An optional string to use as the page URI


The first parameter as an array enables you to track pageviews in multiple accounts without having to repeat the tracking code. The Google tracking method
pageTracker._trackPageview();
can take a parameter to replace the tracked page url.

Simple.

Additionally, if the Google ga.js javascript (notice the first two lines in ms_ga.js?) changes, you only change the ms_ga.js file and all your site pages remain untouched. This extra 'abstraction' layer has been helpful to Moneyspyder in the past.

Debugging
Debugging Google Analytics page tracking can be a lengthy and frustrating process:

  • Try a tracking script

  • Hit a page (along with everyone else who uses your site

  • See if the page view appears in the GA reports

  • Fail? Sigh...Rinse and repeat


We prefer to do this in real time. Yup, results while you wait. Go and get the Live HTTP Headers Firefox plugin. Start it up (Tools menu in FireFox). Go to the 'Config' tab and click the 'filter URLs with regexp' checkbox. Then put this string in the corresponding text box:
google-analytics.com

So now the only values you will see in the HTTP headers output will be related to Google Analytics.

Reload http://moneyspyder.co.uk/ms_page_tracking.html. The HTTP Headers output will contain 4 sections of similar format. The first section is the page including ga.js from Google:

http://www.google-analytics.com/ga.js

GET /ga.js HTTP/1.1
Host: www.google-analytics.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 GTB5 (.NET CLR 3.5.30729)
Accept: */*
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://moneyspyder.co.uk/ms_page_tracking.html
If-Modified-Since: Tue, 13 Oct 2009 18:39:01 GMT
Cache-Control: max-age=0

HTTP/1.x 304 Not Modified
Last-Modified: Tue, 13 Oct 2009 18:39:01 GMT
Date: Mon, 26 Oct 2009 12:14:29 GMT
Vary: Accept-Encoding
Cache-Control: max-age=604800, public
Server: Golfe
X-XSS-Protection: 0
----------------------------------------------------------

This output tells me first of all, the nature of the request being made - we're asking for ga.js from Google. There is some http header info about the browser I am using, an HTTP response code and not a lot else of anything of interest.
The next section gets more interesting:

http://www.google-analytics.com/__utm.gif?

This request won't appear in the HTML source of the page. This is the output from _trackPageview. The data appended to this gif request is how Google Analytics gets the data into Google Analytics. Some interesting values here:

utmp=%2Fskeleton%2Ftracking
utmac=ua-1

utmp is the page path - remember we set this using the url parameter in our own method call? This is where you see the page path that is being sent to GA. utmac is the GA account being used for reporting. We set two accounts to be used - 'ua-1' and 'UA-7862117-1'.

So, the first request we saw in the HTTP Headers output was the request for ga.js:
http://www.google-analytics.com/ga.js

Second, we reported the pageview using /skeleton/tracking and ua-1:
utmp=%2Fskeleton%2Ftracking&utmac=ua-1
ms_ga_pagetrack(['ua-1','UA-7862117-1'],'/skeleton/tracking');

Third, we reported the pageview using /skeleton/tracking and UA-7862117-1:
utmp=%2Fskeleton%2Ftracking&utmac=UA-7862117-1
ms_ga_pagetrack(['ua-1','UA-7862117-1'],'/skeleton/tracking');

Fourth, we reported the pageview using /ms_page_tracking and ua-1:
utmp=%2Fms_page_tracking.html&utmac=ua-1
ms_ga_pagetrack(['ua-1']);

There are quite a few name value pairs appended to the utm.gif. More than should be discussed right now. We'll use this same debugging technique for transactions and website optimiser tracking and tagging.

Next, we'll do some testing!