Analytics in Real Time - What IS A.R.T.?

Web Analytics in Real Time delivering functionality and value.

It is widely known that the percentage of websites using Google's or Yahoo's free Web Analytics is growing massively. With this explosive growth, the methodology of the typical analytics ninja has to change at an equally frenetic rate.

Eric Peterson blogged recently (The Coming Bifurcation in Web Analytics Tools) - Google Analytics alone is simply not enough for truly sophisticated web analytics.

Indeed, any analytics ninja worth their money employs a number of tools to deliver actionable insight and therefore value to businesses.

This blog post is going to describe a new offering from Moneyspyder that is being used in conjunction with Google Analytics (equally suitable for use with a whole suite of analytical tools) to deliver high value actionable insight but in real time!

Gasps of shock and horror! Is real time worth it? Is real time action possible from real time insight? Back in 2006, Avinash Kaushik blogged (Is Real-Time Really Relevant?):

The greatest gift the web gives you is the ability to fail faster.

So how fast is fast? Now? How does now sound? You can Tweet now...send 10,000 emails now....up your bid on a range of keywords now. Don't you need to measure and understand the performance of these measures now? Consider Avinash's take on real time insight actionability from 2006:

...is getting real-time data really relevant? Do you really need it?

Fair question but as mentioned above, given that we're marketing to our customers and our customers are responding to emails and searching in real time, real time actionability is now strongly relevant and necessary. The act of using websites isn't after all an asynchronous activity!

The FUD (Fear, Uncertainty & Doubt) issues can be summarised:
- Do we need more data?
- Is actionable insight delivered in real time or are we just setting up another
useless reporting stream?
- Our current choice of analytics solution doesn't do real-time so what is the business case for changing to a potentially inferior real-time capable solution?
- Surely more powerful resource is required to provide the real-time capability?
- Don't we need more business processes to handle real-time action?
- A culture of reporting rather than analysis is going to be created!

Thus the gnashing of teeth seeks to extinguish the real time flame. In response:

Current free analytics solutions are not geared up to deliver actionable insights at the same velocity as our marketing efforts. That is not to say throw your current solution in the trash – far from it. Using the right tool for the right job is why we are using a suite of tools already:

- Analysis of click stream data
- Competitive Intelligence
- Voice of customer

None of which are geared to enable you to respond as quickly as you should be able to so another weapon is required in our armoury. One that is easy to use. One that is benign, safe, secure, scalable, value for money, fits with our current processes but above all, delivers actionable insights not just reports!
So, enter A.R.T...Analytics in Real Time.

A.R.T. makes no apologies for it's operation being based on existing great engineering ideas. These simple building blocks deliver a hugely functional product that is an utter no-brainer to use. The out-of-the-box setup requires a javascript include line in the page header and nothing else. It is benign in operation. It won't mess with your pages or affect your site speed.

Outcome-centric
A.R.T. is outcome-centric. The insights stem from the focus on measuring your website's ability to do what you wanted it to

- measure sign ups
- basket/cart actions
- searches
- posting comments
- form completion
- scrolling or any other dom event
- all things selling!
- Sales AND revenue

A.R.T. is not just any old reporting tool spouting vanity metrics. Real value stems from seeing a return on your investment. A.R.T. is the frontline tool that enables you tune finely tune your business to respond to realtime demands. A.R.T. doesn't try to replace your current analytics solution, although it is a great companion. We are quite strict on the amount of data that is held and for how long. The insights that A.R.T. yields can be extracted from your current analytics tools so A.R.T. will only ever hold a rolling 24 hour window of your clickstream and outcome data. Need more? Go to Yahoo/Omniture/Google/other weapon of choice.

Flexible & Customisable
Flexibility and customisability stem from the small number of moving parts in the system. A.R.T can be as sophisticated or vanilla as you need or want. Your challenge is to engage A.R.T. system in valuable insight revealing metric gathering endeavours.

Scalable infrastructure
A.R.T. is built using Ruby on Rails and is hosted on highly scalable cloud based infrastructure.

Processes to support change and build a test driven culture
Marketing in realtime clearly requires processes to support realtime action based on realtime results. Clearly, given we have processes in place in our organisations that enable marketing in realtime, we have the ability to act in realtime already! The necessary change that allows action and capitalisation on realtime insight is a cultural shift towards test driven optimisation. Contemporary software development methodology promotes rapid action and optimisation based on measured results – think Agile. A.R.T. Provides a feedback loop that enables rapid optimisation in a similar fashion. Consider firing out 3 variations of an email – 3 small batches at the time of known peak traffic levels. A small, low cost, low risk (due to small numbers) test like this would generate real-time actionable insight into which version of the email works best in terms of the desired outcome. Obviously, statistical significance is required before committing to a more risky or expensive course of action but this caveat exists for the current marketing/analytics process. So, having engaged in a cycle of 'release and refactor' we have what looks like a process that could deliver optimised value from future campaigns.

Rinse and repeat. Thanks A.R.T. ;-)

A summary of A.R.T.:

What does it do?
• Visit, visitor and pageview metrics - Who, What, Where, When and How?
• Outcome metrics with hourly breakdown - What is really important and really happening?
• Definable - YOU decide what your site is supposed to do and measure it!
• Revenue metrics with hourly breakdown
• 24 hour rolling window of data - need more? Go to your current analytics provider
• Traffic source data - Measure marketing in Real Time!
◦ campaign awareness through utm tagging
◦ Search engine metrics & keywords
◦ Referring sites - who loves you baby?!
• Individual customer identification, technical, loyalty & recency data - Love your top customers - personally
• Clickstream capture, analysis and replay
• Flexible page event definition – pageview, scroll, add to basket
• Filterable data – exclude/hide internal clickstream data
• Mobile interface – iPhone and Android - Real-Time and on-the-go!

Who is it for?
• Marketeers
• C level execs
• Technicians
• Content managers
• Business Owners

What does it cost?
The basic unit of currency for A.R.T. is the page view, hence, the fairest, most scalable pricing model is based on the number of pageviews per month and the length of the contract.

There is no limit on the number of pageviews with A.R.T. The infrastructure and costs scale according to your usage.

There is no advertising in the pricing model. The focus is on delivering insight, not just data or reports but real actionable insight to help you improve your business.

Moneyspyder exceeds targets in 2009.

Happy New Year!

Last year was certainly a good one - 2010 is set to be even better. It'll be Spring soon - warmer weather, more daylight and more records to break.

Moneyspyder exceeded many targets last year. We were delighted with the resilience and scalability of all our clients' sites during the festive period (as were our clients!). The run up to Christmas saw new ground broken in terms of scalability, performance and transactional throughput.

Overall, 2009 saw 99.96% uptime across all our clients. Bearing in mind that the majority of that miserly amount of downtime was planned. Our twice monthly scheduled upgrades of sites are timed and performed with precision so as to maximise effect and return whilst minising interruption and downtime. We don't miss opportunities to introduce new A/B and Multivariate tests with Google Website Optimiser based on our deep dive analytics.

It has to be said that our infrastructure partners played a huge part in our success during 2009. A 'big shout out' to Site Confidence for the monitoring and uptime reporting and especially Engine Yard for the AWESOME Rails hosting. We're delighted to be part of Engine Yard's Select Partner Programme and look forward to moving onwards an upwards with all our client's and partners in 2010!

Google Analytics report bookmarking hacks

Help HiPPOs!
Even though Moneyspyder is firmly behind the anti-HiPPO movement we recognise it is still important to help HiPPOs. After all, they do rule the business world.

Generally speaking it is best to deliver insight to HiPPOs. Make the news - don't just deliver the news! However, there is merit (on occasion) to furnish them with 'vanity metrics' or 'outcome proxies' as we tend to think of them via the medium of 'the dashboard'.

Better Dashboards
The dashboard in the context of Google Analytics will likely take the form of a custom report. Custom reports rock. F.A.C.T. A custom report based dashboard can lift the value of the deliverable. You can move from report 'puking' to actually delivering insight by placing the report in context. You can do this in a few simple ways:

• Multi-tabbed


• Use advanced segments


• Use relevant date ranges


• Use comparison date ranges

Express delivery!
Okay, you have the dashboard setup - it offers context as well as just raw numbers. It yields actionable insight (so go do some insightful actions already!).

You'll probably want to schedule the delivery of this report via a monthly PDF attachment in an email. Simple in Google Analytics.

I suggest dropping a note in with the email via the description:



The note should start delivering context for the HiPPO. Help them already before they open the attachment! Now, the real meat of this post. Provide a link to the report.

Scary controversial opinion alert!

Invite the HiPPO into Google Analytics...Give them a specific read-only login to one profile that contains the dashboard/custom report.

Use the options in the report URL to customise and control what they see.

Here is a standard custom report link:

https://www.google.com/analytics/reporting/setup_email?id=4867638&seg0=-1&pdr=20091201-20091216&cmp=date_range&trows=50&gdfmt=nth_day&rpt=CustomReport&segkey=medium&tchcol=1&tst=0&tscol=v0&tsdir=0&mdet=WORLD&midx=0&gidx=0&cid=26&afs=false&seg=1&fmt=0

Here is the link with some easily customisable options in the URI:

https://www.google.com/analytics/reporting/custom?id=profile id&pdr=primary date range&cmp=advanced segments&trows=50&gdfmt=nth_day&cdr=comparison date range&segsegment number=-segment number&rpt=CustomReport&segkey=medium&tab=tab number&tchcol=1&tst=0&tscol=v0&tsdir=0&mdet=WORLD&midx=0&gidx=0&cid=Custom report id&afs=false

So, what are the customisable options? this is not an exhaustive list - these are just the ones I find useful right now - this list may grow.

Custom report id
Open your custom report. Write down the id in the url. Use it.

Primary date range
This is the date range that you want to look at. It's optional. If absent, the report will show the default last 30 days. It is in the format: yyyymmdd-yyyymmdd (eg. 20091101-20091116) where the first date must obviously be before the last... ;-)

Comparison date range
This is the date range to compare with - great context. Look at 'the same period last week/month'

Advanced Segments
How cool! Load a report showing only the segments you want to see! The format for the NVP is segn=-m. So, the default will show 'all users' would be ....&seg0=-1&...the default advanced segments follow the order in the drop down list so 'Non-bounce visits; would be seg0=-12. You can show multiple segments in the format: ...&seg0=-3&seg1=-5&seg2=-12&...which would show 'Direct Traffic','Visits with Conversions' and 'Non-bounce Visits'.

Tab number
Have you got a multi tab report? Do you want to default to a tab other than the first? Specify the default tab number here. Simples ;-)

Statistical significance in A/B testing - a little tool to help

Rationale

I read a tweet recently by @tclaiborne about a great blog post on the subject of Easy Statistics For Adwords AB Testing And Hamsters. With a title like that, how could I not take a peek?

It so happened that I was working on a small project to build a tool in Javascript to enable easy analysis of two data sets to compare them for statisticaly significant differences, specifically in the context of A/B and MV Testing.

This post is introducing the prototype of that tool. Just to be clear, this tool is a mash up of Javascript snippets that have been published. I didn't write the whole thing so I'm not taking credit here - I'm just looking to share a cool tool!

A Simple Test to Introduce The Tool

Let's say we've run a test using Google Website Optimiser. We made a change to a page to increase the number of outcomes. We have 6 days worth of data. Here are the conversion rations for the 6 days for the original and the test variation:

Test	1	2	3	4	5	6
Original	6%	6%	5%	6%	7%	6%
Test Page	9%	6%	7%	6%	9%	8%

So, from 5 days worth of data, can we see if the difference in the conversion rates are significant? It's a small data set...the numbers seem to be different but as the blog post referred to earlier says, we humans are really bad at looking at data sets and making accurate judgements.

We need some stats. Enter jsstat.



So, we can drop in our two samples of data as comma separated values. They don't have to be the same size or integer values. Let's hit that 'oh-so-tempting' import button to see what wonders we can find:



Ah, such insights, knowledge and power are ours! We can deliver meaning and value to our clients! Ahem, enough whimsy - what the heck does this mean?

I'm going to keep this high level:

• The differences could have happened by chance.


• The green text tells us the truth


• The results are conclusive.


• The new page converts 1.5% better than the old on average

Try the test yourself using '1,2,3,4,5,6,7,8,9' as both data sets. NOT SIGNIFICANT!

Moving swiftly on

It's a prototype okay? It might not work in crufty old browsers. Stick with a later version Chrome or FireFox to be safe. The graphing is adding little value right now but box plots are coming!

I'd really like a direct export from Google analytics or Website Optimiser into something like this...Hmmm.

Now, the new Google Analytics Intelligence functionality is very similar to this. It's great, don't get me wrong! Different in some ways but based on the same theory...mostly.

We are looking at taking this tool a lot further to supplement Multi-Variate testing results analysis and click stream data analysis.

I'll keep you posted.

First/last click campaign attribution and onsite purchase trigger analysis techniques.

Overview
Understanding which traffic sources contribute to successful outcomes on your website is crucial to maximising Return On Investment. By default Google Analytics supports last click attribution. This means a customer who starts a session with a click on the Adwords Campaign 1 (see below) and then starts a new session with a click on Referrer 1 and ends in the purchase of four products will result in Referrer 1 being attributed with the 'credit' for all four sales.



See the Google Conversion University for more details.

We can add the utm_nooverride=1 parameter to the links in our campaigns to ensure the first campaign that is clicked is credited for sales and goal outcomes:



So, we can measure which marketing initiatives incentivise users to visit our sites but we can't so easily see what motivated a user to complete a goal or purchase while on the site – what is driving positive outcomes from user journeys during their visit? We can't easily see this with default techniques.

You can have your cake or eat it...but not both.

Moneyspyder has developed a technique that couples first or last click attribution data to measure brand engagement with last-click-before-purchase triggers on a 'per basket item' basis to reveal purchase or goal completion triggers.

We say you should have your cake, eat it and have extra sprinkles too!

Technique
We modified the Moneyspyder ecommerce engine such that the purchase trigger (last click before adding to basket/cart) is recorded in the Google Analytics clickstream data as well as being recorded against each order item in the basket such that the value can be used in the 'Category' field on the Google Analytics ecommerce tracking code.

Purchase triggers may include clicking on a feature product on the homepage, clicking on products in on-site search results, related product clicks, products in category listings, product clicks in email campaigns, organic search results and of course 'direct' visits from bookmarks. The limit here is your imagination!

Recording product clicks from on-site or external search engine results generates great purchase trigger data. Using the search term to supplement the data is a golden opportunity not to be missed. Likewise, related product clicks should record the product that was related to the purchased product and category list clicks should record the category.

This modification to our ecommerce engine was straightforward – it should simple be on your software too.

By now you will get a clear picture as to what extra data is being recorded. Now, what can you do with it?

Insights
First of all, let's take a look at the pure clickstream data:



From the standard content report in analytics we can see how clicks on the 'sky-lantern' product came from a multitude of different sources:

• search for lights


• email campaign


• linked from other products


• category links


• direct

We can see unique page views required for conversion metrics, average time on page, bounce rate, exit % and the super insightful $index. If the scope of this technique stopped here, we'd be pretty happy already with the extra insight we have on customer journeys. The extra sprinkles arrive when we consider the magical 'outcomes'.

As described above, of vital importance to getting maximum value from this technique is to record the last click before 'add to basket' in the clickstream data AND the transaction 'category' data. These sets of related data enable amazingly fine grained conversion metrics to be retrieved. For example, we can dig into the search listed above for 'lights' (the bottom row in the table). That's a pretty handy $index! Looking deeper we can see this search generated some pretty handy revenue over a short period of time:



Applying classic analytical techniques yields further insight:



If you're not convinced as to the merits of this technique by now, and indeed, you can get a lot of this data from the onsite search report then we shall take a mighty leap and look at category conversion metrics for the 'Best Sellers' category based on the landing page.

In the Google Analytics report below (Ecommerce → Product Performance → Categories), it looks at first glance that using the homepage as a landing page work pretty well. Sure, lots of revenue but that's not all!



When we take unique pageviews into account we can see the conversion rate for the best Sellers category when the homepage is the landing page is a respectable 4.8%. However, the Sale and Festive Season (Christmas) categories both out perform the homepage as landing pages at 5.4% conversion and the Best Seller category page as a landing page converts at 5.6%.

It's worth bearing in mind at this point exactly what this data means. A customer entered the site on a particular landing page, found their way to the Best Sellers category and put a product in their basket that they bought – the Best Sellers category page was a trigger to purchase. They liked the page and the products so much that they bought – just what site owners want to see and hear.

Conclusion
Through simple modification to our ecommerce engine, Moneyspyder has revealed finer grained, segmentable insights into the aspects of customer journeys that trigger positive outcomes – goal completions or purchases that include first and last click attribution.

This functionality is a great facility for conversion professionals to identify optimal customer journey paths and focus optimisation efforts with greater accuracy and effectiveness.

The modifications required for this technique are entirely portable and are in no way specific to the Moneyspyder ecommerce platform – we really encourage you to explore this technique.

Moneyspyder Biog:
We are a Google Conversion Professional and develop and host state-of-the-art ecommerce solutions using Ruby on Rails. We continuously improve customer experience using web analytics, split-testing and regular site enhancements based on web analytics data.