Say for example, we would like to compare the performance of three different marketing campaigns against one another, by which I mean organic, ppc, online banner advertising, we would want to analyse the conversion rate of the campaigns over time.

First of all, we would take the average conversion rate on a daily basis, by average I mean the central tendency for our data to centre around a particular value, be that mode, median or mode. And then compare these average conversion rates over time for each individual campaign against one another.

However, when we are looking at how the data is dispersed, we may see some points that either stick out too high or too low. So how can we tell whether these are significant data points. If we look at all of the raw data that makes up the averages, it may be that there is one number that is so off the charts that it is completely skewing the whole average – these are called outliers.

It is important to get rid of outliers, otherwise you would have no idea whether or not one campaign is performing better than others.

I think of web analytics like a puzzle. You need to get all the bits together and understand their context and importance in order to get a big picture overview.
 
For example, today if we look at the key conversion rate for your website (whatever that might be). Everytime there is a fluctuation in the conversion rate, upper management will be querying the reasons why and next actions which can begin to become unhelpful and also waste a lot of the web analyst’s time. And usually the more senior they are, the more suggestive (dictatorial) they get.
 
The thing is that a conversion rate, like any other rate, will always naturally fluctuate over time. So what is meaningful and what isn’t? This is where we call statistics into the web analyst’s arsenal to enable us to make more informed conclusions and decisions.
 
For example, here is my fluctuating conversion rate for a website – without any use of statistics. There appear to be a number of worrying/exciting fluctuations. Where do I start?

 Click the chart to make it bigger and see how little help it is.

This is not about needing to become a qualified statistician but knowing enough about statistics to be able use to them effectively in one’s web analytics data and have an intelligent conversation with a statistician, “statistics without tears” in other words. Don’t look up standard deviation in wikipedia or speak to a professional dataminer/statistician and begin to feel statistically challenged/mathematically innumerate, please.
 

My introductory guide to statistics without tears
Here is my guide to statistics without tears to enable you to make the informed decisions you need to make with your web analytics data.
 
In statistics, here are some things that help me to make more informed decisions:
1. the mean – or the average or central location of my data
2. the trend – when we look at data over time, we apply statistics to give us a trend
3. the standard deviation – describes the spread on either side of the mean
4. upper and lower control limits – setting limits on my data to a specified standard deviation to see if anything stands out or is statistically significant
5. statistical significance – a result is called significant in statistics if it is unlikely that it happened by chance.
6. probability modelling – to determine how long until something happens, how many goal conversions will we see over a given period of time and given an opportunity to do something, how many people will choose to do it.
 
Conversion rate analysis using control limits, standard deviation and a trend

Here is my analysis of my conversion rate with the use of standard deviation, upper and lower control limits and a trend line. The brilliant thing about this is that it took me less than 5 minutes to do in excel.

 Click to make this chart bigger and look at the datapoints that stand out.

Instantly we can see numbers that are standing out on either the top end or bottom end of the control limits, which are set to a specified standard deviation.

To do this in excel have two columns, one with the daily conversion rate and the other with each days date. Then insert a line chart which shows the conversion rate over time. Right click on the line in the chart and choose format data series. Choose “error bar”, “display both” and “standard deviation” and specify your standard deviation, for example I specified 1.7 and set values for those data points that stand out. Then add a trend line to your data, again by using excel. This error bar gives us an upper and lower control limit based on a standard deviation from the mean that helps us to see the data that stands out and has a statistically higher chance of being significant/meaningful. The trend line (which I set to be linear) gives a good idea of where our data is going, which we can’t get merely by looking at the data itself.

This is the equation that excel that uses to work out the standard deviation (which is why it is great when excel can do the job for us):

 Where
s = series number
i = point number in series s
m = number of series for point y in chart
n = number of points in each series
yis = data value of series s and the ith point
ny = total number of data values in all series
M = arithmetic mean
 

To conclude:

The result is we can instantly see which data points are standing out from the average conversion rate and correlate these backwards to company activity. For example, was there additional marketing activity such as SEO or direct marketing. Or were changes to the actual website design or copy primarily responsible? Were there periods where we would have expected the conversion rate to stand out, but it underperformed? What could the underlying reasons be? What action can we take as a result of this analysis – for example to examine all marketing activity and changes to the site during a period of a higher than normal conversion rate to see what was contributing to this success and to apply this to the future. And similarly, to see if any company activity can explain the lower than average conversion rate and highlight these as problem areas.

In my next post on statistics, I will be using probability to make projections about company performance that can really help with one’s marketing.

As usual, I very much welcome your feedback, agreement or complete disagreement and most importantly THANKS FOR READING!!