Throughout my years at Semetis, I have come across a lot of different tracking requests. Some were good, others excellent, but some did not really make sense. In this article, I am going to try and highlight what makes up a good metric and why you should or should not measure something.
With the rise of event based analytics - with tools such as Mixpanel or GA4 - I felt it was the right time to come back to the basics of such tools: a tracking plan. A tracking plan will and should be the central piece of your analytics tools for a couple of reasons:
- It should be the document that anyone can refer to internally, being a developper, a product manager or an analyst.
- It should be a comprehensive resume of your objectives and goals.
- It should keep on evolving together with your product, industry, company, etc.
A good tracking plan will be the gateway to data understanding, usage and actionability. So how can you make sure you select the right metrics? Some of the concepts I will cover come from the book Lean Analytics that I can only encourage you to read.
What is a good metric?
A good metric is comparative, understandable, a ratio/rate and actionable.
Being able to compare a metric across time or cohort is essential. Stating that your conversion rate is 12% higher than last week is more relevant than stating that your conversion rate is at 2,36% this week.
A good metric must also be understandable. A good habit to keep when creating or modifying metrics used is to involve different departments and profiles to make sure everything can be understood by anyone. A good way to make sure it is the case is also to ask a newcomer to go through the defined metrics and make sure everything is understood.
Tied to the first one, a ratio or a rate is implicitly comparative. When compared, ratios are easier to act on and make a better health check metric. When creating your tracking plan, think about the ratios you would like to analyse and the different metrics you’re gonna need to make such ratios.
Last and not least, a good metric should be a metric you want to act on. If you don’t think you’ll be able to act on the results given by that metric, maybe it’s not that relevant and you should put it aside. This is probably the most important tip to avoid tracking too much and being overloaded with data you don’t know what to do with. It is also the most difficult rule to apply as you might feel that you’re better off with more data than enough. You might also be under pressure from one or more people internally willing to track every single interaction with your website or app. Keep in mind that tracking too much might negatively impact the data usage across your organization as your data might become irrelevant, messy, out of date or confusing.
If you keep those 4 simple rules in mind when working on your tracking plan, you should already be off to a good start.
Some other elements to consider
Now that we have defined what a good metric is, there are some other elements you want to keep in mind when you choose your metrics.
Beware of vanity metrics. A typical vanity metric is “total signups’. As this number is only going to go up, it won’t help you gather relevant insights to act on. A better metric would be, for example, “percentage of active users”. Indeed, if you are improving your product, this number should go up and you’ll know you improved significantly.
The difference between a leading and a lagging metric. They are both useful metrics, but it’s important to know they serve different purposes. A lagging metric can be churn. It’s super important to be able to measure churn, but while it’s important, the damage is already done. You can still act on it, improve it and measure it again to analyse the effectiveness of your action, but you still lost those customers in the first place. Leading metrics, on the other hand, try to predict what’s going to happen. The current amount of prospects is a good example of a leading metric. If your number is low, you can expect a low number of sales and therefore act on it by trying to increase that amount of prospects.
Correlated versus causal metrics. Correlation is a measure of the extent to which two variables are related. If you take ice cream consumption and drowning over a year, you’d see they are correlated. The more people consume ice cream and the more drowning there is at the same time. Even if those two elements are correlated, reducing the global consumption of ice cream will not impact the amount of drownings. The causal effect in this scenario is the temperature. When the temperature rises, the consumption of ice cream increases. The amount of people swimming, and ultimately drowning increases as well. You prove causality by finding a correlation, then running experiments where you control the other variables and measure the difference.
The last element I would like to emphasize on regarding a good tracking plan is to make it a living thing. As your website, app, product and goals will keep on evolving over time, it is crucial that your tracking plan evolves with it. If you aren't constantly iterating on it, you might end up with what Brian Balfour refers to as the Data Wheel of Death.
Finally, having a good tracking plan and getting your metrics right will be crucial for your business, but that should not be the end of it. You will have access to a lot of valuable insights and it should help your entire organization become more data-driven, but you should not only stick to those quantitative data points. Indeed, those quantitative metrics won’t give you the full picture. One important aspect it does not cover is your customer feedback. Being online or offline, your customer feedback is an incredibly important source of information for your organization. Another important aspect to consider is that not every question might be answered based on the available data. Sometimes, it’s important to trust your gut feeling and encourage a test and learn approach into your organization. You don’t want to confine your employees too much and they should be able to think out of the box.
Grégoire le Hardy
| LinkedinThis email address is being protected from spambots. You need JavaScript enabled to view it.
