Earlier this week, Avinash Kaushik did another one of his thought-provoking posts, “Excellent Analytics Tip #25: Decrapify Search, Social Compound Metrics.” In the post, Kaushik talks about the issues he has with “compound metrics.” He starts by giving an example of a compound metric in the digital marketing world…
Visit Quality = [(% of downloads + % of visits with more than 68 pageviews – % of visits with less 2 pages + % visits with Facebook shares + % of visits with store searches)/5)*1476]
Kaushik argues that compound metrics like this have little value since there is no clarity on the actions that need to be taken to improve the metric. For example, if you find out that your Visit Quality is 68, you probably have a gut feel as to whether that is good or bad based on historical trends. But if it is bad, you have no idea what to do to make it better since there is no transparency on why it is bad.
Although Kaushik would prefer to get rid of compound metrics altogether, he does acknowledge that it may be difficult to do since there is momentum behind them in some organizations, particularly for executives who want a simple metric that communicates the health of a particular area. So Kaushik proposes an approach that blends a compound metric with “focusing factors,” elements that contribute to the metric. Here’s an example he shows in his post (see diagram above)…
From our interactions with customers, we relate to both sides of this issue. We often see the requirement for compound metrics, as organizations want a simple way to communicate – particularly for executives – the performance of an area. A common scenario is the use of compound metrics to symbolize performance across marketing channels, geographic regions, or product lines.
At the same, we agree with Kaushik and feel that there needs to be transparency on the elements that drive the metric (what Kaushik calls the “focusing factors”). Without such transparency, there is no way to know what action needs to be taken in order to effect the metric. The compound metric cannot be a “black box.”
The bottom line is there’s inherent tension between metric simplicity, transparency and actionability. Compound metrics can help with simplicity and can be effective as long as they are designed with an eye towards transparency and actionability.
What do you think about compound metrics? Do you use them within your organization? Let us know your thoughts in the comments section below.