When assessing performance, beware of vanity metrics

The digital world gives us access to more information than ever before. It also gives us new ways to gather that information.

In a commercial setting, businesses can use the tools emerging and evolving every day to find out more and more about their target audiences, as well as which strategies are working, and which aren’t yielding results.

Or at least, that’s the idea. But the truth is that there are obstacles. The utility of data to a business depends entirely on the kind of data being gathered and the business’s ability to understand what the data means.

In an ideal world, you work out what data you need, gather all the relevant data, recognize its implications, and are rewarded with an accurate idea of the success of a given campaign or strategy, for instance.

But approached in the wrong way, data can also be totally useless or— worse— misleading.

Not all metrics are created equal

Performance in one area will break down into different metrics— indicators which, in principle at least, accurately assess the efficacy of each part of that performance. ‘Time-to-fill’ would be a relevant metric for recruitment.

But not all metrics are created equal, and to determine not only if a strategy is useful or not, but where it is succeeding and where it’s failing, you need a range of metrics that together paint a vivid picture. Some of these metrics— those relating to engagement, conversions and click-throughs, for instance— will be more valuable than others.

But it isn’t always obvious, and there has been a proliferation of metrics that look good, but don’t actually mean what we think them to mean.

These are ‘vanity metrics’. They make everyone feel good about themselves while painting an inaccurate or incomplete picture of what’s actually going on.

In the world of business, more cynical types are constantly tempted to use vanity metrics knowingly to make those above them think they’ve been more effective than they have been. But this can’t go on for long. Ultimately, the ability to recognize vanity metrics may be the difference between delivering real value or failing completely.

And it seems not everyone is able to make the distinction between which metrics matter most and which merely look like they matter.

According to Gartner, well over half (60 percent) of all CMOs will cut their analytics departments in half over the next four years because of their failure to deliver on promised improvements, and we can safely assume that vanity metrics will have played a central role in this reconsideration of marketing analytics.

Read between the lines

Take social, for instance. You might record hundreds—even thousands—of followers or shares or likes. But a much more important data value is engagement, and whether that translates to click-throughs, conversions or sales.

Even a billion likes don’t matter a bit when they don’t lead anywhere. And what about content? A high number of backlinks and an impressive domain authority raising might look good. But these metrics tell you nothing about whether your content resonates with its audience, which will make or break your campaign.

Understanding only how many people we have reached is very much ‘vanity’, but helping the European CIC, for example, understand how their content is improving citizens’ relationships with food through engagement and impact data counts.

And then there are email reply rates. It doesn’t matter if every single recipient opens your email if no sales come about as a result. This is not to say these metrics don’t have some value, but they have value only in context, and only if the most important facts are plain to see.

Not all vanity metrics are easy to recognize. Marketing technology isn’t always intuitive, and you can be led to believe that the data you have is more illuminating than it really is. And this is compounded by the fact that marketers rate themselves as fairly low in ‘innovation maturity’ while the percentage of the marketing budget that goes towards technology is rising every year. It’s a perfect storm of high information and low understanding.

As is so often the case, one solution is to remember the fundamentals. This doesn’t mean discarding your data or firing your analytics team. But it does mean using your common sense and experience and looking at the impressive numbers on your computer screen with the time-tested principles of good marketing in mind.

If you do this, it’s clear to see what’s really useful information and what is merely giving you a false sense of success.