There are some common misconceptions about web user analytics terms. I’ve been thinking of this a lot lately because I was recently asked “Does installing Google Analytics automatically gives you all the KPIs that you need?” This question demonstrates a few areas needing clarification that have to do with semantics and inconsistent industry jargon- particularly around the use of the term KPI, or Key Performance Indicator. Let’s break it down:
Measures- Essentially anything we can count. Just because you can count something, doesn’t make it a good idea. You can count the number of times the letter E is on our homepage- it doesn’t make it valuable. It’s also important to remember that correlation doesn’t always imply causation.
Metrics- Measures that we track and tell us something important about our business. We can look for changes over time and this change tells a story. We need the right measures to be qualified as metrics. Not all metrics are created equal. Some can be really important for our business, others not as much.
Analytics- Measures and metrics that computers can track. Again, some are valuable, others are not. Google Analytics, for example, includes a lot of things that aren’t all that useful. Time on page and bounce rate, for example, mean almost nothing- at least by themselves. Let’s take bounce rate, the measure of how many user leave your application or website on a particular page. If someone finds all the information that they need on a page, why is it always a bad thing that users left only viewing one page? What it was only after possibly reading an article for 20 minutes and forwarding it to a friend and tweeting about it? They probably feel awesome about the information that they learned and can go about their business thankful that you helped them accomplish their task or answer their question. It doesn’t mean something sinister about the quality of the data or value of the page/content/publication. Time on page is another great example. If users spend several minutes searching for a particular resource, only not to find it, is that a good thing? Does that mean they are engaged with the tool or that it was impossible to use so much that it turned a 10 second task into a 4 minute chore. Some things you won’t get from analytics, you need to ask users directly to assess their frustration levels while accomplishing key tasks.
KPIs (Key Performance Indicators)- A class of metric that is a long term pre-indicator that is actionable. It is a leading indicator that speaks to business strategy, but now has been watered down to imply an “important metric for your business”. Let me break this down the way the industry and my personal UX “yoda”, Jared Spool, summarized for me recently. A real KPI would be when there is a sudden change (like when people stop reordering their Netflix queue) that something negative is about to happen (like a subscription cancellation). In this example, you have an opportunity for outreach to keep your current business, because the KPI tells you an unsubscription is about to happen. There can also be KPIs that can be a sign that something positive is going to happen. I’d love to take that term back to it’s original meaning because the crafting of a KPI can be extremely useful. That doesn’t make it easy. Getting to the level of actionable KPIs is an effort that is only achieved after having quality metrics in place so that you can determine KPIs and action plans for the future. Actionable KPIs often raise to this level after correlation of important business metrics are analyzed over a long period of time.
So where do we start?
We are going to start with asking the right questions. “What do we want to know about our users? How would this help us?” is a great place to start. For example, how about “What asset class are our pension clients most interested in?” Then, determine how this can be measured. Does reading publications indicate interest? What about types of managers searched for? How about topics they blog about or “like” on LinkedIn? Often an individual measurement or metric on its own isn’t enough. It is the change over time or comparison to another metric that tells a story about the data. The accumulation of some or all of these can be compiled into metrics. The importance of this metric is then determined by then influence it can have. We can create more content and/or collect more data because we know this about our clients.
What do you want to know about our users? How would this help you?
