Customer Experience: Measure What People Do, Not What They Say

In observing thousands of customers over a 10-year period as they try to complete tasks online, I have noticed a very significant gap between what people say and what they do.

Firstly, people are generally not good at predicting their own behavior.  Time and time again, I’ve had people say they do A, B, and C online connected with a specific task, only to observe them do X, Y, and Z.  Secondly, people have failed miserably at a whole range of tasks on a website or app, only to say they were very satisfied with their experience.

These findings lead to the conclusion that it’s essential to observe online behavior if you want to find out about the true experience customers are having.

Millions of customers go online every day to complete tasks.  Online is a much more active medium than television or print.  For example, with Search, the customer, not the brand, is the broadcaster.  The customer is active – the brand, reactive.  The customer is creating an advertisement as they type: “cheap flights Dublin”.  The brand needs to listen and respond.  So, online we really need to know: Are customers completing their tasks?

Speed Is The Essence Of The Online Experience

Time is everything online, and this is particularly true for mobile.  Therefore, if we want to measure what customers actually experience online, we must measure the time they spend.

“Subconsciously, you don’t like to wait,” Arvind Jain, a Google speed engineer told The New York Times in 2012.  “Every millisecond matters.”  Google found that if your pages are 250 milliseconds (a quarter of a second) slower than your competitors, you will lose customers to those competitors.  “Two hundred fifty milliseconds, either slower or faster, is close to the magic number now for competitive advantage on the Web,” Harry Shum, a computer scientist and speed specialist at Microsoft, told the Times.  A quarter of a second is not a lot of time.

Fast-downloading pages are critical to online success:

  1. Every second faster made its pages load, saw a 2% lift in conversions.
  2. Firefox reduced page load times by 2.2 seconds and saw 10 million extra downloads as a result.
  3. The Financial Times found that a 1-second delay in page downloads caused a 4.9% drop in the number of articles read.
  4. The 2012 Obama campaign made their website 60% faster, and this resulted in a 14% increase in donation conversions.

However, as important as page download speed is, it is just the tip of the iceberg when it comes to the overall time involved in completing a task online.  It is estimated that page download speed accounts for between 10% and 20% of overall task time.  The vast majority of the time a typical customer spends online is taken up by scanning, reading, clicking, typing, selecting.  In other words, much more time is taken up using the page than it waiting for it to download.

Thus, to truly understand customer time, you must measure the entire effort involved in completing a task.

Identify Customer Top Tasks

To create the best possible customer experience online you need to do four things:

  1. Clearly identify the top tasks customers are seeking to complete.
  2. Observe whether, in fact, they are successful in completing these top tasks.
  3. Of those who are successful, measure how long it is taking them.
  4. Use the insights gained from these observations to relentlessly focus on maximizing task completion and minimizing time-on-task.

Having analyzed the task preferences of more 300,000 customers in more than 30 languages and countries in a wide variety of sectors, my team has found a constant pattern.  Typically, a task environment, say for dealing with your health, buying a car, deciding on a university, etc., will have between 60 and 100 tasks.

What do I mean by a task?  In relation to your health, a task would be checking systems or getting treatment.  For buying a car, it would be getting a price or comparing brands.  For choosing a university, it would be finding a specific course or getting information on fees.

customer-experience-measure-what-people-do-not-what-they-say-1We have found a constantly repeated pattern when we do task identification research (fig. 1).  When customers identify what is most important to them in a particular task environment with 100 tasks, for example, 15 tasks (top and medium) will get the first 50% of the vote.

The top 5 tasks will get as much of the vote as the bottom 50.  This pattern has occurred like clockwork in over 400 completed top task surveys.

If you truly want to understand customer effort and experience, you must measure the performance of their top and medium tasks.

Measure Top Task Performance

If you carefully observe between 15-20 customers as they seek to complete their top tasks, you will get reliable management metrics for task success rates and time-on-task.

customer-experience-measure-what-people-do-not-what-they-say-2Over the years we have observed thousands of people seek to complete tasks online.  We found that once you carefully observe between 13-18 people, you begin to get stable and reliable task completion metrics (fig. 2).  Observation should occur remotely, using tools such as GoToMeeting or Webex, in a carefully controlled and professionally moderated manner.

While reducing time is how you win online, task failure is where you most definitely lose.  You would be surprised how many top customer tasks fail.  When we carry out a typical task performance exercise, we can find failure rates ranging from 30% to 70%.

“Nearly 60% of all phone interactions saw the customer start on the company’s website,” according to ‘The Effortless Experience’.  So, a great many customer top tasks aren’t even getting to the starting blocks!

Google is a good example of an organization that is obsessed with reducing customer effort and time-on-task.  Five years ago, if you searched for “London weather”, you received ten search results, and ideally, you clicked on the first one.  The task took you about 20 seconds to complete two years ago, the weather appeared on the search results page itself, so you didn’t have to click on any search results.  The task now took you about nine seconds to complete.  Today, if you’re using Google Chrome and you start typing “London wea…” the weather information pops up.  You don’t even need to complete the word ‘weather’.  The task now takes you less than five seconds.

That’s why Google is so successful; because they understand that if you maximize task completion, and minimize time-on-task, you win online.

For Impatient Web Users, an Eye Blink Is Just Too Long to Wait (New York Times)
The Effortless Experience (Matthew Dixon, Nick Toman, Rick DeLisi) Transform: A Rebel’s Guide for Digital Transformation (Gerry McGovern)