It is safe to say that almost none of the analysts looking at the 2016 US Presidential Election predicted Donald Trump would win. A lot of very sophisticated models using polling and economic data were built and widely publicised before the election but even the most bullish on Trump gave him only a 29% chance of winning. What does this have to do with your product?
I would argue that regardless of how you feel about Mr Trump winning the most powerful office in the world, there are deep lessons to be learned across the board and some of the most important ones apply directly to how companies listen to their customers.
Lesson 1: Avoid Sampling Bias
As with Brexit, pollsters’ biggest problem appears to be chronic inability to survey a truly representative sample of the population. For obvious reasons no pollster can ever ask everyone for their opinion. In fact, most of them only ask a few thousand people per region, if that. The true art of this discipline is to find the few thousand that have the same characteristics as the population as a whole. This never quite works but as long as it is true on average we can expect the law of large numbers to help and the data to be predictive. What happens if your errors are not random? For example, what if you systematically miss the young or the rich or certain other demographic or regional groups? Then your predictions go off base.
What this means for customer insight is you should design your programme in such a way that it covers your whole customer base. Too many companies fail miserably here by opting for long, boring surveys hated by their most important customers: younger and busier people in particular. Others make the mistake of relying too much on data sources such as Twitter which, while wonderful in certain settings, also presents a highly biased picture.
The best way to avoid the problem is to:
- Ask every customer
- Ask them in the least disruptive way possible. The NPS, CSAT, reviews and similar methodologies generate much better response rates than long multiple choice surveys.
- Add demographic / regional and other attributes to the data but not by asking people. You should have that data already. Once the results are in make sure to weigh them by demographic factors to match your overall audience.
Lesson 2: Ask The Right Questions
Surprisingly, it seems that asking “Who are you going to vote for?” does not work as well as it should. Much has been said about the so-called “shy Trump” (or “shy Tory” in the UK) effect. The experts disagree whether respondents actively lie to pollsters and whether this has a huge effect. There is some evidence to that effect. For example, in Michigan, one of key states lost by Clinton, up to 60% of women said they were going to vote for her.
Another, less discussed problem are the “undecideds”. We don’t hear about them as much as we do the main numbers in the polls. In this election, the number of those who left it until the very end to make up their mind was four times higher than 4 years ago. In the end they largely voted for Trump.
Whether they voters were really struggling to choose or simply did not want to admit they supported Trump the whole time, asking them a simple question did not help.
In a business setting, this happens in two ways. Quite often standard questions such as “Would you recommend Company A to a friend” do not work because customers might be embarrassed admitting they like the business. Dating sites come to mind first but we’ve also seen this effect with financial companies and certain clothing brands. Even with less controversial companies, people say things like “Love your service but I would not recommend you to my friends because you are too expensive for them”.
A less obvious but actually more serious problem with simple questions is that customers often answer them without much thought just to get through the survey, particularly when an answer is required. We often see companies ask questions such as “Rate your delivery experience on a scale of 1 to 5”. Customers answer them with a rating (typically a 1 or a 5) even when they did not notice the quality of delivery. Those answers are not a valuable guide for building great customer experience.
Open ended questions are a great alternative to simple-but-confusing multiple choice surveys. It’s much harder for a customer to lie or offer irrelevant information if they have to actually type in their answer. We also recommend sticking to vague questions such as “What would you tell your friends about us” or “How would could we improve your experience” that let the customer tell you what’s important, not vice versa. One question is more than enough. Resist the temptation to ask about what you think is important at all costs.
Lesson 3: Ask Often
By all accounts, the scales finally tipped in Trump’s favour only in the last week or so before the election. In fact, many of the models failed to predict the result precisely because they gave too much weight to older poll results and not enough to recent ones. In politics, your support can build or crumble in a matter of days. In business, such dramatic shifts are less common but we have seen them happen. Your delivery partner may mess up all of your orders just before Christmas. Customer data may get accidentally deleted or leaked online. Or maybe some of your brand new flagship phones have a nasty tendency to explode.
Quite often these problems are not noticed at the global level until it’s too late. For this reason, we strongly believe customer experience measurement should be done every day on a rolling basis with data processed in near real time. Annual or even quarterly surveys are just not enough in the modern hyper-connected market. It used to take months for such a problem to become common knowledge. Now it takes minutes. Similarly, whether you are a CEO or a Head of Product, don’t rely on a once-in-a-while report about what your customers think. You should be able to see changes in their attitudes as soon as possible so you have at least a chance at adjusting the strategy before they hit your bottom line.
Let’s sum it up. To avoid unpleasant surprises with your customers, follow these simple rules:
- Make your customer insight process as user friendly as possible to ensure unbiased samples. All you need is one question.
- Focus on open-ended questions over multiple choice ones. The answers will focus on what’s really important instead of what you are interested in. Your customers may surprise you.
- Ask every customer, preferably after a certain touchpoint such as purchase or sign up. This way you will have the data that allows you to spot the problems quickly.
- Make sure your entire team is connected right to the source of customer insight where everyone can find the relevant insight in seconds and be alerted when problems arise.
If you already have a program in place that addresses all of the points above, we would love to talk to you for a follow up post. If not, let us build it for you.
