How to make surveys worth the effort

October 4, 2016
Reading time:

Customer Feedback Data Analysis

Right now in my inbox I have at least 5 emails from companies asking for my feedback as a customer. I may open one of them later on if I have 60 seconds to spare. The rest I will delete. Working in a CX startup, that may come as a surprise; but fundamentally if a company’s survey mechanism is not inviting or well made then I would doubt them using my feedback in any meaningful way at all.

Response rates are the first indicator a company can use to check how engaged their customers are. If they’re low and stay low, your customers are telling you something — you don’t respect their time. The reasons for this usually fall into 2 buckets:

You survey is too long - this is such a big problem that we dedicated a whole blog post to it. In a nutshell, your surveys should be quick and easy to complete. If it takes anything longer than 2 minutes, expect a low turnout. More importantly, your sample will probably be biased and the customers who do finish the survey, will feel less excited about your product by the end.

No follow-up action on issues raised - More often than not, companies are overwhelmed by the feedback they collect. Only 23% of CMO’s said they track or measure customer feedback. So it will sit in a spreadsheet or data warehouse somewhere and be forgotten about.

Ultimately, poorly executed feedback setups do more harm than good. Low quality data leaves companies with little insight into customer experiences and customers feeling ignored. It shouldn’t be like this. Customers should be able to tell you about their experiences in their own words. Companies with a relentless focus on this gain better insights into customer behaviour and make better decisions.


Consider the scenario — a travel company implements a post-trip feedback system to gain insights into what customers find are the most important factors in their trip. Trying out 2 different types of surveys, they get vastly different results.

Survey 1:

Long Survey

There are so many problems with this approach. It’s far too long (and I’ve seen much longer), response rates will be under 5% and of the customers who do respond, they’ll get survey exhaustion and click through the survey to get to the end meaning any useful insight is really hard to extract.

Yes, you can quickly calculate the correlation between the accommodation score and the overall score (or better yet the actual re-booking rate) to estimate it’s effect but a 3 for one person is not the same as the 3 for another. Moreover, you have no idea what weight does each customer assign to accommodation. Maybe they gave you a 5 but did not care one way or another.

Understandably, a company will want as much feedback as possible but it is about the quality of insight, not quantity of questions that tries to cover all the aspects of experiences you can think of. You’ll never be able to cover every single thing a customer experiences into a survey.

Plus, why would you want to? The richest data sets are those from your customers in their own words on the most important factors of their experiences. Not the questions that the Product team want answering, and the questions that Marketing need to figure out.

Survey 2

Gif for blog

By keeping it short and sweet, customers are more likely to take the opportunity to give you their feedback in their own words and mention the most important things about their trip. Using the latest advances in deep learning we can have the machine automatically analyse customer verbatims. This will give you an amazingly rich data set that you can analyse to your heart’s content, observing sentiment and theme trends to uncover why customers feel certain factors impact their customer experience.

The Deep Learning approach to processing feedback can ‘learn’ a bespoke theme structure based on the comments it analyses, catching positive and negative sentiment tags and sub-themes of customer comments that looks something like this:

final theme structure

By now you should be convinced that asking customers for feedback in their own words is the only way to build feedback systems and ultimately make customer surveys worth the effort they take to build. Following the key guidelines above will help you make use of your customer feedback, discover the ‘unknown unknowns’ of the drivers of customer experience and grow the company.

At Chattermill, we use cutting edge Deep Learning approaches to solve these problems and provide companies with complete understandi

See Chattermill in action

Understand the voice of your customers in realtime with Customer Feedback Analytics from Chattermill.