Surveys are a great way to obtain customer feedback and improve internal operations at any given organization. While customer surveys can provide valuable information, that data is contingent on the survey questions asked. Collecting unbiased responses relies on unbiased survey questions so that your data is as reliable as possible.
Bad survey questions can be attributed to a number of factors:
First and foremost, biased survey questions can lead the respondent feeling inclined to select or give a certain response. In legal terms, this is often referred to as a leading question.
In this article, we’ve listed some of the most common culprits of problematic survey questions to make sure your survey responses are reliable, accurate, and can be clearly presented in a survey results report.
1. Avoid Leading Questions
In the courtroom, lawyers will often use the objection, “leading question.” This is when a lawyer asks an individual on trial a question with an implied answer embedded into the inquiry. In other words, they’re putting words into the witness’s mouth.
A leading question is identifiable by a couple characteristics: the first being subjective adjectives and the second being contextualizing certain phenomena in a positive or negative light preemptively.
The problem with leading questions is that they fill in the details and suggest what a witness experiences as opposed to letting the witness share their experience. This correlates with customer feedback surveys — bad survey questions don’t allow the customers to share their experience and instead obstruct their ability to provide valuable feedback. Below, we’ve listed some examples of leading questions and how to amend them.
Examples of Leading Questions
Leading Question: How excellent is our customer service?Good Question: How would you describe your experience with our customer service representative?
Leading Question: What made this product amazing?Good Question: How would you rate your experience with this product?
2. Avoid Loaded Questions
Another type of question that can lead to biased survey responses is a loaded question, or a question that contains assumptions about the respondent or their behavior. Consequently, the respondent can inadvertently end up agreeing or disagreeing with the implied statement.
For example, the common question cashiers ask their customers is, “will you be paying with cash or card?” However, this is technically a loaded question because it implies that the customer has already decided to purchase the item(s). If they answer, they’re also agreeing to the implicit statement that they are making the purchase. This wouldn’t be a loaded question if the customer had already expressed the desire to make the purchase. Loaded questions should be avoided in customer satisfaction surveys by making sure the context is appropriate based on the proper customer data.
Examples of Loaded Questions
A loaded question is identifiable through reading the entire survey because context for the assumption may be derived from the previous question. Here is an example of a loaded question and how to fix it.
Loaded Question: Where do you enjoy drinking coffee?
- Unless the respondent says prior that they drink coffee, this is a loaded question.
Good Question: Do you enjoy drinking coffee? If so, where do you enjoy drinking it?
To prevent loaded questions, you could ask the initial question, and depending on their answer, you could use conditional skip logic. This means that if the respondent answers, “yes I do enjoy drinking coffee,” then they can move on to the question, “where do you enjoy drinking coffee?” Otherwise, if they say that they don’t enjoy drinking coffee, there’s no point in asking them where they enjoy drinking it.
3. Avoid Double-barreled Questions
Asking a question where you are technically asking two is often referred to as a double-barreled question. When you ask two questions at once, it becomes difficult for the customers to answer either one accurately. An easy way to identify a double-barreled question is if it contains the word “and” or “or” in the question. Below, there is an example of a double-barreled question and how to fix it.
4. Try Likert Scale Questions
One of the most common question-answer formats in surveys are Likert Scale questions; these allow you to quantify your respondent’s disposition towards the specific assertion within the context of your research. Most Likert scale questions have three, five or seven point ranking scales that give respondents the option to indicate to which degree they agree with the statement in question.
What makes the Likert scale useful is that the question answers have a midpoint for neutral responses: individuals can neither agree nor disagree. This method is best for quantitative data in research. However, there are two types of Likert scale questions you may encounter: unipolar or bipolar. Unipolar Likert scale questions have one pole where they measure one quality, while bipolar Likert scale questions have two opposites that focus on two different qualities. Here are some examples of Likert scale questions in action.
Question (1) I am an avid reader of books.
- Strongly agree
- Agree
- Neither agree nor disagree
- Disagree
- Strongly disagree
Question (2) How satisfied are you with our customer service?
- Very satisfied
- Somewhat satisfied
- Neither satisfied nor dissatisfied
- Somewhat dissatisfied
- Very dissatisfied
Likert scale questions are the gold standard for good survey questions because they are easy and straightforward. They are also a great tool for unbiased survey questions because they can quantify your respondents’ thoughts and emotions for valid research data. It also takes into account variations in respondents' thoughts and experiences to represent them accurately.
5. Keep it Simple with Dichotomous Questions
Another common type of survey question is one that is dichotomous, meaning that there are only two possible options that are typically parallel: think “true or false” or “yes or no” types of questions. It’s important to note that with this style of question, there is no neutral response option. Dichotomous survey questions are best used when you need a definitive answer, so avoid using them when you need a broader response. Here are some examples of common dichotomous survey questions.
Question (1) Was this article helpful?
- Yes
- No
Question (2) Would you recommend this product to a friend?
- Yes
- No
6. Avoid Double Negative Questions
Double negative questions are more often than not confusing to survey respondents. This is when there are two negatives in the same sentences, like, “Don’t not write clearly.” The simplified version of this sentiment is simply, “Write clearly.” Make sure your survey questions do not have double negatives so as not to confuse your respondents, which could ultimately result in incorrect data. Below are some examples of double negatives and how to fix them.
Double Negative: Do you relate to this statement? I am not unaware of this trend.Corrected Question: Do you relate to this statement? I am aware of this trend.
Double Negative: How true is this statement? I don’t think your product is not too expensive.Corrected Question: Fixed question 2: How true is this statement? Your product is not too expensive.
7. Don’t Ask Vague Questions
Writing clear, concise questions is the key to a successful survey. Vague or ambiguous questions are those that are too broad or are poorly defined. The end result is often generic responses that aren’t useful to your research. In an effort to avoid vague survey questions, make sure that your questions are worded clearly in a way that respondents will find them easy to understand.
8. Avoid using Confusing Jargon
Jargon, or verbiage that isn’t used by the general population, is something you should avoid when writing survey questions. Slang, catchphrases, cliches and colloquial language shouldn’t be included in your questions because they can confuse or distract the respondents. An additional benefit of excluding jargon from your questions is that customers who speak other languages will find it easier to translate the questions and comprehend them.
To identify jargon, it helps to have individuals from different ages, races or backgrounds read through and test your surveys. Here are some examples of jargon and how to edit your questions to remove them.
Jargon: How would you rate our DTC team?Corrected Question: How would you rate your experience with [name]?
Jargon: How often do you check each employee’s KPI?Corrected Question: Is employee performance something you consider important?
Removing unnecessary jargon can not only streamline your survey results but it can also ensure that your data is accurate. It also saves time on behalf of the respondents because they don’t need to spend extra time trying to understand the question. Clear email survey subject lines can also increase open and response rates.
9. Check for Mismatched Scales & Poor Answer Options
Mistakes in your survey questions lead to inaccurate data, especially when it comes to incorrectly using an answer scale. It’s crucial to consider not only the best way to ask a question, but also the type of response that most effectively allows your respondents to give you valuable feedback. An easy way to check this is by making sure your answer options match what the survey question asks.
Mismatched Scales
Identifying a mismatched scale requires you to look and see if the question you’re asking matches the answer responses the respondent can choose. Here’s an example.
Q: How satisfied are you with your experience today?
- Yes
- No
This question is asking about the quality of their experience; however, the answer options being ‘yes’ and ‘no’ don’t allow the respondent to answer accurately. Here’s how we can fix it.
Q: How satisfied are you with your experience today?
- Very satisfied
- Somewhat satisfied
- Neither satisfied nor dissatisfied
- Somewhat dissatisfied
- Very dissatisfied
Poor Answer Options
When listing the answer options for a survey question, it’s important to make sure that the answer options don’t overlap so as not to confuse the respondent. Below, we’ve listed a problematic survey question and proposed revision.
Q: What is your age?
- 18–24
- 24–28
- 28–32
- 32+
The problem with the answer options above is that they overlap: if the respondent is 24, they can technically pick the first two boxes. This can lead to confusion and incorrect data. Here’s a proposed revision.
Q: What is your age?18–2425–2930–3233+
Checking your questions and answers to make sure they correspond is a crucial step in taking your survey for a test run.
10. Don’t Ask Biased Questions
While advocating for your company is vital, doing so in your survey question may lead to bias. To avoid this, make sure that your question is worded neutrally without pressuring the respondent to think a certain way. Here’s what not to do.
Q: Hot Dog Cafe has the highest-rated customer experience in the state. How would you rate your experience at our restaurant?
- Extremely Satisfied
- Very Satisfied
- Less than Satisfied
- Not Satisfied
By leading the question with the “highest-rated customer experience,” two things can occur: the respondent can rate your restaurant lower because they felt they didn’t experience the highest-rated customer experience, or they can rate your restaurant lower so as not to disturb their rating. Either way, this doesn’t accurately represent the sentiment of the respondent. Avoiding biased questions leads to genuine survey data.
11. Don’t Ask Too Many Questions
Receiving feedback from customers is extremely valuable to your company, but so is your customer’s time. Making sure not to ask too many questions in your survey will not only improve the completion rate, but it will encourage them to follow through with the survey. Keep your survey short, sweet and to the point for genuine, valuable feedback you can implement in your business.
Additional Tips for Writing Goodtext in italic Survey Questions
The most successful surveys are those which implement all of the tips and tricks we mentioned above. Making sure that you create a concise, digestible survey will improve the completion rate and perhaps even lead customers to taking more of your feedback surveys in the future. Learn more about how to redefine your brand’s customer experience using unified customer feedback analytics.
