A Deep Dive into Surveys: Open Ended Vs. Close Ended

Survey questions come in two main varieties: open-ended and close-ended. When creating excellent online surveys, it’s critical you make proper use of both question types and know exactly when to use each one across customer touchpoints and behaviors.

The Questions

Close-ended questions

Close-ended survey questions give a limited set of answer options. As such they don’t allow the respondent to provide a unique or unanticipated answer, rather they offer specific feedback about a focused area.

Examples of close-ended questions are:

• Dichotomous or two-point questions: (e.g yes or no)
• Multiple questions: (A, B, C, D or E)
• Checkboxes
• Drop down
• Scaled questions: that is, making use of rating scales (e.g semantic differential)

Open-ended questions

Open-ended questions are those which require more thought than a simple one-word answer. There are no predefined answers, and respondents can create their own answers. The answers could come in the form of a list, a few sentences or something longer such as a speech, paragraph or essay.

Writing a good open-ended research question can be a tricky balancing act. It should be designed so it prompts users to provide useful information and elaborate responses that are free of restraint.

Examples of open-ended questions include:

• Completely unstructured questions: openly ask the opinion or view of the respondent
• Word association questions: the participant states the first word that pops into his/her mind once a series of words are presented
• Thematic Apperception Test: a picture is presented in which respondents  explain their point of view

Close-ended questions

Strengths:

• Demographic studies: These are used if a manager wants to decipher the demographic breakdown of their store visitors, which could include, age, gender, marital status, and employment status and are easily answerable as questions. The manager could determine a profile of their typical customer.
• Measuring KPIs: Understanding how many of your customers who rate their experience with your brand as “extremely” or “very” satisfied or are likely to recommend your product in an NPS survey is an excellent barometer to understand your business's health. The conclusive nature of the data is easily quantifiable making it particularly useful to prove the statistical significance of results and measure over time.

Weaknesses:

• Deep understanding of topic: The researchers must have a clear understanding of the topic before close-ended questions are designed otherwise they will have insufficient answer options for respondents to select. For example, if I asked the question “What mode of transport did you use to get to work today? Car, Bus or Walk.” I would mistakenly miss out ridesharing or cycling.
• Lazy responses: Respondents with no opinion on the research question may answer anyway. It prevents researchers from further exploring the meaning of the responses.
• Stop the conversation: When you ask closed-ended questions, you may accidentally limit someone’s answers to only the things you believe to be true. Worse, closed-ended questions can bias people into giving a certain response.
• Multiple questions required for insight: If you want insight on more than one specific area, you’ll need to add additional questions for each new area.

Open-ended survey questions

Strengths:

• Richer insight: Open-ended questions allow respondents to include more information, including feelings, attitudes, and understanding of the subject. This allows researchers to better access the respondents' true feelings on an issue. Closed-ended questions, because of the simplicity and limit of the answers, may not offer the respondents choices that actually reflect their real feelings. Closed-ended questions also do not allow respondents to explain that they do not understand the question or do not have an opinion on the issue.
• Cut down response error: Open-ended questions cut down on two types of response error; respondents are not likely to forget the answers they have to choose from if they are given the chance to respond freely.  Also, open-ended questions simply do not allow respondents to disregard reading the questions and just "fill in" the survey with all the same answers (such as filling in the "no" box on every question).
• Reveal the unexpected: Due to the nature of open-ended questions, they facilitate an unlimited number of answers that may reveal some unexpected insights and novel answers.

Weaknesses:

• Harder to extract insight from unstructured data: Despite being the richest source of feedback, it is the hardest to interpret accurately at scale without a tool to analyze the topics and sentiment within each response.
• Larger item of non-responses: Greater amount of thought, response time and effort is needed to complete the question.
• Generalized responses:  Questions may be too general for some respondents who then lose direction. Different people will also give differing levels of detail when answering.

Combining close-ended and open-ended questions

When designing a survey it is often advantageous to combine open and close-ended questions. A well-known example of a survey that uses both types of questions is the Net Promoter Score.

Close-ended questions provide the data that can be measured over time.

An open-ended question will give you the insights as to why the customer gave their rating.

Combining both question types will provide the valuable data you need to make informed business decisions. Gaining a better understanding of your customer's experience and pinpoint areas for improvement.

Usabilla & Chattermill Solution Partnership

Collecting feedback via Usabilla gives you valuable insights regarding your customer’s journey. We understand that continued feedback may be challenging when it comes to analysis, especially to process data in a way that scales as data volumes grow.

In order to get the most out of your collected data, Usabilla collaborates and integrates with Chattermill. Chattermill helps you to get a better understanding of the most important topics your customers talk about using AI to interpret theme and sentiment analysis across unstructured feedback.

With the Usabilla-Chattermill integration, we tailor each area of feedback to how your business runs and ensure that we go into as much detail as is required for your business to effectively gain the insight you need so you can make the right decisions.