What type of research is the instrument is open

Posted By Lorène Fauvelle on 12 Apr, 2019

From a very young age, we have been taught what open-ended, and closed-ended questions are. How are these terms applied to qualitative research methods, and in particular to interviews?

Kathryn J. Roulston reveals her definitions of an open-ended and closed-ended question in qualitative interviews in the SAGE Encyclopedia on Qualitative Research Methods. If you want to better understand how qualitative methods fit within a market research approach, we suggest you take a look at our step-by-step guide to market research which can be downloaded in our white papers section [free of charge and direct; we won’t ask you any contact details first].

credits : Shutterstock

Summary

1. Introduction
2. Closed-ended question
3. Open-ended question
4. Examples of closed and open-ended questions for satisfaction research
5. Examples of closed and open-ended questions for innovation research
6. Some practical advice

Introduction

Let us begin by pointing out that open and closed-ended questions do not at first glance serve the same purpose in market research. Instead, open-ended questions are used in qualitative research [see the video above for more information] and closed-ended questions are used in quantitative research. But this is not an absolute rule.

In this article, you will, therefore, discover the definitions of closed and open-ended questions. We will also explain how to use them. Finally, you will find examples of how to reformulate closed-ended questions into
open-ended questions in the case of :

• satisfaction research
• innovation research

Essential elements to remember

Open-ended questions:

• for qualitative research [interviews and focus groups]
• very useful in understanding in detail the respondent and his or her position concerning a defined topic/situation
• particularly helpful in revealing new aspects, sub-themes, issues, and so forth that are unknown or unidentified

Closed-ended questions:

• for quantitative research [questionnaires and surveys]
• suitable for use with a wide range of respondents
• allow a standardised analysis of the data
• are intended to confirm the hypotheses [previously stated in the qualitative part]

A closed-ended question

A closed-ended question offers, as its name suggests, a limited number of answers. For example, the interviewee may choose a response from a panel of given proposals or a simple “yes” or “no”. They are intended to provide a precise, clearly identifiable and easily classified answer.

This type of question is used in particular during interviews whose purpose is to be encoded according to pre-established criteria. There is no room for free expression, as is the case for open-ended questions.
Often, this type of question is integrated into 1-to-1 interview guides and focus groups and allows the interviewer to collect the same information from a wide range of respondents in the same format. Indeed, closed-ended questions are designed and oriented to follow a pattern and framework predefined by the interviewer.

credits : Shutterstock

Two forms of closed-ended questions were identified by the researchers: specific closed-ended questions, where respondents are offered choice answers, and implicit closed-ended questions, which include assumptions about the answers that can be provided by respondents.

A specific closed-ended question would be formulated as follows, for example: “how many times a week do you eat pasta: never, once or twice a week, 3 to 4 times, 5 times a week or more?”
The adapted version in the form of an implicit closed-ended question would be formulated as follows: “how many times a week do you eat pasta? ». The interviewer then assumes that the answers will be given in figures.

The Net Promoter Score [or NPS] is an example of closed question [see example above]

While some researchers consider the use of closed-ended questions to be restrictive, others see in these questions – combined with open-ended questions – the possibility of generating different data for analysis. How these closed-ended questions can be used, formulated, sequenced, and introduced in interviews depends heavily upon the studies and research conducted upstream.

In what context are closed-ended questions used?

• Quantitative research [tests, confirmation of the qualitative research and so on].
• Research with a large panel of respondents [> 100 people]
• Recurrent research whose results need to be compared
• When you need confirmation, and the possible answers are limited in effect

An open-ended question

An open-ended question is a question that allows the respondent to express himself or herself freely on a given subject. This type of question is, as opposed to closed-ended questions, non-directive and allows respondents to use their own terms and direct their response at their convenience.

Open-ended questions, and therefore without presumptions, can be used to see which aspect stands out from the answers and thus could be interpreted as a fact, behaviour, reaction, etc. typical to a defined panel of respondents.

For example, we can very easily imagine open-ended questions such as “describe your morning routine”. Respondents are then free to describe their routine in their own words, which is an important point to consider. Indeed, the vocabulary used is also conducive to analysis and will be an element to be taken into account when adapting an interview guide, for example, and/or when developing a quantitative questionnaire.

credits : Shutterstock

As we detail in our market research whitepaper, one of the recommendations to follow when using open-ended questions is to start by asking more general questions and end with more detailed questions.
For example, after describing a typical day, the interviewer may ask for clarification on one of the aspects mentioned by the respondent. Also, open-ended questions can also be directed so that the interviewee evokes his or her feelings about a situation he or she may have mentioned earlier.

In what context are open-ended questions used?

• Mainly in qualitative research [interviews and focus groups]
• To recruit research participants
• During research to test a design, a proof-of-concept, a prototype, and so on, it is essential to be able to identify the most appropriate solution.
• Analysis of consumers and purchasing behaviour
• Satisfaction research, reputation, customer experience and loyalty research, and so forth.
• To specify the hypotheses that will enable the quantitative questionnaire to be drawn up and to propose a series of relevant answers [to closed-ended questions].

It is essential for the interviewer to give respondents a framework when using open-ended questions. Without this context, interviewees could be lost in the full range of possible responses, and this could interfere with the smooth running of the interview.
Another critical point concerning this type of question is the analytical aspect that follows. Indeed, since respondents are free to formulate their answers, the data collected will be less easy to classify according to fixed criteria.

The use of open-ended questions in quantitative questionnaires

Rules are made to be broken; it is well known. Most quantitative questionnaires, therefore, contain free fields in which the respondent is invited to express his or her opinions in a more “free” way. But how to interpret these answers?

When the quantity of answers collected is small [about ten] it will be easy to proceed manually, possibly by coding [for more information on the coding technique, go here]. You will thus quickly identify the main trends and recurring themes.

On the other hand, if you collect hundreds or even thousands of answers, the analysis of these free answers will be much more tedious. How can you do it? In this case, we advise you to use a semantic analysis tool. This is most often an online solution, specific to a language, which is based on an NLP [Natural Language Processing] algorithm. This algorithm will, very quickly, analyse your corpus and bring out the recurring themes. It is not a question here of calculating word frequencies, but instead of working on semantics to analyse the repetition of a subject.

Of course, the use of open-ended questions in interviews does not exclude the use of closed-ended questions. Alternating these two types of questions in interviews, whether 1-to-1 interviews, group conversations or focus groups, is conducive not only to maintaining a specific dynamic during the interview but also to be able to frame specific responses while leaving certain fields of expression free.
In general, it is interesting for the different parties that the interview ends with an open-ended question where the interviewer asks the interviewee if he or she has anything to add or if he or she has any questions.

Examples of closed and open-ended questions for satisfaction research

Examples of closed and open-ended questions for innovation research

In this type of research, you confront the respondent with a new, innovative product or service. It is therefore important not to collect superficial opinions but to understand in depth the respondent’s attitude towards the subject of the market research.

Some practical advice

As you will have understood, open-ended questions are particularly suitable for qualitative research [1-to-1 interviews and focus groups]. How should they be formulated?

The Five W’s; [who did what, where, when, and why ] questioning method should be used rigorously and sparingly :

• Who? Who? What? Where? When? How? How much? “are particularly useful for qualitative research and allow you to let your interlocutor develop and elaborate a constructed and informative answer.
• Use the CIT [Critical Incident Technique] method with formulations that encourage your interviewer to go into the details of an experience: “Can you describe/tell me…? “, ” What did you feel? “, ” According to you… “
• Avoid asking “Why?”: this question may push the interviewer into a corner, and the interviewer may seek logical reasoning for his or her previous answer. Be gentle with your respondents by asking them to tell you more, to give you specific examples, for example.

In contrast, closed-ended questions are mainly used and adapted to quantitative questionnaires since they facilitate the analysis of the results by framing the participants’ answers.

Image: Shutterstock