Market Research – Why Screening For Talkative Respondents Doesn’t Work


We recently published a special report entitled 25 Common Field Mistakes to Avoid When Conducting Your Qualitative Market Research. In point 12, we suggest that researchers remove open-ended screening questions that seek to identify respondents who are outgoing and articulate.

Here is the full text of point #12:

“If your filter contains open-ended questions meant to elicit expressive types of people, skip those questions. Questions like that don’t work, and they unnecessarily lengthen your selection process. Yes, you want to exclude respondents who can’t not or won’t speak up, but you don’t need an extra question to identify these people. Well-trained recruiters will weed them out within the first few minutes of screening. If you want to be sure that your respondents will be outgoing and talkative, over-recruit and include a pre-discussion telephone interview, which would be conducted by the moderator who would then select the appropriate respondents.”

Our suggestions generated a lot of comments. Some readers agreed with us, others didn’t. One of the most interesting comments we received came from a qualitative fieldwork manager at a large full-service research firm. It went like this: “I disagree with something on your list about excluding reviewers’ open-ended questions. Good recruiters may be able to easily identify eloquent respondents, but tired or distracted recruiters can go on autopilot sometimes… I think a little quality in a screener is worth the time.”

Do you agree with this reader’s comment? Should you?

What is an articulation question?

Articulation questions measure a respondent’s ability to communicate. Articulation questions also judge respondents expected communicativeness in a focus group or interview.

Some synonyms for “communicative” include: outgoing, open, open, talkative, unrestrained, talkative. So who decides what is communicative? The recruiter ? The hiring manager? The client who reads the verbatims of his daily reports? And how much communication is enough? How much is too much?

Even the most experienced recruiters cannot determine how outgoing, outgoing, outgoing, talkative, unrestrained, or talkative a respondent will be in the future. This is a judgment that recruiters are not qualified to make. But we can count on them to identify respondents who have…

  • Language barrier
  • casual attitudes toward the recruiter, the recruiter’s questions, or the search
  • reservations about their ability to assist in research
  • any communication issues during the selection process

What to watch out for…

Respondents get tired or go on autopilot when selection interviews take too long (10 minutes or more).

Articulation questions have no place at the end of your filter. For some reason, joint screening is almost always done at the end of the screening interview. But why is such a supposedly important question asked at the end of the screening tool, when respondents are most likely to be tired or distracted? What do recruiters learn from respondents at this stage of the process that they don’t already know?

Articulation matters don’t belong in the front of your filter either. Well-trained recruiters immediately engage respondents in a conversation regarding research details. It is during this prelude to screening questions that recruiters address respondent questions and concerns and assess a respondent’s communication ability.

Articulation issues are not magic bullets that guarantee good focus group participants. These questions simply ask recruiters to use their own biased judgment in deciding whether a respondent can communicate clearly.

Articulation questions lengthen your filter. Remember this. The longer your filter, the higher your costs.

Respondents become anxious when asked questions out of left field that are not related to screening questions. When asked, “What is a gazinkle?” or “How many different things can you do with a paperclip?” or “If you were a tree…?” could baffle even the most eloquent respondent. Offbeat questions from recruiters confuse and frustrate respondents. This line of questioning is moderator territory.

Of course, group dynamics and respondent personalities affect respondent openness and responsiveness. For example, a person may be open on the phone with the recruiter, but feel intimidated if an aggressive personality dominates the group. Or, a respondent may not be as comfortable with the research topic as they thought and feel misplaced – especially if the topic offered during recruitment was vague. How can recruiters know how respondents will act under various conditions? Managing restricted respondents is the moderator’s area of ​​expertise.

In fact, moderators are best qualified to know what can and should be expected of respondents in terms of communicativeness and articulation. It therefore makes sense that, as we suggest in point 12 of our special report, moderators pre-interview respondents and select the right personalities for research.

So what about useful articulation questions to get distracted or oblivious recruiters out of their daze (as our reader suggested)? Assuming a tired, distracted recruiter missed all the red flags in the selection, will the articulation question suddenly remind the recruiter that the respondent isn’t talkative? What to do with recruiters on “autopilot”? Simple.

The researcher’s job is not to formulate questions that keep recruiters alert and focused. Tired or distracted recruiters are not an asset to your search. They don’t help you get good respondents. And the issues of articulation either. Do not use either.

Source by Mark Goodin

Comments are closed.