A Good Survey Starts with a Robust Screener

A Good Survey Starts with a Robust Screener

Reaching the right people is the key success factor for any survey. This is often more difficult—and important—for business-to-business (B2B) studies, which require specific expertise from respondents. For example, surveying members of a company’s HR department about data center management doesn’t make sense since they won’t have the desired information. It’s important to interview only the people who fit your target profile. This is where the screener comes in.

What is a screener block?

A screener block is a short set of questions designed to prevent anyone outside of your target audience from proceeding further in the survey. It acts as a filter, letting through only those participants who fit the target description. Everyone else? We respectfully thank them for their time and won’t contact them about this particular study anymore.

In a B2B survey, the screener block might include questions about industry, company size (determined by either revenue or employee count,) respondent’s area of responsibility, position/title, decision-making power, awareness/usage of certain products/vendors, and/or level of knowledge on a specific topic.

The best place for the screener questions is the beginning of the survey. The goal is to not waste anyone’s time, so we terminate participants as soon as we know they’re not a fit. The screener’s location is important for another reason: it’s the first thing respondents will see in the survey. Screener design will set their expectations for the rest of the interview. If it’s designed well, it will encourage respondents to proceed further and will decrease the number of dropouts. If it’s full of complex questions with long answer lists, we may lose a portion of the audience before they even reach the main part of the survey.

How is a good screener block designed?

  • Keep it short and sweet. When people learn they’re not a fit after 20 questions, they will likely be upset. Keep the number of screening questions to no more than ten and ideally fewer than five. (Note: since screening usually includes some demographic/firmographic criteria (e.g. company size), it’s tempting to put all questions of that type within the screener block. This is a common error. The screener should only include questions that are critical to the recruitment criteria. Everything else should be asked later in the survey.)
  • Start broadly and narrow down with each next step. For instance, ask about industry first (and screen out anyone outside of your target industry), then move to company size (exclude companies that are too big or too small), then evaluate spend on the target product category (terminate those who spend too little,) and finish with awareness of the target product (end survey for those not familiar with the product.)
  • Consider other survey best practices. These typical survey design best practices still apply to the screener section.
  • Keep screeners neutral and non-leading. That is, don’t tip off respondents as to what answers will help them qualify for the survey. Screening questions should be designed to allow multiple opportunities to fail.

Consider the following examples:

Example 1

Are you responsible for data center management?

  • Yes
  • No

Example 2

Please select your main area(s) of responsibility:

  • Accounting & Finance
  • Data Center Management
  • HR Management
  • Legal
  • Marketing
  • Procurement
  • Research & Development
  • Other

The second example gives no clues as to what the topic of the survey is or what kind of expertise we seek. This more neutral approach will help weed out astute participants of market research who are just responding for the incentive.

Why is a screener block necessary if the panel already includes some information?

Even if targeting is specific, we still need screeners for a number of reasons:

  • Data updates: The screener provides the most up-to-date details about participants’ expertise. Panelists may have recently switched companies, careers, or geographic locations—tags that may be relevant to the study—but haven’t updated their panel profile yet.
  • Targeting: Even in cases where pre-targeting is relatively pinpointed, it’s unlikely the panel has 100% of qualifying criteria as a panel data tag, especially for B2B studies. For example, we may be able to target IT decision-makers, but we may not know what software they are currently using. Screeners serve as an additional layer of security to ensure only relevant respondents participate in the survey.
  • Confirmation: With the screener we have it on record that a certain respondent meets all the qualifying criteria.
  • Analysis: Screeners often include some demographic/firmographic questions that are needed in the analysis stage. It’s useful to have them in the same dataset with the main content of the survey.
  • Data validation: Screeners can help uncover potential data quality issues and logic breaks, such as if screener answers conflict with other responses later in the survey.

Takeaway: The screener block is critical to good data quality. Only people truly belonging to your target audience will be able to provide relevant insights. Take time and care to design the screener section well.