Do you ever wish answering screener questions was more like eating cupcakes? Why choose one when you could choose many?
Good news! We’ve added the ability to create multi-select screener questions for your usability tests, in addition to our existing single-select type questions. With this new feature update, you can now create screeners that allow testers to select multiple answers in a check-all-that-apply style.
To get you up to speed, we’ll show you how it looks and works with a couple of examples, and then provide a quick summary of the update at the end of the article.
First, a few changes to the existing screener features…
If you’ve created a new usability test recently, you may have noticed that we renamed the “Additional qualification requirement” to “Verbal response screener” in order to more clearly convey how this feature works.
Before users begin the test, we show them whatever you type into the verbal response screener textbox. State your requirements (experiences, skills, personality, etc.), and those who meet them will continue on. Then, right at the beginning of their test video, they will verbally describe how they meet your criteria.
There’s also been a slight wording change in our existing single-select type screener questions: whereas previously answers could be set to “Accept” and “Reject,” these options are now called “Allow” and “Reject.” They still work the same way, but the wording is more in line with the new type of screener that’s been added.
Read more: Use screener questions to find your usability testing participants
How to use the new multi-select screener questions
To create a multi-select, check-all-that-apply screener question, click the “Type” dropdown below the question label (shown below). Change the selection from single-select to multi-select.
As an example, let’s pretend we’re an aquarium supplies company, and we want to run a usability test with people who own only aquatic or semi-aquatic pets. Here’s how we might write our screener:
Either the Fish or Turtle answers would allow testers to proceed, while the Dog and Cat answers will prevent testers from continuing.
Previously, if someone owned both a dog AND a turtle, they wouldn’t be able to select both answers, so this question wouldn’t have worked with the existing format.
With multi-select screeners, testers can now select any number of answers. Answers for these questions no longer need to be mutually exclusive, as with the single-select questions. This enables you to ask a greater variety of questions, and screen for things that won’t always have just one discrete answer.
See also: Tips on writing usability testing tasks
Required answers vs allowed answers
Perhaps the most powerful aspect of the multi-select screener questions is the new answer type. In addition to “allow” and “reject,” there’s a third option: “require.”
Although testers can select any number of “allowed” answers and still continue on to the test, if you include a “required” answer, testers will have to select that answer in order to proceed – regardless of how many “allowed” answers they chose.
(Note that if a “rejected” answer is selected, the tester will be denied from proceeding, even if they also selected the “required” answer).
So for example, if we were only interested in people who definitely own a turtle, but could also own fish, we would change the action for Turtle to “require.”
That way, testers who happen to own both will still be allowed to continue on and take the test, but now every tester is guaranteed to be a turtle owner.
Not only that, but since we’ve set the Dog and Cat answers to “reject,” they are also guaranteed to NOT own a dog or cat.
But what if we’re ok with testers having a dog or cat, so long as they definitely have a fish or a turtle?
In the above screenshot, we’ve changed Dog and Cat to both be allowed, and Fish and Turtle to be required. After all, what we really want is to require testers to have either a fish OR a turtle – not necessarily both, but at least one of them.
Now that we have more than one “require”-type answer, a new setting has appeared at the bottom!
“Testers must select (at least) (1) of the ‘Required’ answers to be eligible.”
That’s yet another screening tool you can use to perfectly tailor your screeners to get the right audience.
Using this setting, you can specify the number of required answers testers need to select in order to advance. In this case, we simply want to require that they choose 1 or more of the “require”-type answers.
Now, our screener will identify testers who definitely own either a fish OR a turtle, and maybe a dog or cat too. That’s just fine with us.
Let’s use another example to illustrate the possibilities of this setting.
In this new scenario, we’re a company looking for people who are trilingual in any 3 languages. To do this, we will first change all of our answer actions to “require.”
But of course, we don’t actually need the users to select every single answer in order to take the test. We just don’t care which 3 they choose, so we’re setting them all equal. Now, we will fine-tune how our pool of required answers will work using this special tool.
Click on the dropdown and change “at least” to “exactly,” and enter “3” in the number textbox – we’re looking exclusively for trilingual people, so in this case we only want people who choose exactly 3 of our required answers, no more, no less.
If we were screening for people who spoke 3 or more languages, on the other hand, we would simply change “exactly” to “at least” for the desired effect.
Tips on writing better screener questions
We’ve covered all the ways you can set up your screener questions, with every permutation of answer types. Now let’s talk a little bit about how to write the best possible screeners.
Going back to our pets question, there are some improvements we could make so that it’s a better question.
The main change needed is to provide for the full range of possible answers, so that everyone who responds to this screener can answer accurately.
Previously, when we had only given Dog, Cat, Fish, and Turtle as answers, there would be no way for someone with a different pet or no pets at all to respond. Now, we’ve added Other and None as options, so that everyone has a viable response to pick no matter their situation.
In most cases when writing a multi-select type question, you will need to include an Other and/or None option. Remember to think through all of the possibilities and give an answer for everybody!
We also made a subtle edit to the question text, changing “pet” to “pet(s)” to more clearly convey to the respondents that they can check one or multiple answers. Another way to do this would be to add an instruction like “Check all that apply” to the end of the question.
Recap:
- “Additional qualification requirements” renamed to “Verbal response screener”
- “Multi-select” screener question type added
- Testers may now select multiple answers for one question
- 3 answer types: “require,” “allow,” and “reject”
- “Allowed” actions will not reject users, but will also not be enough to permit users to proceed IF a “required” answer is present and not also selected
- Note that you do NOT need a “require”-type answer for questions to be valid
- Ability to fine-tune number of “required” answers testers must choose
- For questions with more than one “required” answer, designate the minimum or exact number needed for a tester to be eligible to continue
This new variant for your screener questions will add a level of refinement to your audience recruitment, and ensure that you’re getting the best possible results from your usability test.
Currently, screener questions are limited to our Enterprise Plan. Verbal response screeners, however, are included with all plans, as well as our basic demographic targeting.
