Update: Read our latest post (Jun. 2019) to see how we’ve improved screener questions with new customization options
Screening for the right type of users is key to getting accurate, valuable feedback from your usability testing study.
For many sites and apps, the default demographic filters – age, gender, income, education, etc. – are enough. Really, most test screens don’t need to get overly granular; a wide range of people ought to be able to figure out your interface without too much trouble.
In some cases, though, you may need a more particular group of users than are afforded by basic filters. For example, an online store selling pet food and accessories will obviously benefit the most from feedback by actual pet owners. A coat store will learn more from people living in cold places than, say, Miami.
This article will show you how to screen for custom demographics using our new screener question feature and other options.
First, the verbal response screener…
The screening option we’ve always offered, and which is available on all plan types, is our verbal response screener textbox.
This is a free text box where you can write anything at all: “Must be a pet owner.” – “Must care about environmental causes.” – “Must have gone on an international trip in the last year.”
Anything from experiences to special skills to personality traits and more can go in this box. Before users begin the test, we’ll show them your requirements and allow only those who are eligible to continue. For those that move on, we show the requirements again at the very beginning of the video portion and ask them to specifically state for the record, with details, how they meet your requirements.
This box is also where you should write any special requests or privacy concerns. For example, if testers will need to sign up for an account with a real email address, and verify it during the test, write it in the textbox. That way, users who aren’t comfortable doing this while recording their screen won’t take the test and waste their time and yours.
Adding a screener question
Under the “Verbal response screener” heading, there is a link to add a screener question. Clicking on this will show an expanded section where you can enter a multiple choice question and answers.
The screener question accomplishes basically the same thing as the additional qualification requirements but in a more stringent way.
First, write a question that gets at the characteristic you’re looking for. Suppose you were an international flight-and-hotel booking website, and you only wanted people who had vacationed abroad in the past year. You might frame it like this:
Question:
When was the last time you took an international trip for leisure?
Answers:
❍ Within the past 6 months
❍ 6 months to 1 year ago
❍ 1 to 2 years ago
❍ More than 2 years ago
You can then write up to 4 multiple-choice answers to the question. Users will only be able to choose ONE of the answers, so make sure they are mutually exclusive. Make sure also that your question can’t be answered more than one way – for example, no “choose all that apply” type questions.
Label each answer as “Accept” or “Reject.” Users that select a rejected answer will not be allowed to perform your test. Users that choose an accepted answer will continue on to participate in your usability testing study.
Using the screener question
You can use the screener question as a replacement for the qualification requirements box, or in addition to it.
In the former case, just leave the requirements box blank and fill out your screener question. To use both, just keep in mind that users will see the additional requirements first, then those that continue on will answer the screener question.
Here’s an example of how you might use both of these filtering features together to target a specific demographic:
The additional qualification requirement here states: “Must have traveled internationally for leisure within the past year.”
Users who qualify will then go on to provide their answer for the multiple-choice screener question:
Question:
When you travel, do you generally prefer to:
Answers:
❍ Engage in outdoor activities (e.g. hiking, climbing, adventuring)
❍ Relax and be pampered at resorts or spas
❍ Visit cultural sites and landmarks such as museums and temples
In the example above, the first and third answers are “accepted” – the researcher is less interested in luxury travelers, probably because they cater to customers matching the outdoorsy or cultural explorer types.
By combining both of these methods, you can learn more about participants and find the best people to give you feedback.
Of course, you can also put more than one request in the special requirements box, like:
Must be someone who uses public transit to commute to work at least once a week. You will be required to use and verify a real email address during this test.
We recommend no more than 1-3 unique requirements. The more you have, the more likely testers are to skip over parts and only respond to some of your requests. Additionally, it becomes increasingly hard to find matching testers in our pool when too many unique requirements are included.
Choose the important traits that you’re looking for carefully, and make sure they are necessary for getting useful feedback for your site or app. With the combination of our user testing features, we’ll help you find the right users and learn how to improve your UI.
Currently, using screener questions is limited to our Enterprise Plan. Additional qualification requirements are available to all plan levels.
