Designing a user test is a bit like designing a website: inevitably, users find every way you never thought of to misinterpret and misuse what’s put before them. Most of the time, though, that isn’t their fault – in fact, it’s probably yours.
Test designers frequently make oversights and errors that cripple the ability of their tests to gather useful feedback and show the interactions they’re looking for. A poorly designed usability test will seriously impact the results, including
(1) users’ ability to correctly carry out the tasks in your user flow, and
(2) the amount of insight you get into real usability issues with your application.
When your testers don’t understand the tasks, aren’t prepared to approach them properly, or misinterpret the instructions, the returns on your research aren’t being optimized and you may well not learn what you hoped to about your interface.
Here are 5 tips for designing a test that will avoid common errors and maximize the information and insight you get out of your usability research.
1. Create an engaging, immersive scenario
To get the best feedback from your user tests, the tester should be immersed as much as possible in the mindset of someone using your product in a real-life context. To achieve this, the scenario you design for your testers should be detailed and realistic. Write a scenario that sounds like a story, not a set of instructions.
A poor scenario, for example might sound like this: “You need to buy renter’s insurance and want to explore your options.” A great scenario would be, “Your friend just paid thousands of dollars to repair damage caused when a guest accidentally caused a kitchen fire. You have guests over often and want to be covered in case something like this happens.”
This allows the tester to dive into the experience and explore your product with the perspective of a real-life user. If you’ve done your demographic selection well, a detailed scenario like this one may be pretty identifiable or at least imaginable for your testers.
2. Test user impressions
Your landing page is a billboard for your brand. A good way to measure whether that billboard is projecting the right message is an impression test.
The way we do this is to show testers the landing page for up to 15 seconds (the average amount of time visitors spend on a webpage) and then ask users to talk about words they remember from the site, its general feel, products or services offered, and overall impressions. It’s a great way to gauge whether visitors are receiving the message you’re trying to send, and understand the visual and verbal cues that shaped, guided, or obstructed their understanding of what the site is about.
Making the right impression on new visitors can be the difference between losing or keeping them. Including an impression test at the beginning of your user test also helps to orient the user and gives them the chance to understand what the site is for before jumping into the tasks.
Read more: Understanding users’ first impressions
3. Design a journey, not a to-do list
Remember that although your tester will be instructed through your application by clearly delineated tasks, it should mimic a real-life user journey as much as possible. Think about how someone visiting the site might progress through different pages and phases. You probably have a pretty good idea of how visitors move through your site; use that knowledge to lay out a smooth and natural journey that a real user might actually take.
A task doesn’t always have to involve a concrete action like signing up or making a purchase. Sometimes asking the user to browse around or explore for a bit, whether in a specific section of the website or across the whole thing, makes sense. And leaving them to navigate and make decisions for themselves can tell you a lot about the way people process what they see and how they’re approaching the content of your site.
4. Write for first-time visitors
Being so intimately familiar with your own product, it can be easy to accidentally talk about it in a way that doesn’t make sense to new users. Remember that your testers probably know nothing about your product and how it works. Steer clear of industry terms or brand-speak; these are likely to just confuse your testers, and can sometimes result in a tester mistakenly believing he has completed a task and moving on without actually doing it at all.
When you write your tasks, use simple and generic wording and think hard about what will and will not make sense to first-time visitors.
5. Don’t lead the witness
Your word choice when writing tasks should avoid key words used on your application – tell testers the end goal to be achieved, not the action to take. For example: “Save a product you like so you can come back to it later” instead of “Add an item you like to your wishlist.” Not only will this show you how easily the user identifies and locates the way to complete the task, it might reveal a totally different way they may think about achieving that goal – maybe their first instinct is to bookmark it on their browser.
Read more: Writing a usability testing scenario and tasks
The bottom line is, if the words you use are the same as the words that appear on your site, the interactions you see in your results won’t be genuine; you’re effectively giving the tester the ‘answer.’ Refrain from leading the witness, and you’ll learn a lot more.
There are many more ways to optimize your research, but with these 5 you will avoid the main potholes and get valuable, relevant usability feedback. Happy testing!
Shruti Goli