Usability testing is a great way to assess the user experience and predict problems that the user may face within your design. When conducting usability testing, a sampling of your target audience navigates your design and provides actionable feedback on the user experience. This means you must be ready to take any positive critical analysis and make room for any corrections or amendments in the original design based on the results.
However, how do you know that you’ve got the right usability testing sample size? We’re here to offer you some advice on the subject.
Choosing an appropriate usability testing sample size
The criterion for choosing the participants and determining the usability testing sample size should be based on getting statistically significant data to support any changes you deem necessary to make. James Lewis and Sauro came up with a helpful approach to deciding the number of participants. It involves some size calculations, availability, and discoverability of the problems.
Usability testing sample size: The number 5
When considering a possible minimum sample size for a usability test, 5 is a reasonable number to start with.
Problem discoverability refers to the likelihood that there will be at least one participant encountering the problem during usability testing. The average value of problem discoverability for a basic set of projects is calculated to be p=0.31, according to Nielsen and Landauer. If there are some discoverable problems, 5 users would be expected to encounter 85% of those usability problems.
However, Virzi found the value to be between 0.32 and 0.42. It infers that you could get 80% of the problems detected with the help of 4 to 5 participants.
Therefore, to get optimal feedback on your user experience, a minimum usability testing sample size of 5 is all you need. If needed, you can set up more usability testing sessions so that you can obtain more accurate results.
Sign up for a free trial to launch your first usability study with 5 users!
How discoverable are your design’s problems?
To determine the problem discoverability of your design, go for a 2 participant set in the first round. Observe the number of problems and the overlap in the discovered problems. Also, take note of any challenges your users have when navigating the usability test.
Adjust the designs according to the results obtained for the first time. Make adjustments to the test for the next round, so tasks and scenarios are clearer. Repeat the test with at least double the number of participants.
It will help you with more keen observations, as you’re going from a smaller number of participants to more. This will increase the opportunity for problem discoverability.
What factors increase discoverability and make a small usability testing sample size work?
The factors that increase the discoverability and help in working with a small usability testing sample size are as follows:
- The test observers shall be multiple, keen, and expert. They must excel in their field so that any shortcomings may not be missed.
- The designs shall be new and innovative rather than relying on conventional designs.
- The participants shall be new to the experience so that they come around to the problem quickly. An experienced user will find a way around the problem and create a standard for the user.
- A representative and homogenous set of participants will be required.
- There shall be a proper structure and order of the tasks being performed within usability testing.
- There shall be both simple and complex tasks to assess the compatibility and complexity of the project.
- Go for rather small and more precise tests rather than long ones.
Conclusion
All in all, having an appropriate usability testing sample size is important to the efficacy of the user testing process and the data collected. Follow the guidelines above and find out for yourself!
Get started for FREE with a 5-respondent usability test: