The ideal prototype usability testing sample size

Usability testing sample size is defined as the number of users that sit together to test your website or app. While there are no specific figures that are known to be the ideal sample size for usability testing, we have always recommended that you start with 5 users. Today, we will discuss the factors that affect prototype usability testing sample size. Doing so will allow you to look at the bigger picture before conducting prototype usability testing. You will also get a good idea of the appropriate prototype usability testing sample size for your UX research.

Types of prototype usability testing

The appropriate prototype usability testing sample size depends on the type of testing you will be conducting. Here are the different types of prototype usability testing:

• Qualitative testing is focused on emotional response and long-form feedback. This isn’t about the numbers. The outcome focuses on the quality of the user experience without centering numbers and defined data points. This could translate to video reactions and feedback, interviews, or written survey responses.
• Quantitative testing is focused on numbers, data points, and statistics gathered from users. The key here is statistical significance. This means gathering enough data to eliminate the drastic effects of standard deviations. It’s important to be sure that your prototype usability testing is conducted in the presence of an “N” number of users. The bigger this “N” is, the more successful the prototype usability testing is. This form of testing doesn’t really stand on its own in the usability testing space, as companies want to get a full view of the user experience. However, pairing it with qualitative testing proves to be quite effective.
• Card sorting is a technique where users are asked to organize topics, pages, and content into categories based on where it should be located on a website or app. This method also requires more users to have diverse opinions. It makes it easier to identify and validate common sorting patterns among the group. From there, you can establish the best navigation for your website or app. It’s important to note that this is featured very early on in the design process.

These are the most common techniques used for prototype usability testing, with qualitative testing being the most common one. At Trymata, we like to combine qualitative with quantitative testing and data to give more well-rounded information and feedback for you to take back to the UX design team.

What is the right prototype usability testing sample size?

Good question, reader! There are always arguments about the right prototype usability testing sample size. The arguments advocate the participation of more users in prototype usability testing for better results and feedback. Let us look at these arguments:

• “A large website has traffic in millions.” One may say that this is a really deep argument to advocate the increase in prototype usability testing sample size. However, having millions of users visiting an app or website daily doesn’t mean that your UX is accessible or user-friendly. We have to focus on usability instead of the traffic. Suppose the user experience of an app or website is smooth. In that case, millions of people won’t face a problem using it if, for example, 5 to 6 representatives of your target audience don’t face any issues during usability testing. On the other hand, if something as simple as the sign-in or contact us button doesn’t respond, the traffic may enter your site or app but will not convert. It is always QUALITY > QUANTITY in usability testing.
• <strong“Hundreds of options on a website require more users to test them.” This argument is somewhat valid. Let us consider an example where the usability testing sample size is ten and the number of options or features on a website are in the hundreds. You can’t expect all users to check out the assigned features in absolute detail. They will definitely get tired or fumble, and the usability testing results won’t be as accurate. So, there are two options in such a situation. The first is to divide the hundreds of tasks into different batches. Reduce the number of features to be inspected and increase the number of prototype usability testing sessions. This option will make it easier on users to focus their feedback. As well, it will make it easier for your team to work on the main priorities in the design process first then move on to less significant features. The second option is obvious: increase the number of users.</strong
• “Diverse target audience.” Another fact that is valid but only if the target audience contains different groups that will show different behaviors. A good example of this is an online marketplace where buyers and sellers both have to interact with the website or app. In such cases, you can increase the number of users by picking up users from both categories. However, it’s important to be sure that your users represent your target audience properly. This is generally easier to manage with well-crafted user personas for your prototype usability testing sample.

Conclusion

After reading this article, we hope that you will have a clear idea of how many users you should include in your prototype usability testing sample size. Remember, quality>quantity in usability testing.

Related reading:

• Writing tasks for prototype usability testing
• 4 usability testing sample size mistakes to avoid
• Prototype usability testing: Top 8 mistakes to avoid
• Concept testing, prototyping, and product roadmaps
• Prototype testing questions: Your product’s future is in your consumers’ hands