The ideal prototype usability testing sample size

Introduction 

Usability testing sample size is defined as the number of users that you compile to test your website or mobile app. When it comes to the ideal sample size for usability testing, we have always recommended that you start with 5 users. Today, we will discuss the factors that affect prototype usability testing sample size. Doing so will allow you to look at the bigger picture before conducting prototype usability testing. You will also get a good idea of the appropriate prototype usability testing sample size for your UX research.

 

Types of prototype usability testing

The appropriate prototype usability testing sample size depends on the type of testing you will be  conducting. Here are the different types of testing:

• Qualitative testing is focused on emotional response and long-form feedback. This isn’t about the numbers. The outcome focuses on the quality of the user experience without centering numbers and defined data points. This could translate to video reactions and feedback or written survey responses.
• Quantitative testing is focused on numbers, data points, and statistics gathered from users. The key here is statistical significance. This means gathering enough data to eliminate the drastic effects of standard deviations. It’s important to be sure that your prototype usability testing is conducted in the presence of an “N” number of users. The bigger this “N” is, the more successful the prototype usability testing is. This form of testing doesn’t really stand on its own in the usability testing space, as companies want to get a full view of  the user experience. However, pairing it with qualitative testing proves to be quite effective.
• Card sorting is a technique where  users are asked to organize topics, pages, and content into categories based on where it should be located on a website or app. This method also requires more users to have diverse opinions. It makes it easier to identify and validate common sorting patterns among the group. From there, you can establish the best navigation for your website or app.

These are the most common techniques used for prototype usability testing, with qualitative testing being the most common one. At TryMyUI, we like to combine qualitative with quantitative testing and data to give more well-rounded information and feedback for you to take back to the UX design team.

 

What is the ideal prototype usability testing sample size?

Good question, reader! There are always arguments about the right prototype usability testing sample size. The arguments advocate the participation of more users in prototype usability testing for better results and feedback. Let us look at these arguments:

 

• “A large website has traffic in millions.”

One may say that this is a really deep argument to advocate the increase in prototype usability testing sample size. However,  having millions of users visiting an app or website daily doesn’t mean that your UX is accessible or user-friendly. We have to focus on usability instead of the traffic. Suppose the user experience of an app or website is smooth. In that case, millions of people won’t face a problem using it if, for example, 5 to 6 representatives of your target audience don’t face any issues during usability testing. On the other hand, if something as simple as the sign-in or contact us button doesn’t respond, the traffic may enter your site or app but will not convert. It is always QUALITY> QUANTITY in usability testing.

 

• “Hundreds of options on a website require more users to test them.”

Let’s consider an example where the usability testing sample size is ten and the number of options or features on a website are in the hundreds. You can’t expect all users to check out the assigned features in absolute detail. They will definitely get tired or fumble, and the usability testing results won’t be as accurate. The best approach to this is to set up usability testing in a way where you evaluate specific sets of tasks or certain user journeys. You don’t want to create a testing environment where users get testing fatigue. Therefore, it’s better to run several tests on small sets of tasks to make things easier.

 

• “Diverse target audience.”

Another fact that is valid but only if the target audience contains different groups that will show different behaviors. A good example of this is an online marketplace where buyers and sellers both have to interact with the website or app. In such cases, you can increase the number of users by picking up users from both categories. However, it’s important to be sure that your users represent your target audience properly. This is generally easier to manage with well-crafted user personas for your prototype usability testing sample.

 

Conclusion 

After reading this article, we hope that you will have a clear idea of how many users you should include in your prototype usability testing sample size. Remember, quality>quantity in usability testing.

 

---