The following is a guest post by Cherryl Pereira, Head of Content at Chisel Labs
Gathering metrics can be a difficult task, especially if you are new to the world of usability testing.
Reporting is an important part of usability testing because it allows for transparency into your project and offers insights that might not have been achievable otherwise.
This blog post will teach you how to report more effectively with different metrics so you don’t have to worry about being lost in reporting land.
What is usability testing?
Benefits of usability testing
Usability testing metrics that will help you report more effectively
Criteria for questionnaires to keep in mind
Conclusion
What is usability testing?
Usability testing is a technique used to determine how well users can use something you created. It is an excellent way to find out if your product will be as easy and effective as possible for the end-user.
Many different techniques can help answer this question, including:
- A/B testing (comparing two versions of web pages or other digital products)
- Usability interviews (asking respondents questions about their experience using your product)
- Surveys (online or paper-based – either self-completed or administered by someone else), and;
- Direct observation/usability tests (watching real people interact with your site).
Read More: A/B Testing vs Usability Testing: Which, When, Why, and How?
Benefits of usability testing
Usability testing provides a chance to have your website tested by real users. It is the cheapest way of finding out why people are not able to use your website properly, what they do and don’t like about it etc.
Another benefit of usability testing is that it can be done at any stage of website development. It doesn’t matter if you have a rough design draft or even no design, usability testing is still relevant and will help you improve your end product.
Usability testing metrics that will help you report more effectively
Task time:
The first thing to track is how long it takes users to complete tasks or specific activities on your website.
Task success rate:
This metric shows the percentage of time people can accomplish their goals on a given site. For example, if you had five participants who were supposed to buy tickets for an event and only one was successful in doing so then that’s a 20% success rate (which probably isn’t good).
Time spent at each page:
The average amount of time users spend looking at each page can help you understand which pages might need work from designers and developers. It can also show any issues with a particular flow through the site.
The number of clicks per task:
The number of times a user before completing a task can show you where they’re getting stuck or which pages might need more attention from designers and developers.
Time on task:
The average amount of time users spend using a site is also an important metric to know. It gives insight into whether people can complete the tasks they set out for themselves within your design within a 15-20 minute window.
A number of page visits per session:
Averages at how many times participants visit each site during their sessions will help you understand what content people want to see, when they want it, and how quickly they can find that information once on a specific destination. This number may vary depending upon the length of the user testing session, so make sure by checking with participants if they are testing for more or less time.
A number of page errors per session:
This is another important metric to know as it gives insight into how well your design is guiding users through the tasks at hand, whether that be checking out a product detail page or signing up for an account. It’s particularly helpful in identifying places where you might need additional help messages or tutorials since this will show you where people are having difficulty completing their desired task within your site.
Task success rates/Number of participants who were successful on the first try:
Knowing how many participants completed each task successfully on their first attempt can give you valuable insight into what content and functionality may not have been made clear enough during the planning phase so that usability testing results can better inform design decisions.
The number of errors per participant:
Identifying which types of tasks require repeated attempts by individual participants will help you determine where additional messaging may need to be added before a user has to abandon an action due to confusion about what steps must happen in the correct order, such as when completing registration forms or filling out payment information during checkout processes.
Unique insights from each test participant’s feedback:
Being able to identify patterns between all unique perspectives gathered from each participant that you test will allow your team to determine where a design may be confusing for users of different backgrounds, which can include members of populations with disabilities.
The number of total errors per task:
A high number on the aggregate level indicates potential problem areas in an interface or process flow without offering insight into what types of mistakes were made most frequently by participants on any given day. For example, if “error” metrics show that more than half of all shopping cart checkout forms submitted throughout testing have at least one error, it’s likely there is a usability issue within the checkout form itself. If this same metric showed up for a search results page, it would be indicative of an issue with the design or arrangement of those items on that particular screen.
Read More: What user testing metrics should you be measuring?
Criteria for questionnaires to keep in mind
Questionnaires are a type of qualitative research measurement. We convert the data into a numerical score and analyze it accordingly, using a scale of 1 to 5 for all questions.
It helps us determine whether or not you are meeting your goals. It also determines if the research is delivering value for money spent.
NPS
The first criteria is Net promoter Score (NPS). Many marketing teams use NPS to measure the likelihood that a customer will recommend their company or product.
You can use it across marketing channels. It’s also easy to build a survey to track your business progress over time.
NPS is a metric that calculates the likelihood of recommending a business/product to others. It utilizes a single-question survey format with answers that range from 0 (not likely at all) to 100 (definitely yes). NPS works as an effective way for measuring brand, product & company loyalty based on standard questions in questionnaire formats.
It is also one of the most popular metrics among marketers, because it has been proven effective repeatedly in several different industries. Some of these include software testing and mobile app development, BFSI (Banking Financial Services & Insurance), IT, retail, and so on.
This metric is quite accurate for measuring loyalty programs as well. Customers who are likely to refer others tend to spend more money on products from companies they trust; thus improving revenue per user by increasing retention rates.
SUPRQ
The second criteria is the Standardized User Experience Percentile Rank Questionnaire (SUPRQ) developed by the NN group. SUPRQ is a customer loyalty metric that measures the likelihood of customers to recommend products or services.
SS-Net Promoter
The third criteria, developed in collaboration with Parature, was Survey Standardized Net Promoter Score (SS-Net Promoter). SS-Net Promoter calculates the percentage of people who would refer your product after using it.
This way you can easily calculate what % of users are likely to buy again. You can use this number as an indicator for service quality management solution selection. This is especially useful when considering omni-channel touch-points.
SS-NetPromoter also gives companies insights into why their clients feel good about them or not so much, helping improve long-term business performance by understanding current problems & finding solutions for negative impacts on brand loyalty.
CSAT
Customer satisfaction survey (CSAT) is a tool to measure the satisfaction of customers after using your product or service. This metric helps companies understand their current performance. It gives information about how to improve customer experience. You also gain insights into what’s important for users.
SUS
System Usability Scale (SUS) is a tool you use to measure the usability of software user interfaces. John Brooke developed it in 1986 after realizing that there was no way to measure how users feel about an interface. Therefore, he created his scale which consists of ten questions with five being positive and five negative ones. Example of SUS: “I felt very confident using this system.”
SUS is a responsive metric and it doesn’t require any previous knowledge about usability testing. The scale also has an open-ended question. That means users can express their thoughts on how they feel or what’s missing in the interface. This will give more information for companies to improve even further if necessary.
Conclusion
As we discussed, it’s important that you keep in mind that each organization will have different goals and priorities when it comes to usability testing.
You should use metrics tailored to your specific needs, so you don’t waste time conducting tests with little value or accuracy.
Cherryl Pereira is the Head of Content at Chisel. Chisel Labs is a premiere agile product management software company that brings together roadmapping, team alignment, and customer connection.
Tim Rotolo
Shruti Goli