A good usability test will almost always reveal flaws in the UX of your digital products – areas that are frustrating your users, failing to convert, or just in need of general experiential improvements. With the right users and a good task script, you’ll receive a wealth of qualitative insights into the strengths and weaknesses of your designs.
To complement these qualitative insights, a number of psychometric questionnaires have been created for the purpose of quantifying and scoring the user experience.
Not only do numerical scores help UX researchers measure improvements over time, they can also help to quickly identify which users in a study had a good or bad experience.
On top of that, different questionnaire models offer different score breakdowns or vectors, illustrating specifically which aspects of the user experience are better or worse on your website or app.
What is the SUPR-Q?
The Standardized User Experience Percentile Rank Questionnaire, or SUPR-Q, is one of several scoring models we’ve implemented at Trymata to help UX researchers holistically understand and measure product performance.
SUPR-Q is a short but powerful 8-item questionnaire designed by Jeff Sauro of MeasuringU. It differentiates itself from other models with its 4-factor analysis (measuring the 4 sub-scores of usability, appearance, trust, and loyalty).
The SUPR-Q survey consists of 8 questions, shown below (with the relevant score vector indicated alongside). Users respond to the first 7 questions using a 5-point Likert scale, where 1 represents “Strongly disagree” and 5 represents “Strongly agree.”
- The website is easy to use. (Usability)
- It is easy to navigate within the website. (Usability)
- I feel comfortable purchasing from the website. (Trust)
- I feel confident conducting business on the website. (Trust)
- I will likely return to the website in the future. (Loyalty)
- I find the website to be attractive. (Appearance)
- The website has a clean and simple presentation. (Appearance)
- How likely are you to recommend this website to a friend or colleague? (Loyalty)
Note that the final question of the survey is in fact the Net Promoter Score (NPS) question. Unlike questions 1-7, the response scale for question 8 goes from 0 (“Not at all likely”) to 10 (“Extremely likely”).
Like all psychometrics, the SUPR-Q is administered after users have completed a task-based usability testing session on a website, app, or other digital product.
Users’ responses to all 8 questions are then turned into a score ranging from 0-100, encapsulating at a glance how they felt about their experience on the website being tested. By averaging all your users’ scores together, you get an overall SUPR-Q score for your entire usability testing study.
For example, if you had 3 people test your website and their individual scores were 45, 70, and 80, your overall SUPR-Q score for that study would be a 65. Of course, you’d want to pay special attention to the feedback of the user who gave it a 45 – representing a much worse experience than the 70 and 80.
Also remember, of course, that 3 users is not a sufficient sample size! It just makes for easy math in a hypothetical. Read more about usability testing sample size here >
A psychometric with 2 versions
SUPR-Q actually has two variants: an ecommerce version, which is shown in the list of questions above, and a non-commercial version.
As you might have guessed, the difference between the two is a matter of your website’s purpose. If you’re testing a website experience that does not involve monetary transactions, you would use the non-commercial SUPR-Q variant.
This version of the SUPR-Q replaces questions 3 and 4 in the list above with the following items:
- The information on the website is credible. (Trust)
- The information on the website is trustworthy. (Trust)
What makes SUPR-Q unique?
When assessing the quality of a website’s user experience, why should practitioners consider using the SUPR-Q over other psychometrics?
In short, the SUPR-Q is more concerned with a more holistic, broader depiction of users’ experience of and attitudes towards a product, rather than just web usability alone.
This is why it includes scoring vectors for usability, appearance, trust, and loyalty. All four are important components of how a user feels about a digital platform, and thus SUPR-Q incorporates all of these to reflect that.
Want to get a SUPR-Q score for your website or app? Get started for free below:
Comparing SUPR-Q to other psychometrics
The SUPR-Q is just one of a number of psychometric survey models available for quantifying users’ experiences and perceptions about your digital products. Below is a brief roundup of other alternatives, and how they compare to the SUPR-Q.
SUS
The System Usability Scale (SUS) is the most widespread quantitative measurement in UX and usability testing, and for good reason. It generates a highly reliable rating of your product’s usability. Unlike the SUPR-Q, though, it doesn’t offer sub-scores like loyalty and trust that measure broader aspects of the user’s entire experience.
PSSUQ
The Post-Study System Usability Questionnaire (PSSUQ) is similarly great for testing the usability of a website, and is more detailed than the SUS, offering 3 sub-scores in addition to the overall score. Like the SUS, however, it does not attempt to measure other more holistic aspects of the user’s feelings about a website.
ALFQ
Trymata’s own Adoption Likelihood Factors Questionnaire and the SUPR-Q are sister psychometrics in their intentionality. Both measure more than just usability, tapping into the heart of the user’s more abstract thoughts about a product – such as trust and credibility.
One notable difference is the development stage each is best suited for: SUPR-Q is better for established brands that users are familiar with, and whose products have already been through a few design iterations and releases. Meanwhile, ALFQ is built for UX & design teams on the opposite end of the spectrum who are just entering the market, and looking to better understand what fresh-eyed users with no familiarity will think of their platform.
SRS
The Survey Respondent Scale is the brainchild of a partnership between Trymata, MeasuringU, and QuestionPro to address the cognitive stress and fatigue of survey responses on testers. SRS is an extremely specialized metric that is meant more for marketing than UX/design.
What’s your website’s score? Pick from any of the above and find out today:
Who should use the SUPR-Q?
With SUPR-Q, we recommend that a company who has already run usability tests in the past and has resolved glaring usability issues think about utilizing the added psychometric capabilities of SUPR-Q to understand and retain customers.
Ecommerce businesses are a great candidate for using SUPR-Q. The goal of ecommerce UX is not only to make just a single sale, but to retain customers and keep them coming back. Getting the NPS results and credibility rating of your site is ideal for great ecommerce UX, which more than most types of UX, relies on brand trust and passive marketing.
Enabling the SUPR-Q for your usability test
To enable the SUPR-Q psychometric, simply proceed to Step 4 (the “Post-test”) of the Trymata “Create a New Test” flow, scrolling past the survey questions, and check the “Include system usability questionnaire results with my test” box. SUPR-Q will be displayed in the drop-down menu beneath.
You will notice that the NPS box that appears under that menu is automatically filled-in, as it is built-in to the SUPR-Q psychometric.
Note: SUPR-Q and the NPS are also great to use in tandem with A/B testing, for comparing quantitative outcomes of both websites and mobile apps!
Get your first 5 user test results FREE! Includes SUPR-Q scores.
MeasuringU and the SUPR-Q license
The composite scoring for SUPR-Q, unlike other psychometrics, is based on quarterly updates provided by MeasuringU.com, from whom you can purchase a license and import data into the most up-to-date calculation. The SUPR-Q database and algorithm is updated with data from between 5 and 20 new websites. But even without the license, the SUPR-Q is still an accurate and insightful composite of the credibility, desirability, usability of your product.