Your team has been working on the prototypes for a new product, and it’s time to do some user testing.
You probably know that you have a choice between on-site testing, where you recruit your own users and bring them in for a moderated session, and remote user testing, where you collect unmoderated sessions through online platforms (like TryMyUI).
Read more: What can user testing do?
On-site vs remote user testing: Which is better?
Many people may think of remote, unmoderated user testing as the second-best alternative – the one to use when you don’t have the time or budget to do “real” user testing.
Why? It’s cheaper, it’s faster, it’s more hands-off; it has to be the lesser option, right? Isn’t remote user testing the “fast food” of UX research?
In fact, there are a number of reasons why the data collected in remote user testing is higher-quality than from moderated, on-site user testing.
1. Your devices are not your users’ devices
A corollary to the UX maxim that you are not your users. When you bring people in to test, they’re using your devices, not their own familiar one.
For one thing, this introduces another dependent variable into your study that will muddy the findings: are users’ issues attributable to your designs, or to unfamiliarity with the device, browser, or operating system?
Keyboard feel, screen size, mouse speed, scroll speed, button feedback, and gesture meanings all vary from phone to phone and computer to computer. In remote user testing, people test at home on the devices they know, so none of these variables will throw them off as they take your test.
It also matters that your devices are probably not representative of the average user’s. Here’s a demonstrative anecdote from designer Kellen Styler, found in this great article:
When we initially redesigned Infor’s UI paradigm, we emphasized the importance of negative space — more precisely, white space. Most enterprise apps are cluttered with rows upon rows of data and fields. We championed a cleaner approach — one that gave information a chance to breathe.
But soon after we deployed the new paradigm, we heard from users. They told us that while working on decade-old monitors for eight hours a day, five days a week, the white screens were starting to strain their eyes — that’s an experience we never would have replicated in our work space. We really aren’t our users.
2. Under the microscope, people aren’t themselves
Put someone in a lab with a bunch of people they don’t know, prod them with questions while they explore your product, and you can be sure the results won’t closely resemble a real user session.
Just like in school, when you studied all semester only to forget half of it under the pressure of the exam, testing affects the way people act and think. Even though it’s really the product that’s being tested, most users can’t help but feel the pressure to perform. The very fact that they’re being observed makes people act differently.
Browsing the web or playing around on mobile apps is usually a solitary, private activity. Moving that activity into a user testing lab fundamentally changes it. Do you change your behavior on your computer or phone when someone walks up behind you?
Of course, even remote user testing suffers from some of these issues. After all, the session is still being recorded for later viewing. But the environment itself is genuine, and that helps users feel much less “under the microscope.” That’s when you get more honest, untainted feedback.
3. The name’s bond… Human bond
When tests are being run by a moderator, the natural realities of human bonding can exert a major influence on the accuracy of the results. People like to say what others want them to hear, and often don’t like to be critical directly to someone’s face.
The problem is, criticism is one of the most valuable things you can get out of user testing. Compliments are nice, but criticism helps you improve your designs. Moderation makes it more difficult to strike on those hard truths.
Remote user testing, on the other hand, takes the human element out of the equation. Users read tasks and questions directly from the software, and aren’t troubled by concerns over hurting a new human bond.
That makes it easier to be harsh, which ultimately is what you want from your research.
Rethinking user testing options
Remote user testing isn’t just a nice alternative, it’s a great way to get reliable, high-quality usability feedback. Even when budget isn’t the top priority, there are many reasons to choose it over on-site testing (and with the same amount of money, you can collect far more results with remote testing).
Online unmoderated remote user testing represents a definitive step forward in the development of accurate research methods for improving the user experience.
