If you want to understand how a lion hunts, don’t go to the zoo. Go to the jungle.
There are a lot of ways to do usability testing, and each has its own merits. But in terms of getting the most reliable, accurate feedback for your website or app, unmoderated remote user testing has some distinct advantages over the alternatives.
Unmoderated remote user testing vs on-site
On-site user testing, in addition to being much more expensive, also produces less reliable results than remote testing. On-site testing is very artificial: the users are in an unfamiliar environment, using your devices instead of their own familiar ones, with researchers watching over their shoulder and interfering directly in their experience.
There’s no way for them to slip into “normal mode” – browsing around the site naturally as if in real life, like someone visiting of their own volition and to their own ends. They are in “testing mode,” and constantly aware of it (something that is not helped by the presence of a moderator).
More: The problem with focus groups and on-site testing
With remote user testing, the users perform the test in a typical, representative environment using their own device on their own time. While some elements of the test are unavoidably still artificial, it comes much closer to replicating a genuine user experience. That means the feedback you get will be more relevant to how real people are using your site “in the wild.” The most expressive, clear, and eloquent feedback is still useless to you if it doesn’t reflect the reality of your product’s user experience.
In addition, with remote user testing you have access to an enormous and diverse pool of users all over the world. You are not limited to users in your local geographic area, unlike with on-site testing. With such a wide pool to draw from, you can get much closer to finding people in your exact target audience, whose feedback is far more valuable to you.
Learn more:
UX Sprint: Iterative usability testing to sculpt the perfect UX
Unmoderated remote user testing vs moderated remote
Moderated remote user testing avoids some of the issues with on-site user testing, but having a moderator, while it may seem appealing, can decrease the value and accuracy of the results of your study.
People have a natural tendency to try and please others and make themselves look good, and this trait subconsciously comes into play when users interact with a moderator. In trying to please the moderator and do and say the right things, users sometimes stop representing their true experience and instead say what they think the moderator wants to hear.
Removing the moderator from the picture makes it easier for the user to be brutally honest because that human connection is no longer interfering. There’s no name or face for the user to empathize with; so when they voice their thoughts and reactions, it’s more like talking to themselves than to another individual.
More: Common user testing questions and their answers
Moreover, moderating an on-site user test is very hard to do – especially if you are close to the product being tested. One key to professional moderation is to keep your mouth shut most of the time. You have to let the user stumble around and find his way without jumping in to save him. This is often very difficult to do for owners, developers, or designers of a website.
More reliable usability insights
It’s tempting to want more control over your tests, and moderation seems to offer that additional control and oversight. But in the end, it can be more harm than it is worth; even without the artificial environment created by on-site testing, moderation of remote user testing is likely to distort user feedback and make the results less reliable.
The key to great usability testing is not control. It is authenticity. Unmoderated remote user testing comes the closest to realizing a genuine user experience that can provide relevant and reliable insights.
