(Image source: randymora.com)
Recall: the retrieval of information stored in the memory.
We all rely on it every day – Where did I leave my phone charger? Which gas station has the lowest prices? What filename did I save that document under?
In usability research, we rely on recall to get feedback from everyday users, testers, and focus group subjects with surveys, direct questioning, and other methods – What did you think of the registration process? Did you have any trouble finding the contact information? How did the site make you feel?
So what’s wrong with recall?
A number of factors exert a major influence on recall accuracy. These include:
-
Attention – divided attention has been shown to seriously hamper the memory-encoding process that allows recall later on
-
Motivation – the greater the incentive for accuracy, the more reliable respondents’ recollections are likely to be
-
Primacy & recency effects – people tend to be better at remembering the first and last elements of a series than the middle elements
-
Interference – a delay between the encoding of a memory and the subsequent remembering, especially if filled with a separate task, impairs recall
-
Context & state dependency – items are recalled more reliably in the same environment or mental state in which they were initially encoded, and less in different ones
On top of all this, your respondents’ answers can be subject to various other manipulations depending on the research format, including social pressures from fellow testers leading to conformity, bandwagoning, or lying to hide what might be perceived as incompetence; inadvertent pressure from a test moderator to answer a certain way or confirm a given expectation; question-framing issues that influence responses (think ‘leading the witness’); and more.
The human mind is capable of endless shape-shifting, over-imagination, and self-deception – more than enough reasons to think twice about your focus group results.
So if waiting til the end of the session to ask your questions is such a feedback faux pas, what’s the solution?
Problems like division of attention or context dependency might be diminished by confining testers to a controlled environment, but at the cost of losing a genuine, true-to-life look at the user experience. Primacy, recency, and interference could be combated by asking testers questions at the end of each individual task, but at the cost of obstructing the natural flow of their journey through your website.
There is simply no way to eliminate all the distortions at once from a usability study that follows the format of:
1. Have testers use the website
2. Ask questions later
Replacing recall with occurrence
The only way to close the gap between what users do on your site and what they remember doing on your site, or what they say they’ve done on your site, is to look at their occurrent thoughts – that is, the thoughts that pass through their mind in the exact moment of those actions.
No recall is required – no flawed mental filters, no forgetting of middle elements or transitioning between mental states – just verbalization of the thought process as it happens. It’s something that comes naturally to us: most people talk to themselves, especially while alone. Remote usability testing allows you to tap into that natural instinct, listening to testers’ thoughts in real-time and getting the full, accurate, unadulterated picture.
Effectively, you’re looking into your user’s mind, understanding what they do and why at the most direct level. The pitfalls of relying on recall are avoided, and the problems that crop up when testers are influenced by judgmental peers, by hovering researchers, and by the wording of questions never arise. They have simply to open their mouths and let their minds flow out.
Watch: See an example
Occurrence is the key to getting the best feedback on your website. Are you relying on imperfect cognitive processes and delayed responses to understand your site, or are you peeking into your users’ heads?
