5 reasons you should say farewell to focus groups

A number of years ago, British Airways needed to find out what customers wanted. They were adding mini refrigerators to first class seating sections so passengers could help themselves to a snack during long overnight flights, and needed to find out exactly what kind of snacks their passengers would be interested in. So they put together a few focus groups.

Fruit, said the focus groups. Salads. And so the mini fridges were filled with healthy fruits and salads. But one longtime flight attendant objected – after years spent waiting on airline passengers and observing what they wanted in practice, she insisted that a few chocolates and cakes be stocked too. At the end of that flight, the chocolates and cakes were gone, and fruits and salads still filled the fridges.

5 reasons focus groups are so faulty

Why are focus groups so prone to serious error? Because they rely on asking rather than observing, and that extra layer adds all sorts of complications:

1. People are very good at answering questions, even when they don’t know the answer.

This has been shown time and again – for example, in Jimmy Kimmel’s Lie Witness News segments, in which interviewees off the street are presented with a piece of false news and asked for their response. Unwilling to reveal that they don’t know, people concoct blatant lies to tell the interviewer. One man, when asked what he thought of Landon Donovan’s play in the 2014 World Cup (Donovan was cut from the team before the tournament), said it was “definitely pretty good; he took one to the…nose, was it? and he kept playing.”

2. People don’t know why they like the things they like.

Often, our likes and preferences have reasons that we don’t really understand. We know we like something, but we can’t really say why. But once again, when pressed for explanations, people would rather make one up than admit that they don’t know – after all, aren’t we supposed to know why we like the things we like?

3. People aren’t in touch with their subconscious.

This accounts for a good chunk of the reason we don’t know why we like things, and also why we’re bad at predicting our own behavior in hypothetical situations. Much of this information lays within our subconscious, influencing our decisions in the moment without us realizing it. We just don’t have access to it. It’s easy to think you’d want salads in the fridge when you’re not half-awake thousands of feet off the ground in the middle of the night.

4. People want to make themselves look good.

We aren’t likely to give an embarrassing answer, or one that paints us in a bad light or makes us look foolish, when surrounded by other people. We generally have a pretty good handle on the kind of answers that others will respond positively to, or which ones they won’t like, and then make sure to pick the most popular or inoffensive answers. Self-deception plays a big part, too: people are often liable to talk about themselves in ways that they wish were true, rather than what is really true, painting their personality and character in an idealized manner.

5. People want to please the moderator.

Besides their peers, focus group participants also want to make their moderator happy. This means guessing at the answers he or she is looking for, and returning those, instead of more honest replies. We don’t want to be the one giving the bad news, so we play to the hopes and expectations of the person asking the questions. In addition, people may sometimes believe that their own answer is wrong if the questioner, who is assumed to know more about the product, seems to be expecting a different response.

And then there are the problems with recall, the act of retrieving information from the memory after the fact – including divided attention, primacy and recency effects, time delay, context dependency, and more. All of these issues come into play when that extra layer is added between the focus group’s conductor and the participants.

Focus groups are expensive

Cost of recruiting participants. Cost of renting a facility. Cost of transporting participants to that facility. Cost of buying refreshments and meals. Cost of hiring a moderator. Cost of videotaping and note-taking. Cost of paying the participants for their time.

On average, these typically add up to at least $4,000, and frequently more. Market Trends Research says at least two focus groups are necessary to get useful results. So, if you ran focus groups on just one topic per quarter, that’s a low-end cost of $32,000 per year.

Beyond the price tag, there are the costs of time and effort – personnel must commit a good deal of energy to planning and developing the focus group, and then observing it while it takes place. All this for 2 hours’ worth of contorted, questionable feedback.

Now, say you wanted to hire the same number of people to get the same amount of feedback, without the failings of the focus group format and the significant investment of personnel time and energy. Using remote usability testing, you could cut your budget by more than 90%, eliminate the hassle of recruiting, transporting, and feeding participants, and save personnel time too. You could even significantly increase your number of testers, or what you’re getting out of them, or how frequently you test, while still spending just a fraction of what you would be paying for focus groups.

Getting closer to users

Focus groups create a superficial environment that returns superficial advice. To really understand what users want, you have to strip away that extra layer of asking that creates a barrier between you and the user and rely on observation, inobtrusive and neutral.

Get your participants out of the focus group rooms, away from the one-way mirrors, where they can act, judge, and respond independently of others’ influence. Freeing your participants from their social and psychological traps is the best thing you could do for your feedback.