Usability Testing in a Social Context

Dana Chisnell gave a great talk today at Bolt Peters User Research Friday. Her observation was that almost all activities on the web occur in a social context that spills out far beyond the particular product being tested.

Studies that don’t account for the social context are broken

Dana put together a study for a financial company that was providing retirement planning for users. When she put her panel together she realized that it was broken! Specifically she realized that the decisions users make in regard to their retirement planning are made with the help of family members and other people. Any testing that put users in an isolated environment would necessarily be a bad proxy for reality.

The right way in retrospect to conduct this study would have been to ask the recruits who they relied on for financial decision and to invite those members to the sessions.

The Nature of Online is Social.

Social means anything that someone does that affects the behavior of someone else. It involves relationships.

Scale is a game changer.

Task doesn’t mean what you think it means.

Specific isolated tasks in scenarios are not reality. What we need to be thinking about is the high level generalized intent across a broad social network.

Satisfaction may now equal user control.

Users are continuously designing your UI.

Social is there already.

Social is not a sauce that you add to your design. It is already there. You need to find it. You need to properly accommodate when you do find it. And you can’t test social experiences with testers in isolation.

How do we do research on social products?

• Spend a lot of time with users ahead of time. Giving them homework.
• Get recruits to show you their relationships.
• Get recruits to tell you how they met the people in their social networks.
• Try to understand the social context that the proposition presented by the product will involve even if the product does not yet well accommodate that social context.

Usability research is more than finding bugs in flows.

Usability testing is about determining design direction by getting perspective on the broad context in which their user acts. It is about forming cogent hypotheses about how products fit within real people’s lives.

Commodity usability testing is too controlled and too contained.