Tips for writing a usability test

To date at TryMyUI we have  reviewed many thousands of usability test videos based upon tests created by our customers so we’ve learned what types of questions and tasks elicit good test results.  Writing a good usability test is not something that is intuitive, but with a bit of thought and planning, it’s easy to accomplish.  The first question is: what are you trying to learn? Nominally, there are two general types of learnings:

1. Do users understand my value proposition?  Do they understand what services are provide, and are they enticed?
2. I want to see if users understand how to accomplish various tasks (these are presumably based upon the “use cases” or “user stories” that drove development)

Soliciting General Feedback

We already provide a quick impression test to solicit general feedback regarding the value proposition based upon a brief 10-15 second review of the website.  However, you can orient your entire test around soliciting general feedback by asking more specific questions about the services you are providing, such as:

• Tell us what conditions would prompt you to come to this website.  How would you use this in your day-day life?
• How do you currently  accomplish (a service that you provide on your website)
• Would you recommend this service to your friends – why or why not?

In general open-ended questions that elicit “essay” responses are better at eliciting general feedback.

Understanding How Users Handle Tasks

Sometimes you’re trying to understand why a particular feature is not being used, or else, your trying to create a new feature and want to test it out to see if users get it.  In these cases, you  are asking much more specific questions about approaching one or more tasks.  First, it’s important to put the tester in the correct frame of mind by describing a detailed scenario.  The details are important to fill in the story and stimulate the imagination.  So instead of saying “you’re looking for a book”, say “a close family member is having a birthday next week, and as he is an avid reader, you want to find him a book as a birthday present”.  These additional details engage the tester and make him a better proxy for your actual users.

Then, make sure you phrase the tasks to convey intent, not the mechanisms for handling that intent.  For example, don’t say “click on image gallery to see imges” – there’s very little that can be learned from that.  Instead, write your task to say “check the images – how do they influence your perception of the product”.  Now, you don’t tell them how to do something, but rather, what to do, and you phrase the question to solicit a more descriptive response.  Note that if you had asked “do the images help you understand the product?”, you may have gotten a “yes” or “no” answer, which is not terribly helpful in understanding the user’s thought process.