Thursday July 9, 2009
Why You do User Testing
(You’d be crazy not to)
By Colin Lieberman
Librarian extraordinaire Jessamyn West has an excellent post up about her father’s trials and tribulations with an online library catalog:
Dad: Okay I searched for sailing and I get 1500 hits. How do I search for the most popular books?
Me: I don’t know if you can, you can redo your search and sort by relevance.
Dad: Amazon lets me search by popularity. I like that.
Me: Yeah I do too. Can you sort the search you have?
Dad: No, it says there’s more than 500 records so I can’t search.
Me: You may be able to search by subject heading and get a shorter list.
Dad: Didn’t I do that?
Me: No, you searched by keyword [explains difference] or you can search just the books in your library.
Dad: I’m not already doing that?
Me: No, you’re searching the whole SAILS network.
Dad: How can you tell?
And that’s just a snippet. He’s not likely to spend much time with that catalog. If it were a store, he’d be unlikely to spend much money, if any.
User testing is easy. User testing is cheap. User testing will find all kinds of problems with your work, and point in you in the right direction for fixing them.
The majority of problems user testing will find are the kind of problems Mr. West is having: the makers of the software have no idea what false assumptions follow from their familiarity with the material.
How to do User Testing
If you haven’t already, buy a copy of Steve Krug’s Don’t Make me Think, and read it.
Then, identify some tasks you want people to accomplish on your site. Keep them light, fun, and open ended. Instead of saying “choose a book and buy it,” say “your nephew is turning 5, and he’s really in to dinosaurs. Find a present for him.”
A few minutes of small talk warms people up well. So does $50. Pay them up front, then chat for five minutes about anything at all. Smile. When you move on to talk about the tasks you’ll be asking your tester to do, explain that you need her to speak all her thoughts out loud as she goes, so you can understand why she chooses the actions she does. Some people will intuitively get this. Some might need you to demonstrate.
Explain the tasks, shut up, and sit back. And pay attention. It can help to record audio in the room, and do a screen capture on the computer, but this isn’t necessary if your engineers are watching (from a safe distance).
Do this with two or three people, and it only costs you $150 plus a few hours of everyone’s time. And you’re in a much better position to avoid the kind of needless frustration experienced by Mr. West.