Usability testing
A method in which real people try to do specific tasks with a product while being observed, to find out where they get stuck and why.
What it’s for
It helps you uncover the problems the team can’t see: what it thinks is easy isn’t always easy, and that only shows when you watch other people use it.
Key elements
Tasks, not opinions
Ask people to do something, not to say whether they like it.
Observe without helping
If you help, the problem disappears from the test.
Few people, several rounds
Test, fix, and test again.
The serious ones first
Rate each problem by what it blocks and how often it appears, to decide what to fix first.
An example
A banking app wants to know whether people understand how to send money.
The test does not just show that something fails: the person’s words explain why.
How it’s done
Prepare realistic tasks, without saying how to do them.
Observe each person as they try and think aloud.
Note where they get stuck and decide what to fix first.
Applications in design
When to use it
From very early on, even with a paper prototype, and again after each major change.
Avoid
Asking “do you like it?”: measure what people do, not what they think.
Don’t fall for it
Do not explain the interface before starting: that is exactly what you want to test.
Connections
Builds on
Related to
- Agile UX, Approach
- Accessibility audit, Method
- User-centered design, Approach
- Human-centered design, Approach
- Civic design, Specialty
- UX design, Discipline
- System Usability Scale (SUS), Method
- User flow, Artifact
- UX research, Specialty
- Wizard of Oz, Method
- Prototyping, Method
- Tree testing, Method
- Cognitive walkthrough, Method
- Design sprint, Process
- A/B testing, Method
Authors and works
- 1980
K. Anders Ericsson and Herbert A. Simon
Explained when what a person says while thinking is valid as data.
- 1982
Clayton Lewis
Brought the thinking-aloud method to interface design.
- 1993
Jakob Nielsen and Thomas K. Landauer
Proposed a model of how many new problems appear with each additional person.
- 2003
Laura Faulkner
With 60 people she showed that five are not always enough: depending on who they were, five found between 55% and 99% of the problems; with twenty, never less than 95%.
- 2003
Morten Hertzum and Niels Ebbe Jacobsen
Reviewing eleven studies, they found that two evaluators analyzing the same interface with the same method agree on only part of the problems, so it pays to have more than one person analyze.
Sources
Original sources
- Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87(3), 215–251. View source: Ericsson, K. A., & Simon, H. A. (1980)
- Lewis, C. (1982). Using the “thinking-aloud” method in cognitive interface design (Technical Report RC 9265). IBM T. J. Watson Research Center. View source: Lewis, C. (1982)
- Nielsen, J., & Landauer, T. K. (1993). A mathematical model of the finding of usability problems. In Proceedings of INTERCHI 93 (pp. 206–213). ACM. View source: Nielsen, J., & Landauer, T. K. (1993)
- Faulkner, L. (2003). Beyond the five-user assumption: Benefits of increased sample sizes in usability testing. Behavior Research Methods, Instruments, & Computers, 35(3), 379–383. View source: Faulkner, L. (2003)
Academic reviews
- Hertzum, M., & Jacobsen, N. E. (2003). The evaluator effect: A chilling fact about usability evaluation methods. International Journal of Human-Computer Interaction, 15(1), 183–204. View source: Hertzum, M., & Jacobsen, N. E. (2003)
Last reviewed: