Usability testing in one afternoon

In one sentence

Five participants, a prototype, and a script that does not contain the answer will find most of what is wrong with a flow — in about four hours, without a research team.

Why it matters

The alternative to a small, fast test is not a big, rigorous one. The alternative is shipping and arguing about the analytics three months later, when the cost of changing direction is highest and the data still cannot tell you why people dropped out.

Usability testing does one thing that no other method does: it shows you a person failing at a task you were certain was obvious. That observation is almost impossible to argue with, which is why a recorded session is the fastest way to settle a design disagreement that has been running for two weeks.

How it works

Recruit five. The number comes from Nielsen's observation that around five users surface most usability problems in a given design — with two conditions that usually get dropped: they must be from one user group, and you must actually iterate and test again. Five is not a sample size for measuring anything. It is a number for finding problems.

Write tasks, not questions. The difference decides the whole session:

Bad:  "Do you think the filters are easy to find?"
Bad:  "Use the filter menu to narrow results to items under $50."
Good: "You have $50 to spend. Find something you'd actually buy."

The first asks for an opinion nobody can give reliably about an interface they have used for ninety seconds. The second contains the answer — it names the mechanism. The third has a real goal and lets the participant find their own route, which is the only thing you are trying to observe.

Say less than feels comfortable. When someone gets stuck, the instinct is to help. Don't — the sticking point is the finding. Three responses cover almost every situation:

• "What are you trying to do right now?"
• "What did you expect to happen?"
• "Talk me through what you're seeing."

Take notes in a fixed shape. One line per observation, always the same columns: participant, task, what they did, what they said, severity. Structured notes can be sorted and counted afterwards. Prose cannot.

In practice

An afternoon looks roughly like this: five sessions of thirty minutes, ten minutes between each to reset the prototype and finish notes, then an hour to sort. Two people is ideal — one facilitating, one taking notes — but one person can do it if sessions are recorded.

Sorting is the part that gets skipped and is where the value is. Group observations into issues, then rate each issue on two axes:

• Frequency — how many of the five hit it.
• Severity — did it slow them down, or did it stop them completing the task?

Anything that stopped more than one person is a fix before launch. Anything that slowed one person down is a backlog item. Being explicit about this stops the readout becoming a list of everything anyone said, which is the format that gets politely ignored.

Separate observation from explanation in your notes. "Three participants clicked the logo to get home" is an observation. "Users don't understand our navigation" is a theory, and writing it down as though it were data is how a session gets used to confirm what someone already believed.

Common pitfalls

• Testing with colleagues. They know the product, the vocabulary, and what you want to hear. All three make the session worthless.
• Demoing instead of testing. The moment you explain the interface, you have stopped measuring whether it explains itself.
• Recruiting five people who are all from different user groups, then treating the result as coverage. It is five samples of one each.
• Only testing the happy path. The interesting failures are in recovery: wrong input, back button, interrupted flow.
• Running the test, writing the report, and stopping. The five-user number assumes you fix and test again; without the second round it is just a well-documented list of problems.

Further reading

Nielsen Norman Group's article on testing with five users is worth reading in full for the conditions attached to the number. For the writing-tasks problem, Steve Krug's Rocket Surgery Made Easy remains the most practical short treatment of running sessions with no budget.