Mobile App Usability Testing: Watch People Use It

The app works. The person still stops.

Someone has booked an evening class and now needs to move it to Thursday. The app opens, the booking is there and the change button works. Yet the person doesn't tap it. They aren't sure whether changing the date will cancel the booking or lose their deposit.

This is an illustrative example, not a client case. It captures a problem that a functional check can miss: the system can carry out an action, but the person cannot confidently decide to take it. Asking whether they like the screen won't tell you much. Watching them try to move the booking will.

That's the point of mobile app usability testing. You give someone a credible goal, let them attempt it and study the gap between your explanation of the app and their experience of it. The method belongs within the wider app UX design process, but a useful first session needn't become a large research programme.

Choose a question small enough to answer

Start with the decision you need to make, rather than with a request to test everything. In our booking example, the question is whether people can reschedule without worrying that they have cancelled. That is narrow enough to observe and important enough to affect the product.

“Is the app easy to use?” is much harder to answer. Someone might navigate happily and still misunderstand the final confirmation. A second person might struggle to find the booking but understand the change perfectly once they reach it. Those are different problems, with different remedies.

Write down what you will recognise as success before the session: the participant finds the right booking, chooses a new date and correctly explains what will happen to the existing payment. Keep that definition in your notes. It is not an instruction to read aloud.

Usability research also has a boundary. It doesn't establish whether enough people want your business idea; that belongs to app idea validation. Nor does it replace checking crashes, payment failures or data loss. A person can understand a broken feature, and misunderstand a perfectly functioning one.

Use a prototype if it can support the task honestly

You don't have to finish development before inviting people. A clickable prototype may be enough to explore whether the booking, date choice and confirmation make sense. It is less useful for learning about slow networks, keyboard behaviour or real notifications. Choose the artefact that can answer your question; our prototype versus MVP guide explains the distinction.

Prepare a believable starting point. Seed a test account with a booking so that the person can begin with the actual problem, not fifteen minutes of setup. Use invented customer details and a safe payment environment. A study shouldn't accidentally charge a card, cancel a real appointment or send a message to another customer.

If a prototype stops at a particular screen, explain that limitation when needed. Don't quietly interpret a dead link as a usability failure. Conversely, don't explain the intended design in advance just because the prototype is unfinished. The words and choices on the screen are still part of what you are testing.

Recruit people who have a reason to use the app

A colleague knows too much. They may recognise the label because they helped choose it, or overlook a difficulty because they know where the next screen leads. Friends can be helpful for checking that your session runs smoothly, but they aren't automatically suitable product participants.

Look for relevant behaviour. For a class-booking app, someone who recently booked and changed a class is more informative than someone who simply says they enjoy exercise. Ask about that experience without coaching them towards your proposed feature. Include the actual device and language where those affect the journey.

Nielsen Norman Group's guidance on participant numbers describes small groups as useful for qualitative, iterative research. Five suitable people can be a reasonable first round for one narrow audience. Five people split across customers, instructors and administrators are not five studies of each audience. Separate genuinely different needs, and resist turning a small sample into a claim about everybody.

Give the goal, not the route

Task wording can make a confusing design appear obvious. If you tell someone to open Bookings, tap Edit and select Thursday, you have supplied the navigation they were meant to discover.

A better task gives enough context for a real decision and leaves the route open. NN/G's article on task scenarios is a useful reference for this distinction. Here are original examples for the class-booking app:

The second column doesn't name a control or force a successful ending. Someone may reasonably stop because the conditions are unclear. You have learnt something useful even though they didn't finish. Keep your private success criteria separate from what the participant hears.

Let them hesitate without making it awkward

Begin by saying that you are testing the app, not the person. Explain what you will record and ask permission before recording. On a personal phone, private notifications can appear unexpectedly; agree how to handle them and who will have access to the recording. The GOV.UK moderated-testing guidance covers these practical preparations.

During the task, silence feels longer to the observer than to the participant. It is tempting to rescue the moment with “Try the button at the bottom.” Once you do, the rest of the task tells you about assisted use, not independent use. Help if the session needs to move on, but label that help in your notes.

Neutral questions work better than reassurance: “What are you expecting here?” or, after an action, “What do you think happened?” Avoid congratulating the person for finding the intended route. That reaction tells them there is a right answer to please you with. The GDS researchers' moderation advice explains why a comfortable, unsteered session matters.

If someone asks what a label means, don't immediately define it. Ask what they think it means, then allow them to continue. You are not withholding customer support; you are finding out whether the product needs that support in the first place.

A phone changes the conditions

The same journey can feel different on the device people actually use. A keyboard can cover the next action. A person holding a bag may struggle with a control that looked easy on a large preview. Returning after an interruption may reveal that the app has forgotten the date they chose.

Don't stage a theatrical obstacle course. Start with a normal session, then investigate the conditions relevant to the audience. If people commonly change bookings while travelling, it is worth checking whether the unfinished task survives a brief interruption. If larger text matters, test it rather than assuming the layout will adapt.

Language is part of this, too. Test the actual local labels, dates and confirmation wording. An English session cannot establish that the German version is clear. And a general usability round cannot replace focused work with people who use assistive technology; use the app accessibility checklist to plan that work.

*An illustration of the method: notice where the journey breaks before helping someone across it.*

Separate what happened from your explanation

“The button is too small” often sounds like a finding when it is actually a proposed explanation. A more useful note describes the event: the participant opened the booking twice, read the payment line and stopped. When asked what they expected, they said changing the date would lose the deposit. In this hypothetical case, a larger button may leave the real uncertainty untouched.

Keep three things distinct: the observed behaviour, your interpretation and the change you want to try. You might interpret the hesitation as uncertainty about payment. The proposed change could be a short explanation of what stays paid before confirmation. Another session can test that explanation.

Record whether the task was completed independently, with help or not at all. Note the consequence of the problem: a delay, an accidental cancellation, a wrong expectation about payment. “Two of five participants in this round needed help” is an honest description. “Forty percent of our users will fail” is not.

Prioritise consequences as well as repetition. A misunderstanding that could cancel the wrong booking deserves attention even if you saw it once. Repeated hesitation on a harmless decorative detail may be less urgent. This is a product judgement informed by evidence, not an automatic vote count.

Change the cause, then test the change

Resist polishing everything at once. If you alter navigation, wording and layout together, you may get a smoother session without learning which change mattered. Start with the most credible explanation of the problem and make a targeted revision.

For first-use questions, fresh participants help: someone who has already learnt the route may succeed through memory. Returning participants can still be useful for other questions, but be clear about what you are learning. Ask people to explain the final result as well as reach the last screen. A confirmation isn't helpful if they believe the old booking is still active.

After release, app analytics can show where people leave a journey; observation can help explain why. Wider beta testing adds experience across devices and situations. Neither makes the small study worthless, and neither turns it into a guarantee.

For a first round, a single page of preparation is enough: your question, relevant participants, a few realistic tasks, the starting account, observation notes and the next decision. If that decision needs a development team, bring the uncertainty to Appfyl or include it in a short project brief. A useful brief describes what the customer cannot yet do confidently, not just which screens you want built.

Questions people ask