How to test a vibe-coded prototype with real users

Stop giving the tour

You built it in a weekend with your AI coding tool. It works. You share your screen with a possible user and say, "So this is the dashboard, and over here you can add a job, and this button does the export." Ten minutes later they say, "Looks great." You learned nothing.

You used the product for them. They watched you succeed, so you never found out whether they could.

The fix is simple to state and uncomfortable to do. Give people a realistic task, say nothing about the interface, and watch. The UK government's service guidance describes moderated usability testing in these terms: you watch participants try to complete specific tasks.¹ Below is a kit for doing that with a prototype built in days.

Two questions, kept apart

A prototype session answers two different questions, and they need different tools.

1. Can they use it? You answer this with tasks. You watch what people do.
2. Does it matter to them? You answer this with a short interview about their real situation, after the tasks.

Keep them apart because each can fail alone. A tool can be easy to use and still solve nothing the person cares about. A standard introduction to usability treats ease of use and utility (whether it does what you need) as equally important, and says it matters little that something is easy if it is not what you want. The same article advises against deferring user testing until you have a fully implemented design, so do not wait for a polished build.² Run the tasks first, so you have watched what people do before they tell you what they want.

Watch behaviour over opinion. Research of this kind is about what people do, how they do it and why, and asking whether someone likes something tells you nothing about their behaviour.³ When a participant says "nice, clean design" and then spends four minutes hunting for the save button, believe the four minutes.

Before anyone arrives: data and a reset plan

Seed realistic data. Never start with an empty app, and never use placeholders like "Test Customer 1". Load ten to fifteen plausible records with real-sounding names, amounts and dates. Where you can, ask each participant beforehand for a non-sensitive sample of their own material, such as a price list or a template, and load it. The same government guidance advises that, whenever possible, participants carry out tasks using their own data and documents.¹

Write a reset plan. Assume your prototype will break mid-session. Decide now what you do:

• Keep a saved copy of the seed data and know how to restore it in under two minutes. Pause the task clock while you reset; the four closing minutes absorb it.
• Sign the participant in before the session starts (or send them a link that opens already signed in), so login is not a task.
• When something breaks, say "That one is on my build. I'll reset it and we'll carry on with the next task." Then do it. Use this line only when the app has crashed or blocked the task. Never explain or promise fixes when the person is struggling with something that works as built.
• If an ordinary action broke it, that is a finding. Write it down.

Plan rounds of five sessions, and freeze the build for the round. Do not fix things between sessions. Patch a bug after session one and session two tests a different prototype.

Rehearse. Run the whole session with a colleague first. Find the ambiguous wording before your first participant does.

The session kit

Five parts. Prepare them in this order; on the day, the script's opening comes first.

1. Tasks, not tours

A task scenario is a short story with a goal the person already has. Make it something they would really do at home or work, and ask them to do the action rather than describe how they would do it. Leave out the interface's own words: if your menu says "Estimates", the scenario must not say "estimates". Scenarios that include interface terms, or that describe the steps, hand over the answer and bias the result.⁴

Rules for the set:

• Keep it to three tasks per session.
• Start with the core task, the one your app exists for. Order the rest in the order the job happens, or from easiest to hardest.
• Put the facts on a card (names, amounts, dates) so nobody is tested on memory.
• Give each task a finish line the person can recognise: "until you have a price ready to send".
• Cap each task at eight minutes. If they are still stuck at the cap, say "Let's move on" and mark it not done.
• Allow one planned hint per task, and only late. If someone has been stuck for four minutes and the rest of the task depends on the step, name the screen but not the control ("Look at the page for the price you sent"). Mark the task "with help". Every other hint is off limits.

A forty-five minute session fits like this: 5 minutes of welcome, 24 minutes for three tasks (eight each at most), 12 minutes of interview, 4 minutes to close.

2. The moderator script

Read the opening aloud, word for word, the same way every time.

Thanks for doing this. We are testing the prototype, not you, so you cannot get anything wrong. It was built quickly and some things will break; if they do, that is useful to us.

I will give you three situations on cards. Do them as you would normally, and say out loud what you are looking at, what you are thinking and what you expect to happen. I will mostly stay quiet while you work, because I need to see what happens without me. If you get completely stuck, I might tell you which page to look at, or ask you to move on. You can stop at any time.

Before we start, may I record the screen and your voice so I can write up notes? [Follow your organisation's consent rules here.]

Then do a thirty-second warm-up: ask them to think aloud while finding tomorrow's weather forecast on their own device. This is the think-aloud method: participants use the system while continuously thinking out loud, and interruptions from an untrained facilitator can easily change what they do. So once the tasks start, you are quiet.⁵

When a participant goes silent, wait 15 to 20 seconds and then ask "what are you thinking now?". Use neutral acknowledgements such as "uh huh". Do not correct or challenge what people say or do.⁶

Neutral prompts you can use:

• "What are you thinking now?"
• "What did you expect to happen there?"
• "Say a bit more about that."
• "If I were not here, what would you do next?"

Things you must not say. Apart from the one planned hint above, all of these are off limits:

• "Try the menu at the top." (an unplanned hint)
• "Yes, it does that." (an answer they should find alone)
• "You are on the right track." (a signal)
• "Sorry, that is a bug, it will be fixed." (defending the build)
• "Do you like it?" (invites a polite answer)

When they ask "Can I click this?", reply "What would you do if I were not here?" When they criticise the design, say "Thank you, that is useful" and write it down.

3. The observation sheet

Have a second person take notes if you can. Use one sheet per participant, one row per task.

Fill in Where they hesitated with the screen or element, not your interpretation. Write What they said as a quote. Record Error as the thing they did that did not work, such as "clicked Customers instead of the sent quote". Outcome is judged by you from what happened, not by what they claim. "With help" means you gave the one planned hint.

Under the table, add two lines for the interview. Question 5 answer: yes, no or not sure, plus what they said they would need first. Real job sent: left blank until a week after the round, then yes or no.

4. Questions about their real situation

After the last task, ask for the last time they did the job, not what they would do in theory (question 5 is a concrete offer, not a prediction, and only a real job sent confirms it); our guide to interview questions that go beyond surface answers goes deeper on that craft.

1. "Tell me about the last time you [did the job this prototype is for]. What happened, step by step?"
2. "How long did it take, and who else was involved?"
3. "What did you use? What was annoying about it?"
4. "How close were those tasks to your real week?"
5. "Would you try this on one real job in the next week, with your own information? What would you need first?"

Twelve minutes will not hold all five. Always ask 1 and 5, then add one or two of the others, chosen from what you saw in the tasks.

Treat a person who comes back with a real job as stronger evidence than any compliment.

5. Findings to changes

After the round, add up the sheets. Turn each problem into a row, then into a request your coding agent can follow. A problem only one person hit still gets a row; keep its change small.

Send the agent one change at a time, with the evidence, a boundary and a finish line:

Change request [n] of [total]
Screen: [name of the screen]
Problem seen: [what people did, with how many of how many]
Goal: [what a person should be able to do]
Do: [one change]
Do not: [what to leave alone]
Done when: [task result you can check by hand]

A worked example

Here is an invented example to show the method. The product, the people and the results are all made up.

A founder has built a prototype for small landscaping firms. It turns site-visit notes into a price the owner can send to a customer. She recruits five owner-operators for 45 minutes each, loads fourteen realistic jobs and a sample price list from each, and runs the first round.

She writes her three tasks on cards. The first is the core task; the other two follow the order the job happens:

1. "This morning you visited Mrs Okafor's garden: patio clean, two hedges, three bags of green waste. Get a price ready to send her before you leave tonight."
2. "Mrs Okafor rings back. She wants the hedge work only. Make sure she gets the new price."
3. "It is Friday and you have not heard back. Find out whether she has looked at what you sent."

Task results across the five sessions. Each "with help" is the one planned hint, given after four minutes stuck:

Participant 3's sheet for task 2 reads: hesitated on the sent-quote page; said "Where do I change it? I would just start again"; error: deleted the quote and began a new one; recovered: no; outcome: not done.

Her findings table:

Reading row 4, among the 3 round-two people who take an evening to price a job: if 0 or 1 say yes to question 5, rethink who the product is for. If 2 or 3 say yes, keep going for now. If you kept going, a week later count real jobs sent: if none of the 3 has sent one, treat the yes answers as unproven and rethink before building more; if one or more has, the yes answers stand.

The last row is the useful surprise. The core task works, though changing a sent price and following up do not yet, and for three of the five it also solves a problem they barely have. That is a result you can only get by keeping the two questions apart. She sends the first three requests to her coding agent one at a time, then runs round two: five new people, three of them chosen because pricing takes them an evening, the same three tasks, the same script.

What this will not tell you

Five sessions will show you problems; they cannot tell you how many people have them. Do not turn "three of five" into a share of your market. Where you can, recruit people who do this job and do not know you. A prototype with seeded data is not your live product with messy customer data. Usability testing does not certify security, privacy, payments or legal terms of your app; for those, get a qualified review and follow your organisation's rules. If you test with real customer material, follow the same rules.

Your next step

Today, write three task cards for your own prototype, with no interface words on them. Seed the data, write down your reset plan and rehearse once with a colleague. Book five people. Run the round, change nothing until it is over, send your coding agent one change at a time, and test again. Split your effort across several small rounds and retest after each redesign; do not save it all for one big study.⁷