Three different things get sold as "AI customer research", and only two of them involve a customer. When a slide says "research shows", find out which of the three it came from before you let it change a plan.
Here are the three:
- AI-moderated interviews. Software asks the questions. A real person answers.
- AI analysis of real material. A model reads transcripts, support tickets or survey comments that real people wrote, and summarises them.
- Synthetic users. A model generates the answers. No person is involved at all.
The first two are evidence about your customers, with the usual limits of that kind of evidence. The third is a draft. It can help you prepare, but it cannot tell you what a customer thinks, because no customer thought it.
This guide does not rank the three by accuracy. It says what each one is made of, and so what each can carry, with a table, a checklist and a slide label you can copy.
What each kind of evidence is made of
The useful question is not "how good is the AI?" It is "who produced these words?"
An AI-moderated interview produces words from a person who lived the experience. The software only changes who asks. Every limit of a human-run interview still applies, and we cover those below.
AI analysis of real material produces a summary written by a model, built on words written by people. The people are real. The summary is a second layer, and it is only as good as your ability to get from the summary back to the original words.
A synthetic user produces words from a model. In one published method for simulating survey respondents, the model is conditioned on backstories taken from real participants in earlier surveys, and then it generates answers for each backstory.1 The backstories come from people. The answers do not. Nobody had the experience, nobody can be asked what happened next, and nobody can contradict the output.
That difference decides what each can support. The table below sets it out.
The evidence-origin table
| Type | Who produced the words | Can support | Cannot support | Safe use |
|---|---|---|---|---|
| AI-moderated interview | A real person, answering questions put by software | What happened, in their account; why they say they acted; the words customers use; surprises you did not plan for | How many customers feel this; what people do rather than recall; anything the questions never reached | Findings about experience and reasons, labelled as interview evidence and traced to transcripts |
| AI analysis of real material | Real people wrote the source text; a model wrote the summary | What people already wrote; a pointer to which sources to read | Anything the source people were never asked; any theme you cannot trace to specific sources | A first-pass sort of tickets, comments or transcripts, with every theme linked to its sources and spot-checked |
| Synthetic user | A model, with no person involved | Nothing about your real customers | Any claim about what customers think, want, feel or would do | Drafting and stress-testing a question guide; rehearsing a session; listing hypotheses to ask real people. Never used as evidence |
Read the table row by row before you read any AI research result. If you cannot place a finding in a row, you do not yet know what it is.
AI-moderated interviews are still interviews
Swapping the moderator does not change what an interview can show. Three limits carry over unchanged.
Self-report is memory. When people tell you what they do, they are telling you what they remember doing.2 Two sociologists named the habit of treating verbal accounts as proof of what people do in the moment: the attitudinal fallacy.3 So read an interview as an account of an experience. Check it against a record whenever you hold one: an order history, a recording of a task, a log.
The people are chosen, not sampled at random. Qualitative studies pick people for the information they can give: "information-rich cases".4 That is the right way to find out what exists and why. It is the wrong way to find out how many. A set of interviews cannot tell you the share of your customers who feel something. Say who you talked to and how you found them, every time.
Question quality decides answer quality. Good interview guidance asks for stories and real examples and avoids generalities and talk of how things "should" happen.5 Read the questions an AI moderator was set to ask, as you would read a human moderator's guide. If the questions ask for opinions about the future ("would you pay for this?"), the answers are guesses, whoever asked. For question craft, see our guide to questions that move beyond surface answers.
Insist on transcripts or recordings. Without them you cannot trace a finding to words, and the checklist below fails at question 3. And remember that a recording of a weak question is still a weak question.
AI analysis of real material: real words, a second layer
Support tickets, survey comments and interview transcripts are good raw material, because real people produced them. A model can be used to sort them. Your job is to keep the summary honest.
Good research practice keeps what people said and did apart from what you think it means. The UK government's service manual tells teams to record exactly what they saw or heard rather than their interpretation, and to make recordings available so people can confirm their observations and get verbatim quotes.6 The same rule works for a model's summary. Each theme should point to the specific sources behind it, and you should open a sample of them.
Then name what the material cannot show. Read support emails as the voice of people who chose to write in, and read survey comments as the voice of people who chose to answer. Another page of the same service manual lists interviews, observation and existing data such as analytics and support logs as separate things to use.7 Treat them as separate lenses, not one pile.
Ask any summary two things: how many sources sit behind each theme, and can I see them?
Synthetic users: a draft, not a witness
A synthetic respondent is a useful sparring partner and a poor witness. Be plain about both.
Where it helps, use it freely:
- Draft a first version of a question guide, then cut the questions you would not ask a real person.
- Stress-test a guide for wording that is confusing, leading or too long.
- Rehearse a session before the first real one.
- List possible reasons, objections or words customers might use, and treat the list as hypotheses to check.
Where it does not belong: in the evidence column. A synthetic answer cannot be traced to a person. Ask it for a specific recent example, and any example it gives describes something that never happened.
A good rule: anything a synthetic user tells you becomes a question for a real one. If it says customers would pay extra for flexible delivery slots, write down "ask people what they did the last time a delivery slot did not suit them", and ask that.
Run this checklist on any AI research finding
Copy this and run it before a finding changes a plan. Answer each in one line.
- Who said this? Name the people, or the writing, behind the words.
- Was a real person involved? Did a real person have the experience and give the answer?
- Can you trace the finding to their words? Can you open the quotes, transcript or ticket behind each claim? If you can open only a sample, answer yes only when every source you opened holds.
- How were those people chosen? Who is missing, and who declined?
- What was asked? Read the actual questions. Were they about a recent, specific experience, or opinions about the future?
- Said or done? Is it what people recall, or something recorded or observed?
- What decision is this strong enough for? Name one decision it can inform and one it is too thin for.
How to read the answers, in order:
- Question 2 is no: it is a draft. Use it to prepare questions. Do not act on it as evidence.
- Question 2 is yes and question 3 is no: it is unchecked. Do not act until someone has traced the claims to their sources. If they hold, it becomes evidence. If they do not, correct the claim from what the sources show, or drop it.
- Questions 2 and 3 are both yes: it is evidence. Questions 4 to 7 set how far it can take you: a small pilot for a claim resting on recalled accounts, a larger change for a claim that matches records you hold.
This is a working tool, not a validated scoring system. Change the questions to fit your work. It applies equally to research with customers, users, your own staff and partners.
Put the origin on the slide
A finding can lose its origin once it is pasted into a deck. Add this label under every claim that rests on AI research. Fill in the brackets.
Evidence origin: [people interviewed / documents written
by people / model output, no people]
Method: [AI-moderated interview / AI analysis / synthetic user]
Traceable to quotes: [yes / no]
Status: [evidence / unchecked / draft]
Method names the row in the table above. Written on one line, in the same order (origin; method; traceable; status), three labels read like this:
- People interviewed; AI-moderated interview; yes; evidence.
- Documents written by people; AI analysis; no; unchecked.
- Model output, no people; synthetic user; no; draft.
A reader who sees "draft" on a slide knows not to build on it. A reader who sees no label should ask for one. Our guide on what makes an insight evidence-backed goes deeper on what a claim needs before it earns that word.
An example, start to finish
Here is an invented example to show the method.
A meal-kit service lets customers pause a subscription today only by contacting support. It is deciding whether to add a self-serve "pause my subscription" button. The analyst's deck has three claims.
Claim A: customers pause for holidays and money. It comes from nine interviews with customers who paused or cancelled in the last three months. An AI moderator asked the questions and every transcript is stored. Running the checklist: the words come from customers (1), a real person gave each answer (2), and every theme links to quotes (3). They were recruited from an email list of people who had already paused or cancelled. Fourteen were invited, nine took part and five did not reply, and people who stayed quietly subscribed are missing (4). The questions asked about the last time they paused or cancelled (5). It is a recalled account (6). Verdict: evidence. It can justify a small pilot of the self-serve button. It cannot say how many customers would use it (7).
Claim B: late deliveries are the top complaint. It comes from a model's summary of 120 support emails. Customers wrote the emails (1), real people (2), but the summary lists no sources (3), so the verdict is unchecked. The team asks for the emails behind each theme and counts them: 31 are about late deliveries, 64 are about missing items and 25 are about something else, which adds up to 120. Missing items, not late delivery, is the biggest group. The claim is rewritten, now traced to its sources, so it is evidence, and it carries a note that it covers only people who chose to write in (4). Nobody was asked anything: they wrote unprompted (5). The emails are complaints as written, not order records (6). It can justify looking into why items go missing first. It cannot say how often customers who never wrote in hit the same problem (7).
Claim C: customers would pay extra for flexible delivery slots. It comes from 50 model-generated respondents. No person was involved (2). Verdict: draft. It leaves the evidence column and becomes a question for the next round of interviews: what did you do the last time a delivery slot did not suit you?
The deck now carries three claims with three labels, written in the order origin; method; traceable; status:
- Claim A: people interviewed; AI-moderated interview; yes; evidence.
- Claim B, after tracing and correction: documents written by people; AI analysis; yes; evidence.
- Claim C: model output, no people; synthetic user; no; draft. It has turned into a question for real people.
What this will not tell you
A label shows where a finding came from. It does not show that the finding is true. Accounts are recalled, as above, and a transcript supports a claim only as far as the question and the sample allow. A pile of support emails shows only the people who wrote in.
The checklist does not rank methods, and it says nothing about what any model can or cannot do today. It asks about origin and traceability, which you can check in your own material. And "draft" is not an insult: a synthetic answer can improve the questions you ask, and that is a real use.
Your next step
Take the last deck or report you received that mentions AI research. For each claim, answer questions 1 to 3 of the checklist. Add the origin label to each claim, and ask for the sources behind any claim you cannot trace. Do the same to your own work before you present it.
