You interviewed your best shop managers. One told you to book the customer's next visit before they leave the till. Another said booking at the till is a mistake and you should phone them three weeks later. Someone in the room asks, "So what do we tell all the other shops?"
Do not average them, and do not pick the manager with the best story. Disagreement between strong operators is the most useful thing in your notes. It tells you where context is doing the work. Your job is to find out which differences hide a shared principle, which are context-specific tactics, and which are genuine conflicts you cannot settle by talking.
This guide gives you one table to fill in and one three-way sort to apply to it. It starts where top performer analysis ends: you have done the interviews, and the practices do not line up.
Why you must not average strong operators
Start from the assumption that there is no one best way to run a shop, a branch, a sales territory or a franchise. Assume two excellent operators can do opposite things and both be right, because their customers, staff, space and neighbourhood differ. A playbook that says "do what the majority did" erases exactly the differences that explain why each of them wins.
Realist evaluation builds this into how it asks questions. It is a theory-led approach that seeks to understand what works, for whom, in what circumstances.2 You can borrow the question without borrowing the whole method.
That gives you three questions to ask of every practice you collected:
- In whose hands and in what setting does it work?
- What is it for? What does it make the customer, or the team, do or understand?
- Does another operator reach the same result a different way?
The table and the sort below make you answer all three.
The comparison table
Make one row per operator and practice. If an operator has two practices, give each its own row and repeat the context in full. Keep every row whole. A method paper on analysing interviews in a matrix, with one row per case, notes that this keeps each person's views connected to the rest of their account, so the context is not lost.3 That is the point of the layout: the practice never travels without its context.
| Operator | What they do | Context | What it is for | Evidence | Conflicts with |
|---|---|---|---|---|---|
| Name or code | The action, its trigger and who does it | Site, customers, team, constraints | So that [who] [does or understands what] | Said, shown or in records | Which other rows it clashes with |
How to fill each column:
- What they do: an action someone could copy on Monday. "Good follow-up" is a mood. "Phones every buyer three weeks after the sale" is a practice.
- Context: the facts that would make a practice hard to copy. Size of team, type of customer, footfall, opening hours, the one person who holds it together. Write what you saw or were told, not what you assume.
- What it is for: one sentence in the shape "So that [who] [does or understands what]". Write what people do in response to the practice, not what the operator hopes. Do not skip this column; it is the one that does the sorting. The closest realist idea is the mechanism: the combination of what a programme provides and how people respond to it.1
- Evidence: mark each row as said (they described it), shown (you watched it happen) or in records (a log or count backs it over time). A practice that is only said is still worth a row. Keep the label.
- Conflicts with: name the rows it clashes with. Leave the cell blank if none do; do not force it.
The people who will receive these practices judge whether their own shop looks like yours. You cannot judge that for them. What you can do is describe the setting well enough for them to decide.5 A thin context column makes that impossible.
Ask what each practice is for
Two practices can look opposite and do the same job. One case-based comparison method, qualitative comparative analysis, calls this equifinality: different pathways can reach the same outcome.4
To get to the "so that" sentence, go back to the operator with one question: "What would go wrong if you stopped doing this?" Listen for what it is for in the answer. Then write it in your own words and in the same shape each time:
So that [who] [does or understands what].
Write it as a sentence a second operator could agree or disagree with. "So that customers feel valued" is too soft to sort anything. "So that the customer leaves with a next step and the shop holds it" can be checked against another row. It must also be narrower than your study's goal. "So that more customers come back" only restates the goal; say who responds and how.
Do this for every row before you compare any two. If you compare tactics first, you will see only clashes.
The three-way sort
Take the "what it is for" column and apply these rules in this order. Stop at the first one that fits. Every row lands in one of the three, because the third is the catch-all.
| Sort | Rule | What you do |
|---|---|---|
| Shared principle | Two or more rows have the same "so that" sentence, even though the tactics differ | Offer the principle for a trial. Let each shop choose a tactic that fits its setting. |
| Genuine conflict | The "so that" sentences differ and one shop cannot run both tactics | Keep both. Write the condition under which each works. Test one. |
| Context-specific tactic | Everything else: no genuine conflict, no shared sentence | Offer it for a trial in shops that fit, with its conditions attached. Say who it will not work for, or "no special conditions found". |
"Offer for a trial" means: put it to the shops that fit and try it there first. Share it widely once the test supports it.
Four notes on using it.
Reconcile before you declare a genuine conflict. Treat an apparent clash as a purpose you have not yet written down. Write both "so that" sentences in the same shape and plain words first. Only what is left after that step is a genuine conflict.
Tactics that can coexist are not conflicts. If two rows seem to clash but one shop could run both tactics side by side, they are not a genuine conflict. Sort each on its own, without the conflict rule: if they share a "so that" sentence they fall under the first rule, otherwise under the third.
Write the condition, even if it is a guess. For a genuine conflict, finish this sentence for each side: "This works when [condition], because [reason]." Label it as your best guess until a test says otherwise.
Do not count heads. A single operator with a clear purpose and a clear context is a complete row. It earns its own row and its own sort. A pair is not proof either; both go to a test. Nor does a confident storyteller outweigh a quiet operator with records behind them.
An example, start to finish
Here is an invented example to show the method.
A chain of bicycle shops wants more customers to come back for their first service, which falls due about six weeks after purchase. You interview five shop managers, all chosen from the shops' own records. Each row below is one manager and one practice, so five rows in all.
| Operator | What they do | Context | What it is for | Evidence | Conflicts with |
|---|---|---|---|---|---|
| Amara | Books the first service at the till, before the customer leaves | City-centre shop; commuters; one busy till | So that the customer leaves with a next step and the shop holds it | Shown: you watched a booking made at the till | Ben |
| Ben | Never books at the sale; tells each buyer "I'll ring you in three weeks" and does | Rural shop; regulars; customers do not know their diary six weeks ahead | So that the customer leaves with a next step and the shop holds it | In records: call log | Amara |
| Chen | Saturday drop-in check with no appointment; it is not the first service, and the mechanic books the first service during the visit | University town; students who live within walking distance; a spare mechanic on Saturdays | So that a student brings the bike back without booking | Shown: you watched a Saturday drop-in | |
| Dalia | Charges full price for the first service and calls it a planned upgrade, with a helmet fit included | Established suburb; mostly experienced riders upgrading | So that an experienced rider books the service as a planned upgrade | Said | Ed |
| Ed | First service free for every buyer | New housing estate; mostly first-time riders | So that a first-time rider books the first service instead of putting it off over cost | In records: free-service tickets counted | Dalia |
The Conflicts with column lists the apparent clashes; the sort decides which are genuine. Now apply the sort rules literally, in order, to each row.
- Shared principle: Amara and Ben, two rows. The tactics are opposite. Written side by side, their "so that" sentences are the same: the customer leaves with a next step, and the shop holds it. Offer that principle for a trial. Amara books at the till; Ben rings at three weeks. Each shop picks the tactic that matches its customers.
- Genuine conflict: Dalia and Ed, two rows. Their sentences differ even written in the same shape, and a shop sets one price for the first service: it is free for every buyer or it is full price. Keep both. Write each side's condition in the prescribed shape, as your guesses about a shop's customer mix. Dalia's works when most customers are experienced riders, because they will pay for a service they see as an upgrade. Ed's works when most customers are first-timers, because cost is what makes them put it off.
- Context-specific tactic: Chen, one row. Her sentence is not shared, and no tactic stops a shop running hers. Her drop-in is not the first service, so the price question does not touch it, and a shop could run it alongside Amara's booking. Her sentence differs from Amara's and Ben's even though the mechanic books the first service during the visit: what her tactic gets students to do is come back at all without planning ahead, and the booking follows. That is why her Conflicts with cell is blank. Her tactic needs a spare mechanic on Saturdays and students who live within walking distance. Offer it for a trial with those conditions attached. A shop with no students within walking distance does not meet them, so leave that shop out of the trial.
That is five rows sorted into three groups: two, two and one.
The next test, kept short
You now have a shared principle, a genuine conflict and a context-specific tactic. Test them one at a time, in this order:
- The shared principle first. By definition it already covers more than one operator, and in the example it needs no new tool or price change.
- One side of a genuine conflict next, in a shop that matches that side's written condition. Never test both sides at once in the same shop.
- The context-specific tactic last, and only in a shop that meets its conditions.
Before any test starts, write down what you expect to see, what you will measure, how long you will wait and what result would make you stop. Keep similar comparison shops running the current way alongside, so you can tell the practice from everything else that changed.
In the example, you run only the first test. Two shops with no routine for giving customers a next step adopt the shared principle, each with the tactic that matches its customers; these are the adopting shops. Two similar shops stay on the current way; these are the comparison shops. That is four shops. You expect more buyers to come back for the first service in the adopting shops than in the comparison shops. You count the buyers who return in each group. You wait twelve weeks, long enough for bikes bought in the first weeks of the test to reach their first service. You stop early if booking delays or complaints rise in the adopting shops. The Dalia and Ed genuine conflict and Chen's drop-in wait.
What this will not tell you
This is a way of organising what operators told you. It is not proof that any practice works.
- A handful of interviews cannot say how common a practice is across the chain. The table shows what strong operators do, not how many shops do it.
- A "so that" sentence is your reading of what the operator said. The operator may disagree with it. Show it to them.
- Realist evaluation is a method for evaluating programmes, with its own rules for evidence. The "what it is for" column borrows its idea as a note-taking frame; it does not make this a realist evaluation.
- A genuine conflict can stay unresolved after a test. Then keep both practices, with their conditions, in the playbook.
- The conditions you write are guesses until a test supports them.
Follow your organisation's consent and privacy rules for anything you record about staff or customers.
What to do this week
Open your interview notes and build the table with one row per operator and practice. Fill the "what it is for" column for every row before you compare any two. Sort the rows in the order above, and pick the first test. If you have not run the interviews yet, start with the top performer guide. If you want to run this kind of study with your own people, there is a top performer analysis use case.
