A percentage can make AI receptionist benchmarks look straightforward. Before using it to make a decision, ask what was counted and what the number means for your practice.
A selected demonstration, a scripted test and a live service report can all provide useful evidence. They answer different questions. The label beside the result should make that distinction clear.
Start with the unit being measured
“Calls answered”, “requests captured” and “requests completed” describe different events. A call may be answered successfully while the practice still has work to do. A request may be captured correctly and appropriately require a member of staff.
Ask the supplier to define the measure in plain language. Then ask for the count behind the percentage and the period it covers. This makes the result easier to compare and harder to misunderstand.
Know what kind of evidence you are reading
| Evidence | What it can help show | What to check |
|---|
| Selected demonstration | How a chosen example behaves | Configuration, context and what happened after the conversation |
| Scripted test set | Performance on specified scenarios | Success definition, complete results and the limits of the set |
| Live service report | Observed activity over a defined period | Patient and workflow mix, exclusions and work left for staff |
A high score on a selected set should be described as a result on that set. It should not silently become a claim about all future patient calls.
Compare equivalent outcomes
If one supplier reports conversations completed and another reports requests resolved without further staff work, the percentages are not directly comparable. Establish a common definition before ranking them.
The same applies to speed. Time until the telephone is answered, the pause before a reply and the wait for a human are different measures. A useful comparison names the experience it is timing.
Ask for enough detail to judge the result
Buyers need the assessment date, product scope, relevant configuration and a clear description of the method. They do not need a supplier’s confidential architecture or full library of test material to ask sensible questions.
Record unresolved points alongside the results. A comparison is more credible when it shows where the evidence is strong, where products appear similar and where a further demonstration is needed.
Use evidence to improve the decision
At QuantumLoopAI, the conversation about EMMA should connect evidence to the proposed practice workflow. EMMA supports NHS GP patient access during surgery hours. Ask the team to explain what a result establishes and which questions remain specific to your configuration.
Use our supplier evaluation guide or arrange an EMMA demonstration.
