Evaluating a phone-agent demo
What to test in an AI receptionist demo
By ByTomorrow · Published
A prepared demo usually shows the conversation the agent expects. Your callers may interrupt, correct a detail, ask for something unsupported or change their minds. Those cases determine whether the system deserves a live pilot.
Use the acceptance script below with fictional caller details and agreed test destinations. These are proposed tests, not results from a ByTomorrow implementation.
What to look for
Where the decision can go wrong
The evaluator supplies the happy path
If you only ask the suggested questions, you learn how the rehearsed path works. Write your own cases before the presenter starts.
The conversation ends before the result is checked
An agent can say it created a booking or sent a task without the expected record appearing. Require evidence in the destination, not only a smooth closing sentence.
Practical worksheet
Use this with your own business details
Case: a corrected detail
Say a fictional name or callback detail, then correct it. Expected result: the final record uses the corrected value and the agent reads back the relevant information. Record the exact failure if old and new values are mixed.
Case: an unknown answer
Ask a business question absent from the supplied facts. Expected result: the agent acknowledges the gap and uses the agreed next step. A plausible invented price or policy fails the test.
Case: immediate human request
Ask for a person before finishing intake. Expected result: the agent follows the permitted human route without demanding unrelated qualification details. If nobody answers, observe the fallback.
Case: unavailable action
Use an agreed test setup where the requested booking or handoff cannot complete. Expected result: no success claim, a clear explanation and a recoverable next step. Do not intentionally disrupt a production service to create this test.
Case: interruption and changed intent
Begin a new inquiry, interrupt, then ask to change an existing appointment instead. Expected result: the agent follows the latest request, preserves relevant corrected details and avoids continuing an obsolete sales sequence.
Case: a completed action
Ask for an allowed action using a designated test record. Expected result: the correct destination shows the result, its identifier is recorded and the caller receives accurate confirmation. Arrange cleanup of test records with the owner.
Put it to work
Work through the decision
Agree the test boundary
Confirm the environment, allowed actions, test contacts and who owns cleanup. A tool test may affect a real connected system.
Write expected outcomes first
For every case list the requested behavior and the evidence that would demonstrate it. Keep an untested category.
Run conversation and audio checks
Use text or simulation for logic where supported, then listen to real audio for timing and interruptions.
Use a real phone test for telephony
Retell distinguishes browser audio from phone testing. Carrier behavior and actual human transfers need the applicable phone path.
Keep a defects list
Record scenario, observed result, expected result, responsible person and retest outcome. Retest changed behavior before treating the demo as accepted.
Questions
Straight answers.
No. Retell documents phone tests for telephony behavior. A browser conversation can help assess the voice and flow but is not the same test.
Define severity in advance. A fabricated promise, wrong recipient or false booking success deserves a different response from a cosmetic wording preference.
Cover the important actions and failures in your actual scope. This guide does not prescribe a universal count or claim that a passed checklist guarantees future behavior.
Sources and further reading
- Retell testing overview
Retell distinguishes conversation tests, browser audio tests and real phone tests for carrier behavior and transfers.
Checked .
- Retell call-transfer documentation
Retell documents cold and warm human transfers; destination availability and failed-transfer behavior need configuration.
Checked .
- Retell integration tools
Retell documents mapping tool responses into later actions and warns that create/update tests write real provider records.
Checked .
Make the next step specific
Bring the cases that matter to your business and ask to see the resulting records. A demo should leave a test log you can review.
Talk through your workflow