When the agent says something wrong, you need two answers: can you reproduce it, and how often does it happen? The turn actions in Test LLM answer both without a phone call.
Debug: regenerate a turn ten times#
Hover an agent turn in Test LLM and choose Debug. Tried generates that turn again 10 times from the same history (two rounds of 5 requests in parallel) and lists the results under the turn.
- The header reads "N variants, M distinct". M is the number of different texts.
- Variants that called tools start with the tool names in brackets, for example
[check_availability] Let me check.... - Use replaces the turn's text with that variant (without the tool prefix), so you can continue the chat from the version you want to test.
- Close the list with the X.
How to read it:
| Result | What it suggests |
|---|---|
| 1 distinct of 10, and it is wrong | The prompt reliably produces this answer. Change the instruction. |
| Several distinct, most correct | The answer is unstable. Make the instruction more specific, add an example, or lower the model temperature. |
| Some variants call a tool and some do not | The tool's description or the prompt does not say clearly when to call it. |
| Variants call the wrong tool | Tool names or descriptions overlap. |
In mock mode every variant is the same Mock reply to: "..." text, so Debug always shows 1 distinct. Use it with real keys.
Debug uses your Dynamic variables and Tool mocks like any other turn.
Rerun, Edit and Delete#
| Action | Use it to |
|---|---|
| Rerun (agent turn) | Generate one new version of a turn after you change the prompt, without retyping the conversation. |
| Edit then Save and rerun (user turn) | Ask the same thing a different way and see if the agent copes. Everything after the edited turn is dropped. |
| Delete | Remove a turn and its tool calls to try a different path. |
Rerun and Save and rerun save your pending edits first, so a rerun always uses the prompt in the editor.
Replay after a prompt change#
Replay (below the message box) clears the chat and sends your user lines again from the top, one by one, against the current prompt. Use it to check that a fix for one turn did not break the turns before and after it.
Look inside a turn#
Expand Tool Invocation: name to see the arguments the model produced. Wrong or missing arguments usually mean the parameter descriptions need work. A Mocked badge means the result came from Tool mocks; an invocation with no result means the tool has no mock and the turn stopped.
In a conversation flow, Node transition → node_id shows each move. If the agent answers from the wrong node, check the transition conditions on the edges out of the previous node.
Reproduce a real call#
From Call History, open the call and choose View in Test Playground. It opens the agent's builder, where you can rebuild the conversation in Test LLM:
- Set the call's dynamic variables with the
{ }button. - Add Tool mocks with the tool results from the call's transcript.
- Type the caller's lines up to the turn that went wrong, then use Debug on it.
For a whole-conversation problem, copy the caller's side into a simulation test case so it runs before every publish.
When the text chat cannot reproduce it#
If the turn is fine in Test LLM but wrong on calls, the cause is probably in the voice path:
- Speech recognition misheard the caller. Check the user turns in the call's transcript.
- Interruptions cut the agent off or the agent replied to half a sentence. Check Responsiveness and Interruption Sensitivity.
- Prompt differences. Test LLM adds a short voice-call hint to the prompt but does not run reminders, backchannels or the full runtime.
Reproduce those with Test Audio or a phone call.
Ask Conductor#
Select calls in Call History and choose Ask Conductor about these calls. Conductor reads the calls and lists the most common issues. See Conductor: call and chat history.