Skip to content
TriedDocs
Test

Test LLM

Chat with your agent's draft over text in the builder's test panel, with dynamic variables, tool mocks, turn edits and replays.

Test LLM is a text chat with your agent in the builder's test panel. It runs the agent's prompt, functions, multi-prompt states or conversation flow nodes, knowledge base and handbook, and streams the reply. There is no audio and no per-minute cost, so it is the fastest way to iterate on a prompt.

It is available for single prompt, multi-prompt and conversation flow agents. Custom LLM agents use their own testing tools.

Test LLM in the test panel, with a user turn and the agent's reply

Start a chat#

Open the agent#

Open the agent from Agents. The test panel is on the right.

Choose Test LLM#

Select the Test LLM tab. The empty state reads "Chat with your agent".

Let the agent start, or type#

Choose Let the agent start to get the welcome message, or type a message and press Enter. Shift+Enter adds a new line.

Each request saves your pending edits first, so you always chat with what you see in the editor.

How the opening turn works#

  • Single and multi-prompt agents: if the agent speaks first and has a welcome message, the first agent turn is that message with dynamic variables filled in. Otherwise the model writes the first turn.
  • Conversation flow agents: if the flow speaks first and the start node is a conversation node with a static instruction, that text is the first turn.

What the chat shows#

Item Meaning
User and Agent bubbles The conversation. Agent text streams in as it is generated.
Tool Invocation: name The agent called a function. Expand it to see the arguments and, if mocked, the Result. A Mocked badge marks results from the Tool mocks tab.
Node transition → node_id A conversation flow moved to another node.
"The flow reached an end node" The flow reached an end node.
"Subflow finished: reached the component exit" When testing a single component, the conversation reached its exit node.

Multi-prompt state changes are not shown as a bubble, but the next turn runs in the new state.

Dynamic variables#

The { } button at the top of the test panel opens Dynamic variables: "Values used for {{variables}} in test calls and chats." It starts with the agent's default dynamic variables. The count next to the button shows how many are set. The same values are used by Test Audio and the simulated user.

System variables such as the current time are filled in automatically, with the call id playground and the call type web_call.

Tool mocks#

The Tool mocks tab (wrench icon) holds canned outputs: "When the agent calls one of these tools during a test chat, it gets this output instead of calling your endpoint."

The Tool mocks tab with a mock for check_availability
  • Choose Add mock, enter the tool name exactly as defined (for example check_availability) and the output (usually JSON).
  • The tab icon shows how many mocks are active.
  • Mocks are saved in your browser, per agent. They are not shared with teammates or with simulation test cases (test cases have their own Function mocks).

When the agent calls a mocked tool, the output is fed back to the model and the agent continues the same turn. The agent can chain up to 3 rounds of mocked tool calls in one turn.

Unmocked tools do not run

Test LLM never calls your real endpoints. A tool call without a mock is shown with its arguments and no result, and the agent's turn stops there. Add a mock for every tool the agent may call in the conversation you are testing.

Transition functions are handled for you: moving between multi-prompt states and following conversation flow prompt edges happen without mocks.

Edit, rerun, delete and replay#

Hover a turn to see its actions.

Action On What it does
Edit User turns Opens the text for editing. Save and rerun replaces the turn, drops everything after it and generates a new agent reply.
Rerun Agent turns Drops this agent turn (and anything after it) and generates it again from the same history.
Debug Agent turns Regenerates the turn 10 times and lists the variants. See Debugging responses.
Delete Any turn Removes the turn. For an agent turn, its tool calls and transitions go with it.

Below the input:

  • Replay runs your user lines again from the top against the current prompt. Use it after a prompt change to see the whole conversation with the new version.
  • Reset clears the chat.

Test a single component#

In a conversation flow, a component can be tested on its own with Test subflow in the component editor. The test panel then shows only Test LLM, the simulated user and Tool mocks, with the note "Testing this component on its own: the chat starts at its first node." The chat ends when the component reaches its exit node.

Mock mode#

Without an OpenAI key (or with TRIED_MOCK_PROVIDERS=1), the API returns deterministic replies: "Hello! How can I help you today?" for an opening turn with no welcome message, and Mock reply to: "<your message>" for every other turn. Static welcome messages are still used. Mock mode is for checking flows and wiring, not the quality of replies.

Differences from a real call#

  • Test LLM adds "You are on a voice call: keep replies short and conversational." to the prompt. A real call uses the full voice runtime.
  • Speech features do not apply: speech recognition errors, interruptions, backchannels, pronunciation, reminders after silence.
  • Tools do not execute (only mocks), and transfers, SMS and keypad digits are not performed.
  • Knowledge base retrieval searches with the last user message.

Use Test Audio or a phone call for those.

API#

The panel streams from POST /agent-playground-completion/{agent_id} as server-sent events. Each event is a line data: <json>.

Body field Type Description
messages array of { role, content } The conversation so far. role is user, agent, assistant or system. Default [].
dynamic_variables object Values for {{variables}}. Default {}.
current_node_id string Conversation flow node to continue from.
current_state string Multi-prompt state to continue from.
agent_version integer Version to test. Default: latest.
tool_mocks array of { tool_name, output } Up to 50 mocks.
start_component_id string Start inside this flow component.
Event type Fields
delta content
tool_call tool_call_id, name, arguments
tool_result tool_call_id, name, content, mocked: true
node_transition former_node_id, new_node_id
end node_id, reason (end_node or component_exit)
done content, current_node_id, current_state
error message

Pass current_node_id and current_state from each done event into the next request to keep the conversation's position.

curl -N -X POST https://api.tried.com.au/agent-playground-completion/agent_46117b4b00fe0358e861a6d3 \
  -H "Authorization: Bearer $TRIED_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{ "role": "user", "content": "Do you open on Saturdays?" }],
    "dynamic_variables": { "business_name": "Harbourside Dental" },
    "tool_mocks": [{ "tool_name": "check_availability", "output": "{\"slots\":[\"14:30\"]}" }]
  }'