LLM WebSocket

Protocol reference for custom LLM agents: the frames Tried sends to your server and the frames it accepts back.

title: LLM WebSocket description: Protocol reference for custom LLM agents: the frames Tried sends to your server and the frames it accepts back. order: 1#

An agent with response_engine: { "type": "custom-llm", "llm_websocket_url" } gets its replies from your WebSocket server instead of a built-in LLM. This page lists every frame. For a walkthrough with a working server, see Build a WebSocket server.

Connection#

For each call, the agent runtime opens {llm_websocket_url}/{call_id} (a trailing / on the URL is removed).
Every frame is a JSON text frame. Frames from Tried have interaction_type; frames from your server have response_type.
After connecting, the runtime waits up to 0.5 s for a config frame from your server, then continues.
If the socket cannot be opened, the call ends with error_llm_websocket_open.

Test a URL before you attach it with Test custom LLM connection.

Frames from Tried#

`interaction_type`	Fields	When
`call_details`	`call`	Once after connecting, only when your `config` set `call_details: true`.
`ping_pong`	`timestamp` (epoch ms)	Every 2 s, only when your `config` set `auto_reconnect: true`.
`update_only`	`transcript`, `turntaking` (`agent_turn` or `user_turn`)	When an utterance is added to the transcript. No reply is expected.
`response_required`	`response_id`, `transcript`	The agent needs to say something now.
`reminder_required`	`response_id`, `transcript`	The caller has been silent for `reminder_trigger_ms`; say something to prompt them.

transcript is the whole conversation so far as [{ "role": "agent" or "user", "content" }].

response_id increases by one with each request. A new request replaces the previous one: chunks for an older response_id are ignored.

call object in call_details#

Field	Description
`call_id`, `call_type`, `agent_id`, `direction`, `from_number`, `to_number`, `metadata`	From the call.
`retell_llm_dynamic_variables`	The agent's default variables, then the call's variables, merged.
`collected_dynamic_variables`	Variables collected so far.
`transcript`	Plain text transcript so far.

Frames from your server#

config#

{ "response_type": "config", "config": { "auto_reconnect": true, "call_details": true } }

Field	Type	Description
`auto_reconnect`	boolean	Turn on `ping_pong` keepalive and reconnects (see Reconnects).
`call_details`	boolean	Ask for the `call_details` frame.
`transcript_with_tool_calls`	boolean	Accepted. The runtime currently sends only `transcript`.

response#

Stream the reply in one or more frames with the same response_id. The last frame has content_complete: true.

{ "response_type": "response", "response_id": 3, "content": "Sure, Friday at 10 am is free. ", "content_complete": false }
{ "response_type": "response", "response_id": 3, "content": "Shall I book it?", "content_complete": true }

Field	Type	Description
`response_id`	integer	Must match the latest request.
`content`	string	Text to speak. Chunks are joined in order.
`content_complete`	boolean	`true` on the last chunk.
`end_call`	boolean	Hang up after speaking (`agent_hangup`).
`transfer_number`	string	Cold transfer to this number after speaking.
`show_transferee_as_caller`	boolean	With `transfer_number`: show the caller's number to the transferee.
`digit_to_press`	string	Press these keypad digits after speaking.
`no_interruption_allowed`	boolean	The caller cannot interrupt this reply.

The runtime waits up to 30 s for each chunk. If nothing arrives, the call ends with error_llm_websocket_runtime.

A response with response_id: 0 sent before any request is treated as the opening line, so your server can speak first.

agent_interrupt#

Makes the agent speak without a request, for example when a background job finishes.

{ "response_type": "agent_interrupt", "interrupt_id": 1, "content": "Your booking is confirmed.", "content_complete": true }

Chunks with the same interrupt_id are joined and spoken once content_complete is true. It accepts end_call, transfer_number, digit_to_press and no_interruption_allowed.

update_agent#

Changes turn-taking settings for the rest of the call.

{ "response_type": "update_agent", "agent_config": { "responsiveness": 0.8, "reminder_trigger_ms": 8000 } }

agent_config accepts responsiveness, interruption_sensitivity, reminder_trigger_ms and reminder_max_count.

tool_call_invocation and tool_call_result#

Record your server's own tool use in the call's transcript_with_tool_calls.

{ "response_type": "tool_call_invocation", "tool_call_id": "t1", "name": "lookup_order", "arguments": "{\"id\":\"A-1042\"}" }
{ "response_type": "tool_call_result", "tool_call_id": "t1", "content": "{\"status\":\"shipped\"}" }

ping_pong#

Reply to each ping_pong with your own, echoing a timestamp. The runtime measures the round trip and reports it as latency.llm_websocket_network_rtt on the call.

{ "response_type": "ping_pong", "timestamp": 1792310400000 }

metadata#

{ "response_type": "metadata", "metadata": ... } is accepted and not used.

Reconnects#

With auto_reconnect: true, if no ping_pong arrives from your server for 5 s, or the socket closes, the runtime reconnects (at most 2 times) and sends the pending response_required again on the new socket. Without auto_reconnect, a closed socket ends the call.

Disconnection reasons#

Reason	Cause
`error_llm_websocket_open`	The first connection failed.
`error_llm_websocket_lost_connection`	The socket closed, or reconnects were used up.
`error_llm_websocket_corrupt_payload`	A frame was not valid JSON, or had an unknown `response_type`.
`error_llm_websocket_runtime`	A reply chunk took longer than 30 s, or another error occurred while reading.