LLM options

The language model writes every reply and decides when to call functions. Pick it with the model picker at the left of the toolbar, and tune it in the LLM settings popover next to it (the sliders icon). The default is GPT 5.6 Terra (gpt-5.6-terra).

Models#

The picker groups models into Versatile and highly intelligent and Fast and cost-efficient, and shows each model's LLM price per minute.

The LLM price is one line of the agent's per-minute cost. The Cost estimate in the header adds infrastructure, voice and telephony.

Needs keys Each provider needs its key on the platform (OpenAI, Anthropic or Google). Without it the model runs in mock mode, which returns canned replies.

Choosing a model#

• Start with GPT 5.6 Terra. It is the default and the model the latency estimates are built around.
• For short, simple replies where speed matters more than reasoning, try a fast model such as GPT 5.6 Luna or a Gemini Flash model. Test function calling carefully after switching.
• The Latency estimate in the header changes with the model. For example, the estimated LLM time is 500-800 ms for gpt-5.6-terra and 300-500 ms for gpt-5.6-luna.
• Replies from OpenAI models are capped at 300 completion tokens per turn, which keeps answers phone-sized.

LLM settings#

Open the popover with the sliders icon next to the model name.

LLM Temperature#

model_temperature, 0 to 1, default 0. Lower values give more consistent replies; higher values give more variety. The slider runs from Deterministic to Creative.

Structured output#

tool_call_strict_mode, default true. Function arguments always match the function's JSON schema. The first call after a function changes can be slower while the schema is prepared. Turn it off only if a schema uses features that strict mode rejects.

Fast tier#

model_high_priority, default false. On OpenAI models, requests use the priority service tier for faster and more consistent latency. The dashboard describes it as about 25% faster. The LLM part of the call is billed at 1.5 times, shown as an llm_priority line in call_cost. A Fast badge appears on the model picker when it is on.

Ultrafast#

model_ultrafast. Fast tier plus the lowest-latency settings for the model: on GPT 5 models, no reasoning effort and short answers; on Gemini, no thinking budget. Turning it on also turns on Fast tier. Billed like Fast tier. Best for short, simple replies.

Fast tier and Ultrafast are not available for Claude models. Choosing a Claude model turns both off.

Speech to speech#

A speech-to-speech model takes the caller's audio and returns audio directly, replacing separate transcription, LLM and voice. The last group in the model picker, Speech to speech (audio in, audio out; replaces transcription and voice), lists:

When s2s_model is set, the model field is not used for replies, and the picker shows a Speech to speech badge. Picking a normal model clears s2s_model. The speech-to-speech price replaces the LLM and voice lines in the cost estimate. Each call records latency.s2s, the time from the end of the caller's speech to the first agent audio.

Needs keys Speech-to-speech needs an OpenAI key on the platform.

Trade-offs to weigh before switching:

• The voice comes from the speech-to-speech model, so the voice library, voice speed and pronunciation settings do not shape the audio the same way.
• Test functions, transfers and knowledge base answers again; behaviour differs from the text pipeline.

Fields#

On the Retell LLM (Update Retell LLM):

On a conversation flow the same choices live in model_choice:

{
  "model_choice": { "type": "cascading", "model": "gpt-5.6-terra", "high_priority": false, "ultrafast": false },
  "model_temperature": 0,
  "tool_call_strict_mode": true
}

Set model_choice.type to "s2s" with s2s_model for speech to speech. Flow nodes can override the model per node with their own model_choice. See Conversation flow.

The post-call analysis model is separate: post_call_analysis_model on the agent, set in Post-Call Extraction. See Analysis.

Workspace settings that affect the LLM#

Under Settings, Reliability:

• LLM fallback model and LLM fallback after (ms): if the agent's model sends no first token within the timeout (default 4,000 ms), the turn is retried on the fallback model. See Reliability.
• LLM token limit: the context budget per request. When a long call exceeds it, the oldest turns are dropped; the system prompt is kept. The default is 32,768 tokens.