Skip to content
TriedDocs
Reliability and latency

Actual latency

The per-call latency fields Tried measures on every turn, what each one covers, how percentiles are computed and where to read them.

Every call records measured latency for each agent turn. The voice runtime reports one sample per turn per stage, and the API stores the samples and returns percentiles on the call object as latency.

Where to see it#

  • Call drawer. Open a call in Call History. The latency popover lists End to end latency, Transcription latency, LLM latency, TTS latency, and, when present, Knowledge base latency and Speech to speech latency, each with Median, P90 and Min. A stage with no samples shows No data.
  • The call object. GET /v2/get-call/:call_id and the call_ended and call_analyzed webhooks include latency. See The call object.
The call detail drawer

Fields#

Field What it measures Source in the runtime
e2e From the end of the caller's speech to the first agent audio LiveKit e2e_latency on the agent message
asr From the end of the caller's speech to the final transcript LiveKit transcription_delay on the user message
llm Time to the first LLM token LiveKit llm_node_ttft
tts Time to the first TTS audio byte LiveKit tts_node_ttfb
knowledge_base Knowledge base retrieval time for the turn Timed around the retrieval call
llm_websocket_network_rtt Round trip to your Custom LLM WebSocket server Custom LLM ping
s2s Speech-to-speech engines: time to the first agent audio after the caller's turn Realtime model ttft

All values are milliseconds. A stage only appears when at least one turn reported it.

Note

e2e is measured on the server, from where the runtime decided the caller stopped. The caller also hears the phone network's delay in both directions, and e2e does not include the end-of-turn wait before the runtime decides. What a caller experiences is longer than e2e. Independent phone tests of voice platforms show a gap of several hundred milliseconds between platform-reported and caller-heard latency.

Percentile object#

Each field is an object:

{
  "latency": {
    "e2e": { "p50": 912, "p90": 1310, "p95": 1310, "p99": 1310, "max": 1310, "min": 702, "num": 9, "values": [702, 845, 880, 902, 912, 950, 1010, 1180, 1310] },
    "llm": { "p50": 540, "p90": 720, "p95": 720, "p99": 720, "max": 720, "min": 410, "num": 9, "values": [410, 455, 480, 520, 540, 575, 610, 690, 720] }
  }
}
Key Meaning
p50, p90, p95, p99 Nearest-rank percentiles of the samples
min, max Fastest and slowest turn
num Number of turns measured
values Every sample, in the order the turns happened

Percentiles use the nearest-rank method: the value at position ceil(p / 100 x num) in the sorted samples. With few turns, p90 to p99 are often the same as max, as in the example above.

Reading the breakdown#

e2e is roughly asr + end-of-turn decision + llm + tts, with some overlap because the LLM can start on the interim transcript. When e2e is high, compare the stages:

Pattern Likely cause Where to look
llm high Model choice, long prompt or long history, provider load Troubleshoot latency
tts high Voice model, first-sentence length Troubleshoot latency
asr high accurate or custom STT with long endpointing Troubleshoot latency
e2e much higher than the sum Waiting for end of turn (low responsiveness), tool calls before the reply Troubleshoot latency
knowledge_base high Large knowledge bases or many chunks Troubleshoot latency
llm_websocket_network_rtt high Your Custom LLM server is far from the runtime Host it in the same region

Compare the measured median with the builder's estimate on Estimated latency and cost.

Across many calls#

Tool What it gives
GET /analytics/summary latency.e2e_p50, e2e_p90, e2e_p99 and e2e_avg over the calls in the range
Analytics charts The e2e_latency metric, averaged by default
Alerting Rules on e2e_latency_p50 or e2e_latency_p90 that fire when latency crosses a threshold
Alerting rules

Measuring what callers hear#

The repository's latency-lab tools measure outside the platform:

Module What it measures
latency_lab.analyze_call Mouth-to-ear latency and agent cut-ins per turn from a dual-channel recording made on the caller's side
latency_lab.vendor_probe Live LLM time to first token, ElevenLabs time to first audio and Deepgram finalize time from the current region (needs vendor keys)
latency_lab.net_probe Warm and cold HTTPS round trips to each vendor API from the current host

Run them from the region your runtime is deployed in.