Address metric issues

AI QA tells you where calls go wrong. This page maps each low result to the evidence to look at and the agent settings that usually fix it. Every setting named here is a real field on the agent or its response engine; change it, test it, then watch the cohort's Overview for the effect.

A workflow for fixing issues#

You can also select failing calls in Call history and use Ask Conductor about these calls for suggested prompt changes. See Conductor.

Hallucination#

Look at: details.issues on the QA sheet: each issue's type (fabrication, contradiction, confusion), severity and the agent utterance.

Fixes:

• Add the missing facts (prices, hours, policies) to the agent's knowledge base or prompt so the agent has a source.
• Tell the agent in the prompt to offer to take a message or transfer when it does not know.
• For facts that change, look them up with a custom tool instead of the prompt.
• Lower model_temperature on the LLM (range 0 to 1, default 0) if you raised it.
• Turn on Echo verification in the Agent Handbook so names, numbers and dates are repeated back.

Heuristic fabrications flag any number, price or time not found in the agent's context or the caller's words, so a number in a dynamic variable or a tool result counts as grounded.

Knowledge base recall#

Look at: the call's Knowledge and workflows tab in Call History: each retrieval's query, rewritten query and chunks with scores.

Fixes:

• If chunks are missing or irrelevant, improve the source documents or split long pages so one chunk answers one question.
• Adjust kb_config on the LLM: top_k (1 to 10, default 3) and filter_score (0 to 1, default 0.6). A lower filter_score returns more chunks; a higher one returns fewer, closer matches.
• Use kb_instruction (up to 500 characters) to steer query rewriting toward your domain.
• If chunks were right but unused, tell the agent in the prompt to answer from retrieved information.

Word error rate#

Look at: Transcription accuracy on the QA sheet after you correct a few calls. WER is only meaningful on calls you corrected.

Fixes:

• Add product names, people and place names to boosted_keywords.
• Set vocab_specialization to medical for clinical vocabulary.
• Try stt_mode accurate, or custom with custom_stt_config to choose another provider.
• Set language (and languages for multilingual callers) to match how callers speak, for example en-AU.
• Raise denoising_mode to noise-and-background-speech-cancellation for noisy callers.

Overlapping speech#

Look at: details.overlaps and overlap_percent, and listen to the recording around the overlaps.

Fixes:

• Lower responsiveness (0 to 1, default 1) so the agent waits longer before replying, or turn on enable_dynamic_responsiveness to adapt to slow talkers.
• Lower interruption_sensitivity (0 to 1, default 0.9) so background noise does not cut the agent off.
• Shorten agent turns in the prompt; long turns invite interruptions.
• If backchannels are on (enable_backchannel), lower backchannel_frequency.

Overlap detection needs per-word timings from speech to text.

Sentiment#

Look at: calls with Negative sentiment, and the User Sentiment of the calls in Call History.

Fixes:

• Pick a warmer Personality (for example Friendly) in the Agent Handbook, or set handbook_config.high_empathy through the API so the agent acknowledges feelings before solving.
• Check that callers can reach a person: a transfer tool for complaints.
• Check whether the User Sentiment prompt matches how you define a negative call. See Post-call analysis.

Naturalness#

Look at: details.repeated_turns, robotic_phrases, very_long_turns and the average words per turn.

Fixes:

• Ask for one or two short sentences per turn in the prompt.
• Turn on Natural filler words and Speech normalization in the Agent Handbook.
• Remove instructions that make the agent repeat the same sentence (for example a fixed confirmation every turn).
• Add a pronunciation_dictionary entry for words the voice says wrong.

Tool call accuracy#

Look at: details.tool_calls: each call's name and reason (invalid arguments, no result, failed). Expand Tool Invocation and Tool Result in the call transcript.

Fixes:

• Make the tool's description say exactly when to call it and what each parameter means.
• Keep tool_call_strict_mode on (default true) so arguments follow the schema.
• Fix the endpoint when results are errors or status 400 and above.
• Check the Custom function failures alert to catch this early. See Alerting.

See Tools.

Transition accuracy#

Look at: details.problems (self-transitions and loops) and the Node Transition chips in the transcript.

Fixes:

• Make edge conditions specific and mutually exclusive.
• Add an exit edge for the case that loops.
• Review global nodes that pull the conversation away and back.

See Conversation flow.

Transfer success and wait time#

Look at: transfer items in transcript_with_tool_calls (started, bridged, cancelled), the reason on cancelled transfers, and disconnection_reason.

Fixes:

• Check the destination number and that someone answers within the transfer option's transfer_ring_duration_ms (5000 to 300000 ms).
• For long waits, play hold music with on_hold_music on the transfer option.
• Use the Transfer call failures alert.

Latency#

Look at: the latency breakdown in the call drawer (End to End Latency hover): which stage is slow.

Fixes:

• Slow LLM: try a faster model, or turn on model_high_priority or model_ultrafast on the LLM.
• Slow knowledge base: lower top_k.
• Slow tools: set the tool to speak during execution so the caller hears something while it runs.
• Long prompts add LLM time: trim the prompt.

Low criteria pass rates and top questions#

• A criterion that fails on most calls is either a real gap or a criterion the judge cannot see. Read the explanations on a few QA sheets; rewrite the criterion if the transcript does show the behaviour. See Resolution criteria.
• A common question with a low Resolved rate in Top questions usually needs a knowledge base answer or a transfer path.
• If you disagree with verdicts, mark calls as passed or failed and Calibrate. See QA results.

Related#