When the agent mishears the caller, its reply is wrong even with a perfect prompt. Check the transcript first: in the call drawer's Transcription tab, compare what the caller said in the recording with the user lines. If the words are wrong, it is a transcription problem and the settings below apply. If the words are right but the reply is wrong, see Wrong responses.
Settings that affect transcription#
| Setting | API field | Effect |
|---|---|---|
| Boosted keywords | boosted_keywords |
Words and names the recogniser should prefer, such as product names, suburbs and staff names. Sent to Deepgram Nova-3 as key terms |
| Transcription mode | stt_mode |
fast (default) or accurate; see below |
| Custom STT | stt_mode: "custom" with custom_stt_config |
Choose the provider (deepgram, assemblyai, soniox, azure, openai), the model and endpointing_ms (10 to 5,000, default 300) |
| Medical vocabulary | vocab_specialization: "medical" |
Uses Deepgram nova-3-medical (English) |
| Language | language, languages |
The language the caller speaks. Setting language to multi, or more than one entry in languages, switches Deepgram to multi-language mode |
| Noise handling | denoising_mode, denoising_enhancement_level |
Noise cancellation, or noise and background speech cancellation |
Fast or accurate#
| Mode | Deepgram settings | Use when |
|---|---|---|
fast |
Nova-3, 25 ms endpointing, no smart formatting | Most conversations. Lowest latency |
accurate |
Nova-3, 100 ms endpointing, smart formatting and numerals | Callers read out numbers, dates, amounts or IDs, and you need them formatted |
accurate adds about 200 ms to the latency estimate; see Troubleshoot latency.
Noise handling#
denoising_mode |
denoising_enhancement_level |
Result |
|---|---|---|
no-denoise |
any | No noise processing |
noise-cancellation (default) |
not high |
Noise cancellation |
noise-and-background-speech-cancellation |
not low |
Removes noise and other voices (a telephony-tuned model on phone calls) |
any except no-denoise |
low |
Noise only |
any except no-denoise |
high |
Noise and background voices |
Use background speech cancellation for callers in cars, shops and offices where other people talk nearby.
Problems and fixes#
| Problem | Fix |
|---|---|
| Names of products, people or places are misheard | Add them to boosted_keywords. Turn on smart matching in the Agent Handbook |
| Phone numbers, dates or amounts come out wrong or in words | Use stt_mode: accurate. Turn on echo verification in the handbook so the agent repeats them back |
| Spelled codes and emails are wrong | Turn on the NATO phonetic alphabet in the handbook, so the agent confirms letters as Alpha, Bravo, Charlie |
| Medical terms are wrong | Set vocab_specialization to medical |
| Caller switches language | List the languages in languages, or set language to multi |
| Background voices are transcribed | Use noise-and-background-speech-cancellation or level high |
| Australian accents or terms | Try another provider with stt_mode: custom and compare WER in AI QA |
Agent Handbook rules#
The Agent Handbook (handbook_config) adds short instructions to the prompt that work around recognition errors:
smart_matching: when what the caller says sounds like a known name, product or option, assume they meant it and confirm, instead of saying it was not found.echo_verification: repeat names, phone numbers, dates and amounts back and wait for confirmation before acting.nato_phonetic_alphabet: spell letters in codes and emails with the NATO alphabet when confirming.
Measure accuracy with AI QA#
AI Quality Assurance computes word error rate (WER) on the caller's side of each analysed call. To get a true WER, correct the transcript by hand on the QA result: each correction targets one caller utterance, and the call is re-scored right away.
Use WER to compare settings: run a set of calls with each stt_mode or provider and compare the averages.
Provider keys#
The default Deepgram transcription is included in the voice infrastructure price. Other providers need their API key on the runtime Needs keys and add a small per-minute line to the builder's cost estimate. Deepgram can use its Australian endpoint when the operator sets DEEPGRAM_BASE_URL.
Related#
- Wrong responses
- Reliability settings for the speech recognition fallback