Conductor supports AI Quality Assurance in two ways: a guided tour that walks you through setting up QA, and QA cohorts as context in a Conductor conversation. It does not create cohorts, change criteria or score calls itself.
Guided tour: set up AI QA#
In the Conductor side panel, type "QA", "quality" or "score" in Guided tour and choose Show me, or ask "Walk me through setting up AI QA" in a conversation.
| Step | What it highlights |
|---|---|
| 1. Open AI QA | The AI Quality Assurance link in the sidebar under MONITOR. |
| 2. Create a cohort | Create cohort: filter calls by agent, date, duration and extraction fields; set sampling. |
| 3. Add criteria and metrics | Create cohort again: AI conditions plus metric judges such as hallucination, WER and latency, each with a weight. |
| 4. Calibrate | Calibrate: mark a few calls as passed or failed, then calibrate the pass threshold. |
The tour card shows "Set up AI QA · N of 4" with Back, Next (or Done on the last step) and a close button.
Ask about a QA cohort#
Attach a cohort (Cohorts tab in Attach context, from the side panel or the + button on Home). Conductor reads:
- the cohort name and id,
- how many calls have been scored,
- the average score (weighted score where available).
It does not see individual results, per-criterion pass rates or transcripts. To dig into failing calls, attach the calls themselves as well (up to 10 attachments in total) and ask Conductor to investigate them. See Call and chat history.
Example questions:
- "Our QA cohort average is low. Here are five calls that failed, what do they have in common?"
- "Write test cases for this agent based on these failed calls." (with the agent and calls attached)