Skip to content
TriedDocs
Account

Create QA cohort

Create an AI QA cohort that scores calls against your criteria and metric judges.

POSThttps://api.tried.com.au/qa/cohorts

Needs keys Creates a QA cohort: a set of filters that picks ended calls, and the criteria and metrics each call is scored against. A first run starts in the background right after the cohort is created, and new calls that match are scored after their post-call analysis.

Scoring needs an OpenAI key for AI judging. See AI QA.

Mock mode

Criteria are judged by OpenAI (gpt-5.6-terra) when an OpenAI key is set. Without a key, or if the model call fails, a keyword scorer runs instead: a criterion passes when every phrase in double quotes in its prompt appears in the transcript, or, with no quoted phrases, when at least half of its keywords appear. A criterion with no keywords passes unless the call was marked unsuccessful.

Request#

Field Type Required Description
name string Yes 1 to 120 characters.
criteria object[] Yes 1 to 20 criteria. Each has name (1 to 80 characters), prompt (1 to 2000 characters) and an optional weight (0 to 100, counted as 1 when left out).
agent_ids string[] No Only calls from these agents. Default [] (all agents).
from_timestamp integer or null No Only calls created at or after this Unix ms time.
to_timestamp integer or null No Only calls created at or before this Unix ms time.
min_duration_ms integer or null No Minimum call duration, 0 or more.
max_duration_ms integer or null No Maximum call duration, 0 or more.
disconnection_reasons string[] No Only calls that ended with one of these reasons. Default [].
sampling_percent integer No 1 to 100. Default 100. Sampling is deterministic: a call is always in or always out of a cohort's sample.
weekly_max integer No Most calls scored in any 7 days, 1 to 10000. Default 100.
extraction_filters object[] No Up to 20 filters on post-call extraction, metadata or dynamic variables. Each has source (analysis, metadata or variable, default analysis), key (1 to 128 characters), op (default eq) and value. See List QA metrics for the operators.
metrics object[] No Metric judges to run on each call. Each has metric, weight (0 to 100, default 1) and an optional threshold (0 to 100) that overrides the metric's default. See List QA metrics.
pass_threshold number No Weighted score a call needs to pass, 0 to 100. Default 70.

Only calls with status ended and a non-empty transcript are scored.

Example request#

curl -X POST https://api.tried.com.au/qa/cohorts \
  -H "Authorization: Bearer $TRIED_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Booking calls",
    "agent_ids": [
      "agent_4b7e2c9a1f3d5e8b0c6a2d1f"
    ],
    "min_duration_ms": 30000,
    "sampling_percent": 50,
    "weekly_max": 200,
    "criteria": [
      {
        "name": "Confirms the booking",
        "prompt": "The agent reads back the date and time and says \"you are booked in\".",
        "weight": 2
      },
      {
        "name": "Offers a reminder",
        "prompt": "The agent offers an SMS reminder before the appointment."
      }
    ],
    "extraction_filters": [
      {
        "key": "appointment_booked",
        "value": true
      }
    ],
    "metrics": [
      {
        "metric": "latency"
      },
      {
        "metric": "sentiment",
        "threshold": 60
      }
    ]
  }'
const res = await fetch("https://api.tried.com.au/qa/cohorts", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.TRIED_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    "name": "Booking calls",
    "agent_ids": [
      "agent_4b7e2c9a1f3d5e8b0c6a2d1f"
    ],
    "min_duration_ms": 30000,
    "sampling_percent": 50,
    "weekly_max": 200,
    "criteria": [
      {
        "name": "Confirms the booking",
        "prompt": "The agent reads back the date and time and says \"you are booked in\".",
        "weight": 2
      },
      {
        "name": "Offers a reminder",
        "prompt": "The agent offers an SMS reminder before the appointment."
      }
    ],
    "extraction_filters": [
      {
        "key": "appointment_booked",
        "value": true
      }
    ],
    "metrics": [
      {
        "metric": "latency"
      },
      {
        "metric": "sentiment",
        "threshold": 60
      }
    ]
  }),
});
const cohort = await res.json();
import os, requests

res = requests.post(
    "https://api.tried.com.au/qa/cohorts",
    headers={"Authorization": f"Bearer {os.environ['TRIED_API_KEY']}"},
    json={
        "name": "Booking calls",
        "agent_ids": [
            "agent_4b7e2c9a1f3d5e8b0c6a2d1f",
        ],
        "min_duration_ms": 30000,
        "sampling_percent": 50,
        "weekly_max": 200,
        "criteria": [
            {
                "name": "Confirms the booking",
                "prompt": "The agent reads back the date and time and says \"you are booked in\".",
                "weight": 2,
            },
            {
                "name": "Offers a reminder",
                "prompt": "The agent offers an SMS reminder before the appointment.",
            },
        ],
        "extraction_filters": [
            {
                "key": "appointment_booked",
                "value": True,
            },
        ],
        "metrics": [
            {
                "metric": "latency",
            },
            {
                "metric": "sentiment",
                "threshold": 60,
            },
        ],
    },
)
cohort = res.json()

Response#

201 Created

{
  "qa_cohort_id": "qa_cohort_2c4e6a8b0d1f3e5a7c9b1d3f",
  "name": "Booking calls",
  "agent_ids": [
    "agent_4b7e2c9a1f3d5e8b0c6a2d1f"
  ],
  "from_timestamp": null,
  "to_timestamp": null,
  "min_duration_ms": 30000,
  "max_duration_ms": null,
  "disconnection_reasons": [],
  "sampling_percent": 50,
  "weekly_max": 200,
  "criteria": [
    {
      "name": "Confirms the booking",
      "prompt": "The agent reads back the date and time and says \"you are booked in\".",
      "weight": 2
    },
    {
      "name": "Offers a reminder",
      "prompt": "The agent offers an SMS reminder before the appointment."
    }
  ],
  "created_timestamp": 1792395000000,
  "last_run_timestamp": null,
  "extraction_filters": [
    {
      "source": "analysis",
      "key": "appointment_booked",
      "op": "eq",
      "value": true
    }
  ],
  "metrics": [
    {
      "metric": "latency",
      "weight": 1
    },
    {
      "metric": "sentiment",
      "weight": 1,
      "threshold": 60
    }
  ],
  "pass_threshold": 70,
  "calibration": null
}
Field Type Description
qa_cohort_id string Cohort ID, prefixed qa_cohort_.
last_run_timestamp integer or null Unix ms of the last run.
calibration object or null Result of the last applied calibration.

The other fields echo the cohort settings. See Create QA cohort.

Pricing

Scored call minutes are billed to the workspace: the first 100 minutes each calendar month (UTC) are free, then $0.15 per minute. See Get QA usage.

Errors#

Status When
400 Body validation failed, for example criteria: Array must contain at least 1 element(s).
401 Missing or invalid API key.
403 The key's IP allowlist or scopes reject the request. This route needs the analytics scope. Dashboard users with the member role get The member role is read-only.
429 Rate limit exceeded. See Rate limits.

See Errors for the body format.

Retell compatibility#

Retell's public API reference has no AI QA endpoints, so this route has no Retell SDK equivalent.