Calibrate QA cohort

Fit a QA cohort's pass threshold to your human pass and fail labels.

POSThttps://api.tried.com.au/qa/cohorts/{id}/calibrate

Finds the pass threshold (a whole number from 0 to 100) that best matches the results you labelled with Update QA result. Ties go to the threshold closest to the current one. With apply on, the cohort's pass_threshold is updated and every result's passed is recomputed.

Request#

Field	Type	Required	Description
`id`	string	Yes	QA cohort ID, for example `qa_cohort_2c4e6a8b0d1f3e5a7c9b1d3f`.

Body:

Field	Type	Required	Description
`apply`	boolean	No	`true` (default) saves the new threshold. `false` returns a preview only.

Example request#

curl -X POST https://api.tried.com.au/qa/cohorts/qa_cohort_2c4e6a8b0d1f3e5a7c9b1d3f/calibrate \
  -H "Authorization: Bearer $TRIED_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "apply": true
  }'

const res = await fetch("https://api.tried.com.au/qa/cohorts/qa_cohort_2c4e6a8b0d1f3e5a7c9b1d3f/calibrate", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.TRIED_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    "apply": true
  }),
});
const calibration = await res.json();

import os, requests

res = requests.post(
    "https://api.tried.com.au/qa/cohorts/qa_cohort_2c4e6a8b0d1f3e5a7c9b1d3f/calibrate",
    headers={"Authorization": f"Bearer {os.environ['TRIED_API_KEY']}"},
    json={
        "apply": True,
    },
)
calibration = res.json()

Response#

200 OK

{
  "applied": true,
  "calibration": {
    "threshold": 76,
    "accuracy": 0.9,
    "previous_threshold": 70,
    "previous_accuracy": 0.7,
    "samples": 10,
    "timestamp": 1792402200000
  },
  "cohort": {
    "qa_cohort_id": "qa_cohort_2c4e6a8b0d1f3e5a7c9b1d3f",
    "name": "Booking calls",
    "agent_ids": [
      "agent_4b7e2c9a1f3d5e8b0c6a2d1f"
    ],
    "from_timestamp": null,
    "to_timestamp": null,
    "min_duration_ms": 30000,
    "max_duration_ms": null,
    "disconnection_reasons": [],
    "sampling_percent": 50,
    "weekly_max": 200,
    "criteria": [
      {
        "name": "Confirms the booking",
        "prompt": "The agent reads back the date and time and says \"you are booked in\".",
        "weight": 2
      },
      {
        "name": "Offers a reminder",
        "prompt": "The agent offers an SMS reminder before the appointment."
      }
    ],
    "created_timestamp": 1792395000000,
    "last_run_timestamp": 1792395060000,
    "extraction_filters": [
      {
        "source": "analysis",
        "key": "appointment_booked",
        "op": "eq",
        "value": true
      }
    ],
    "metrics": [
      {
        "metric": "latency",
        "weight": 1
      },
      {
        "metric": "sentiment",
        "weight": 1,
        "threshold": 60
      }
    ],
    "pass_threshold": 76,
    "calibration": {
      "threshold": 76,
      "accuracy": 0.9,
      "previous_threshold": 70,
      "previous_accuracy": 0.7,
      "samples": 10,
      "timestamp": 1792402200000
    }
  }
}

Field	Type	Description
`applied`	boolean	Whether the threshold was saved.
`calibration.threshold`	integer	Best threshold found.
`calibration.accuracy`	number	Share of labelled results the new threshold agrees with, 0 to 1.
`calibration.previous_threshold`, `calibration.previous_accuracy`	number	The threshold before, and how well it agreed with the labels.
`calibration.samples`	integer	Labelled results used. Only results with a weighted score count.
`calibration.timestamp`	integer	Unix ms of the calibration.
`cohort`	object	The updated cohort. Present only when `applied` is `true`.

Errors#

Status	When
`400`	Fewer than 2 labelled results: `Mark at least 2 calls as passed or failed before calibrating`.
`401`	Missing or invalid API key.
`403`	The key's IP allowlist or scopes reject the request. This route needs the `analytics` scope. Dashboard users with the member role get `The member role is read-only`.
`404`	No cohort with that ID in the workspace: `QA cohort <id> not found`.
`429`	Rate limit exceeded. See Rate limits.

See Errors for the body format.

Retell compatibility#

Retell's public API reference has no AI QA endpoints, so this route has no Retell SDK equivalent.