Create batch test

Needs keys Starts a batch test and returns at once. Each test case becomes a test run. The runs execute one after another in the background: a simulated caller talks to the agent through the same engine as the Test LLM playground, then a judge decides pass or fail against the test case's success_criteria. Poll Get batch test for results.

Request#

Example request#

curl -X POST https://api.tried.com.au/create-batch-test \
  -H "Authorization: Bearer $TRIED_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "agent_id": "agent_3e4f5a6b7c8d9e0f1a2b3c4d",
    "test_case_definition_ids": ["test_case_8f7e6d5c4b3a29180f1e2d3c", "test_case_1a2b3c4d5e6f708192a3b4c5"]
  }'

const res = await fetch("https://api.tried.com.au/create-batch-test", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.TRIED_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    agent_id: "agent_3e4f5a6b7c8d9e0f1a2b3c4d",
    test_case_definition_ids: ["test_case_8f7e6d5c4b3a29180f1e2d3c", "test_case_1a2b3c4d5e6f708192a3b4c5"],
  }),
});
const batchTest = await res.json();

import os, requests

res = requests.post(
    "https://api.tried.com.au/create-batch-test",
    headers={"Authorization": f"Bearer {os.environ['TRIED_API_KEY']}"},
    json={
        "agent_id": "agent_3e4f5a6b7c8d9e0f1a2b3c4d",
        "test_case_definition_ids": ["test_case_8f7e6d5c4b3a29180f1e2d3c", "test_case_1a2b3c4d5e6f708192a3b4c5"],
    },
)
batch_test = res.json()

Response#

201 Created

{
  "batch_test_id": "batch_test_2c3d4e5f6a7b8c9d0e1f2a3b",
  "agent_id": "agent_3e4f5a6b7c8d9e0f1a2b3c4d",
  "status": "in_progress",
  "total_count": 2,
  "pass_count": 0,
  "fail_count": 0,
  "error_count": 0,
  "created_timestamp": 1760850000000,
  "completed_timestamp": null
}

How a run ends#

A conversation stops when the simulated caller ends it, the agent ends the call, the mock script runs out, it reaches 400 utterances (agent and user turns), or it runs for 10 minutes. The judge then grades the transcript. A run that throws (for example, the agent fails to respond) gets status error with an error message. Tool calls are answered from the test case's tool_mocks.

Mock mode#

Without an OpenAI key:

• Simulated caller. The user_prompt is split into sentences, turned into first person ("You are Sam" becomes "I am Sam"), and spoken one sentence per turn, followed by "Thanks, that's all. Goodbye."
• Judge. If success_criteria has phrases in quotes, the run passes only when every quoted phrase appears in the transcript. Otherwise it picks the significant words of the criteria and passes when at least 60% of them appear. Criteria with no checkable words pass. Each explanation starts with Mock judge:.

Errors#

See Errors for the body format.

Retell compatibility#

Same path and method as Retell, with a different body and response. Tried takes an agent_id instead of a response engine, accepts up to 50 test cases, and returns batch_test_id with created_timestamp and completed_timestamp.