Create test case definition

Define a simulated caller, its success criteria and tool mocks for testing an agent.

POSThttps://api.tried.com.au/create-test-case-definition

Creates a test case for an agent. A test case describes a simulated caller (user_prompt) and what a good conversation looks like (success_criteria). Run one or more test cases with Create batch test. See Test agents for the dashboard view.

Request#

Field	Type	Required	Description
`agent_id`	string	Yes	Agent the test case belongs to. Must exist in the workspace.
`name`	string	Yes	1 to 120 characters.
`user_prompt`	string	Yes	Persona and goal of the simulated caller, written to the caller ("You are..."). 1 to 8000 characters.
`success_criteria`	string	Yes	What the agent must do for the run to pass. 1 to 4000 characters. Put exact phrases in quotes if you rely on mock mode (see Create batch test).
`dynamic_variables`	object	No	Values for `{{variables}}` in the agent. String, number or boolean values, stored as strings. Default `{}`.
`tool_mocks`	array of objects	No	Fixed results for the agent's tools. Default `[]`.
`llm_model`	string	No	Model that plays the simulated caller when an OpenAI key is set. Default `gpt-5.6-terra`.

Each item in tool_mocks:

Field	Type	Required	Description
`tool_name`	string	Yes	Name of the tool to mock. Any call to this tool gets `output`.
`output`	string	No	Result returned to the agent. Default `""`.

A tool call with no mock gets a placeholder result, and the agent does not continue on it.

Example request#

curl -X POST https://api.tried.com.au/create-test-case-definition \
  -H "Authorization: Bearer $TRIED_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "agent_id": "agent_3e4f5a6b7c8d9e0f1a2b3c4d",
    "name": "Reschedule appointment",
    "user_prompt": "You are Sam. You have a booking on Tuesday at 3pm. You want to move it to Thursday morning.",
    "success_criteria": "The agent confirms the new time and says \"Thursday\".",
    "dynamic_variables": { "first_name": "Sam" },
    "tool_mocks": [
      { "tool_name": "check_availability", "output": "{\"slots\":[\"Thursday 10am\"]}" }
    ]
  }'

const res = await fetch("https://api.tried.com.au/create-test-case-definition", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.TRIED_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    agent_id: "agent_3e4f5a6b7c8d9e0f1a2b3c4d",
    name: "Reschedule appointment",
    user_prompt: "You are Sam. You have a booking on Tuesday at 3pm. You want to move it to Thursday morning.",
    success_criteria: 'The agent confirms the new time and says "Thursday".',
    dynamic_variables: { first_name: "Sam" },
    tool_mocks: [{ tool_name: "check_availability", output: JSON.stringify({ slots: ["Thursday 10am"] }) }],
  }),
});
const testCase = await res.json();

import json, os, requests

res = requests.post(
    "https://api.tried.com.au/create-test-case-definition",
    headers={"Authorization": f"Bearer {os.environ['TRIED_API_KEY']}"},
    json={
        "agent_id": "agent_3e4f5a6b7c8d9e0f1a2b3c4d",
        "name": "Reschedule appointment",
        "user_prompt": "You are Sam. You have a booking on Tuesday at 3pm. You want to move it to Thursday morning.",
        "success_criteria": 'The agent confirms the new time and says "Thursday".',
        "dynamic_variables": {"first_name": "Sam"},
        "tool_mocks": [{"tool_name": "check_availability", "output": json.dumps({"slots": ["Thursday 10am"]})}],
    },
)
test_case = res.json()

Response#

201 Created

{
  "test_case_definition_id": "test_case_8f7e6d5c4b3a29180f1e2d3c",
  "agent_id": "agent_3e4f5a6b7c8d9e0f1a2b3c4d",
  "name": "Reschedule appointment",
  "user_prompt": "You are Sam. You have a booking on Tuesday at 3pm. You want to move it to Thursday morning.",
  "success_criteria": "The agent confirms the new time and says \"Thursday\".",
  "dynamic_variables": { "first_name": "Sam" },
  "tool_mocks": [
    { "tool_name": "check_availability", "output": "{\"slots\":[\"Thursday 10am\"]}" }
  ],
  "llm_model": "gpt-5.6-terra",
  "created_timestamp": 1760850000000,
  "last_modification_timestamp": 1760850000000
}

Field	Type	Description
`test_case_definition_id`	string	Test case ID, prefix `test_case_`.
`created_timestamp`	integer	Unix ms.
`last_modification_timestamp`	integer	Unix ms of the last change.

Errors#

Status	When
`400`	Body fails validation, for example `success_criteria: Required`.
`401`	Missing or invalid API key.
`403`	The key's IP allowlist or scopes reject the request, or the caller has the read-only member role. This route needs the `agents` scope.
`404`	`Agent <agent_id> not found`.
`429`	Rate limit exceeded. See Rate limits.

See Errors for the body format.

Retell compatibility#

Same path and method as Retell, with a different body. Tried ties a test case to an agent_id instead of a response engine, and grades it against one success_criteria text instead of a list of metrics. Tool mocks match on tool_name only.