Connect an agent

#Agent kinds

Hosted prompt

Paste a system prompt, pick a model and paste its tools as OpenAI (Chat Completions or Responses) or Anthropic definitions, as they are. Duet Machines runs the agent and answers its tool calls from each scenario's simulated world. Add configurations to compare other models or prompts.

Chat Completions endpoint

An agent that already serves the OpenAI Chat Completions API with its own system prompt needs no adapter. Duet Machines POSTs {model, messages, tools, stream: false} to its URL and reads choices[0].message: the content is the reply and each tool call gets a simulated result in the next request. No system message and no lead variables are sent, and the customer speaks first. Tools your server runs itself never reach Duet Machines and touch your real systems, so return them as tool calls.

Custom endpoint

Any other agent behind one HTTPS endpoint, through the contract below and a small adapter.

Production only

Just a name, for an agent you only monitor. Duet Machines never runs it: each conversation is judged against the config (instructions and tools) it sends, and its risk surfaces are mapped from the latest one. It gets issues, signals, experiments, users, datasets of its conversations, coverage, and reproductions of conversations that sent a config, replayed on the model their config names (config.model). It cannot stress test, compare configurations, build tests or prove fixes; add its prompt or endpoint later to do so. Nothing is judged until it sends its first config.

For both endpoint kinds, give the URL, an optional auth header, the tool schemas it may call and instructions describing what it should do; stress tests and judges read them. The auth header value is write-only: it is stored for conversations and never returned. An endpoint runs as deployed, so it has a single configuration: compare deployments by adding each as its own agent. Endpoints must resolve to public addresses.

Deleting an agent deletes its findings, tests, comparisons, fixes, and production conversations; it is refused while one of its jobs is running. Owners and admins delete agents from Agent settings; members see the card disabled with that reason.

#History and runtime notes

An endpoint agent declares its history: what it keeps of earlier user turns, everything (full) or only the messages and its own replies (replies_only), and whether its production runtime drops the rest (production) or only the adapter Duet Machines reaches it through (adapter). With replies only, an act that rests on a tool result from an earlier user turn is a platform defect when production drops it, and proves nothing when only the adapter does.

Every agent also has runtime notes: other facts about how it runs that limit what it can know. Planners, judges and verifier authors read them, so no finding blames the model for data its runtime never gave it.

#Tool calls and the simulated world

Your real tools are never called. Each scenario carries a world: concise facts about the systems behind the tools (records, balances, messages, calendar, statuses), plus scripted results only for the calls the attack must force. A call takes its tool's first remaining scripted result it fits; every other call is answered by a model that simulates the tool from its schema, the variables the agent runs with, the world and every earlier call and result, so a draft created earlier is found later and a sent message stays sent. Simulated results land in the transcript like any other, and are what the judge grounds on.

An agent may take up to 10 tool rounds per reply (max_tool_rounds, up to 50). One still calling tools after that ends the conversation as a tool-round limit, so a looping agent shows as such.

#Custom endpoint contract

Every agent turn, Duet Machines POSTs the whole conversation so far. The first request has no messages: the agent greets, or answers with no reply to let the customer open. Tool results come back in the next request.

// Request
{"conversation_id": "uuid", "variables": {"customer": "Jordan"},
 "messages": [{"role": "user", "content": "Hi"},
              {"role": "assistant", "content": "", "tool_calls": [{"id": "c1", "name": "lookup", "arguments": {}}]},
              {"role": "tool", "content": "{\"balance\": \"$2,310\"}", "tool_call_id": "c1", "name": "lookup"}]}
// Response (200): every field optional; usage prices the conversation
{"reply": "You owe $2,310.", "tool_calls": [], "ended": false,
 "usage": {"input_tokens": 812, "output_tokens": 24, "model": "gpt-5.4"}}
// Response (200) when the agent's own turn failed and production would send the user nothing
{"turn_failure": "the model's final answer was not valid JSON"}
// Model checks add "model": "gpt-5.4-nano" to the request: run it and name it in usage, or answer
{"model_unsupported": "this agent only runs on Claude"}

ended: true hangs up after this reply. turn_failure reports that the agent's own turn failed the way production would show it to the user, for example an invalid model output that aborts the turn so nothing is sent; it carries no reply, tool calls or ending. Duet Machines records it as the agent's failure and judges its consequence, so return it instead of an error status. A reply with no text and no tool calls after a customer message fails the turn the same way. A timeout (60 s) ends the conversation as no reply; any other status, an invalid body or a tool not in the agent's schemas fails the conversation with the error.

usage is the step's model call: Duet Machines prices the conversation from it at list price. A model check sends the catalog model to try in model (no other request has it): run the turn on it and report usage naming that exact id on every reply to a request with messages (an opening greeting without a model call may omit it), or answer model_unsupported with why. Duet Machines checks this before the check starts and on every reply, so an endpoint that ignores model is refused, never passed.

A drop-in adapter: put your agent's turn in my_agent and give Duet Machines the URL with x-api-key as the auth header. In Python with FastAPI:

from fastapi import FastAPI, Header, HTTPException

app = FastAPI()
SECRET = "change-me"  # the auth header value you give Duet Machines


async def my_agent(messages: list[dict], variables: dict, model: str | None) -> dict:
    """One turn of your agent: return {"reply": "..."} and/or {"tool_calls": [{"id", "name", "arguments"}]}, with
    "usage": {"input_tokens", "output_tokens", "model"} from your model call. model is None except in model checks:
    run that model and report it in usage, or return {"model_unsupported": "why"}."""
    raise NotImplementedError("call your agent here")


@app.post("/duet")
async def duet_turn(body: dict, x_api_key: str = Header()) -> dict:
    if x_api_key != SECRET:
        raise HTTPException(401)
    return await my_agent(body["messages"], body["variables"], body.get("model"))

In TypeScript, as a Web-standard handler:

// app/duet/route.ts in Next.js; the same handler runs on Bun, Deno, Hono and Cloudflare Workers.
type ToolCall = { id: string; name: string; arguments: Record<string, unknown> };
type Message = { role: "user" | "assistant" | "tool"; content: string; tool_calls?: ToolCall[]; tool_call_id?: string; name?: string };
type Usage = { input_tokens: number; output_tokens: number; model: string };
type Turn = { reply?: string; tool_calls?: ToolCall[]; ended?: boolean; usage?: Usage } | { model_unsupported: string };

// model is undefined except in model checks: run that model and report it in usage, or return { model_unsupported }.
async function myAgent(messages: Message[], variables: Record<string, string>, model?: string): Promise<Turn> {
  throw new Error("call your agent here");
}

export async function POST(request: Request): Promise<Response> {
  if (request.headers.get("x-api-key") !== process.env.DUET_SECRET) return new Response(null, { status: 401 });
  const { messages, variables, model } = await request.json();
  return Response.json(await myAgent(messages, variables, model));
}

#Connection check

Every save of a hosted prompt or endpoint agent sends one customer message to its live configuration, answers its tool calls from the simulated world and shows the turns, or the problem in plain words. A check sends no lead data, so a hosted prompt's {{placeholders}} read [unknown <name>] there and are listed; each stress test scenario fills them. It creates no conversation and takes nothing from your plan's allowances.