Quickstart

#The flow

Each agent's pages follow the work in order:

1. Traffic. Send production conversations and the business outcomes they lead to: one request per conversation, an SDK, OpenTelemetry or a platform webhook. See Send conversations and Business outcomes.
2. Review. Your expert reads real conversations one at a time in the studio, marks each pass or fail and writes why. A model groups the fail critiques into failure categories the expert edits and approves. Failures the judge proves with verbatim quotes wait in Issues to be accepted or dismissed. See Review studio and Issues.
3. Grader. One grader per failure category, built from the expert's critiques and measured on conversations it never learned from: how often it catches the expert's fails and passes the passes, each with its interval. A published grader grades every production conversation and is calibrated against outcomes. See Graders and Signals.
4. Tests. Each accepted failure that recurs when its conversation is replayed becomes a test: the scenario plus a verifier hardened against disguised repeats. Stress tests attack the agent for failures production has not shown yet. See Stress tests and Fixes.
5. Models. Cheaper models are priced on the agent's real tokens and checked on its tests before any switch; datasets train an open model on its own traffic. See Model checks, Savings and Datasets and training.
6. Rollout. The OpenAI-compatible gateway serves the agent, shadows a candidate on the same requests, then splits conversations at random, advancing only when gates on cost, your standard and your outcomes pass. See Gateway and rollout.
7. Report. The migration report states from the agent's own data which cheaper model it can move to, what that saves each month and what it rests on, as one sheet to download or share with whoever decides. See Migration report.

Each agent's Overview is its migration cockpit: where the migration stands and its next step, its monthly model cost today against the best proven candidate, its grader pass rate and outcome rates, where the rollout stands, and the migration report. Duet Machines is paid for the inference its models serve: a share of what your current model charges for the same tokens. See Plans and allowances.

#Words we use

Every page, email and API answer uses these words with one meaning each:

Agent

One AI workflow you run at volume: its instructions, tools, traffic and everything below.

Conversation

One exchange between the agent and a customer: production ones you send, simulated ones Duet Machines plays against the agent.

Grader

A check for one failure category, built from your expert's critiques and measured against their labels. Its pass or fail on a conversation is a grade.

Test

A scenario from a real failure or a stress test, with a verifier. Replaying the agent's tests on a model is how a candidate is checked.

Candidate

A cheaper model the agent could move to. The current model is the one it runs today (incumbent in the API).

Rollout

Moving traffic from the current model to a candidate through the gateway: passthrough, shadow, split, full.

Gate

A live comparison of the candidate with the current model, on cost, quality, outcomes, latency or errors. A rollout advances only when every gate passes.

Report

The migration report: which candidate to move to, what it saves and the evidence, as one fixed sheet.

Signal

The rate of one behavior across judged conversations, on the agent's Grader. See Signals.

#Your first agent

Create an account at /signup; it creates a workspace you own. An empty workspace opens on Home, which starts from production traffic:

Production traffic

Pick how it arrives (your coding agent, the Python or TypeScript SDK, OpenTelemetry, the gateway or a JSONL import) and name the agent; its Connect panel opens on that way with a new API key and the snippets filled in (Set up with your coding agent is the fastest). The panel listens for the first conversation and, once it arrives, opens it and says where its judging stands.

No traffic yet

Connect the agent's prompt or endpoint and run a first stress test from its Tests; connect its traffic when it launches.

To fill in an agent, paste its system prompt or a few real conversations (OpenAI messages as JSON, JSONL or CSV). Duet Machines drafts its name, instructions, tools, lead variables and runtime notes in one pass; review them before you save. See Connect an agent.

Review conversations in the agent's Review studio with the keyboard: P passes, F fails, C writes the critique, Enter saves and opens the next. In Issues, A accepts an issue and D dismisses it with a reason; J and K move.

#Next steps

• Declare the outcomes your systems see after a conversation, and send them.
• Group the fail critiques and publish graders once each has enough held-out labels.
• Check a cheaper model against your tests before switching.
• Serve the agent through the gateway and roll the candidate out behind gates.
• Build the migration report and share it with whoever decides.