Send conversations

Duet Machines setup prompt 2026-11-01.2

Instrument this repository's conversational AI agent so every conversation it has is sent to Duet Machines (https://platform.duetmachines.com), which judges each one and proves failures with quotes. The Duet Machines agent uid is <agent uid>.

Ground rules
- Add recording only: do not change what the agent says or does. A Duet Machines failure must never raise into the agent; catch it, log it, continue. When DUET_API_KEY is not set at runtime, recording is skipped with one logged warning; the agent still starts.
- Keep the diff minimal: one small module plus the calls that feed it. Do not commit; I review the diff.
- Secrets: the code reads the API key at runtime from the environment variable DUET_API_KEY. Never write the key, or any other secret, into a file of this repository (code, .env, tests, docs, fixtures) or a commit message. You may add an empty DUET_API_KEY= line to an existing .env.example. If DUET_API_KEY is not set in your shell, stop and ask me to export it (the agent's Connect panel in Duet Machines creates one and shows it once).
- First run git status. If the working tree has uncommitted changes, list them and ask me before editing.

1. Detect the stack. Read the dependency files and the code that calls the model, tell me what you found, and follow exactly one path:
   A. OpenAI Agents SDK (openai-agents; from agents import ...).
   B. Vercel AI SDK (the ai package; generateText or streamText).
   C. LangChain or LangGraph.
   D. LiveKit Agents (livekit-agents).
   E. Raw OpenAI or Anthropic clients, or any other code that keeps a message list.
   F. Code that already exports OpenTelemetry spans in the GenAI conventions, OpenInference (Arize, Phoenix) or OpenLLMetry (Traceloop).
   G. Vapi or Retell: no code. Tell me to set the webhook from the agent's Connect panel (Vapi https://platform.duetmachines.com/api/integrations/vapi/<agent uid>, Retell https://platform.duetmachines.com/api/integrations/retell/<agent uid>) and stop.

2. Paths A to E: write one small module. There is nothing to install: Duet Machines' SDKs are not published to PyPI or npm, and those registry names are not Duet Machines', so never run pip install duetmachines or npm install @duetmachines/sdk. With the language's standard HTTP client, the module POSTs the whole conversation so far as JSON to https://platform.duetmachines.com/api/ingest/conversations with the header Authorization: Bearer <DUET_API_KEY from the environment> and a 10-second timeout, in the background or after the reply is sent. Body: {"agent": "<agent uid>", "external_id", "user_id", "agent_version", "started_at" (ISO 8601 with offset), "turns": [{"role": "customer" | "agent", "text", "at", "usage"} or {"role": "tool", "text": "", "at", "duration_seconds", "tool": {"name", "arguments": {...}, "status": "executed" | "failed" | "blocked", "result": <string>}}], "ended", "config", "kind"}. Duet Machines keeps the latest post per external_id; 202 means stored, and any other status is logged with its body.

3. Call it after every turn, from where the path's conversation lives.
   A. OpenAI Agents SDK: after each Runner.run, map result.to_input_list() (user and assistant messages, function calls with their outputs) to turns, with the agent's instructions and function tools as the config and the app's conversation or thread id as external_id. Time a tool call inside the function tool if the code can; the run's items carry no timing.
   B. Vercel AI SDK: after each generateText, or once a streamText's response has resolved, map [...messages, ...result.response.messages] to turns, with the system prompt and each tool's name, description and JSON schema as the config; in a serverless handler, await the post before returning. Messages carry no tool timing.
   C. LangChain or LangGraph: after each turn, rebuild the conversation from the graph's message state: customer for each HumanMessage, agent for each AIMessage's text, a tool turn for each ToolMessage with its AIMessage tool call. Time tool calls where the code runs them.
   D. LiveKit Agents: before session.start(), subscribe to the session's conversation_item_added (user text a customer turn, assistant text an agent turn), function_tools_executed (each call and output a tool turn, duration_seconds from output.created_at minus call.created_at, status failed when output.is_error) and close (ended: true) events; https://platform.duetmachines.com/docs/platforms#livekit shows the event shapes.
   E. Raw OpenAI or Anthropic clients, or any other message list: wherever the code appends the model's reply to its list.
   F. Point the exporter at Duet Machines: OTEL_EXPORTER_OTLP_ENDPOINT=https://platform.duetmachines.com/api/otlp, OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf and OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer%20$DUET_API_KEY,x-duet-agent=<agent uid>", set in the deployment, never in a committed file; record message content. Group each conversation with gen_ai.conversation.id (OpenInference: session.id) and set duet.conversation.ended=true on its last span. The attributes read are at https://platform.duetmachines.com/docs/opentelemetry.

What every path sends:
- external_id: the app's own conversation id, the same on every turn.
- user_id: a pseudonymous id, never a name, email or phone number.
- agent_version: the release, when the app has a version or git sha setting.
- config: {"instructions": the system prompt as written, "tools": the tool definitions the model gets (OpenAI or Anthropic format; [] for none), "variables": {...}, "model": the model id the code calls with that prompt}. Model checks and reproductions replay your logged prompt and tools on that model, and a new model there records a switch. If the prompt is filled with per-conversation data, send the template with each value as a {{name}} placeholder and the values in variables, never the filled text.
- Each tool call: its name, arguments, status, result and, where your code runs the tool, duration_seconds measured around it.
- usage on each agent turn: {"input_tokens", "output_tokens", "model", and "cached_input_tokens" when the response reports cache reads (OpenAI usage.prompt_tokens_details.cached_tokens; Anthropic: input_tokens + cache_read_input_tokens + cache_creation_input_tokens as input_tokens, cache_read_input_tokens as cached_input_tokens)} exactly as the model call's response reports them. Cost per conversation and spend by model are priced from it.
- If this repository also serves Duet Machines' endpoint contract (a POST handler taking conversation_id, variables and messages), honor its optional "model": when present, run that turn on the named model instead of the configured one and return "usage" whose "model" is that exact requested id; if the code cannot run it, return {"model_unsupported": "<why>"} alone.
- Only what the code knows: never invent or derive a value to fill a field (no hand-written durations, no user id hashed from a name, no made-up version). Leave a field out when the app has no value for it, and say so in your summary.
- ended: true when the conversation is over; otherwise Duet Machines ends it after 30 minutes without a new turn.

4. Send a check. Run one conversation through the code you added with kind "check" (the module's "kind": "check"; OpenTelemetry duet.conversation.kind=check) and the id setup-check-<8 random hex characters>. Prefer running the agent itself on one customer message; if it needs credentials that are not set, call your module directly with a short hand-written exchange that includes a tool call if the agent has tools. Never contact real customers or real external systems. A check is listed in Duet Machines but never judged or counted.

5. Confirm it arrived: curl -s -H "Authorization: Bearer $DUET_API_KEY" "https://platform.duetmachines.com/api/agents/<agent uid>/production-conversations?kind=check&q=<your check id>", then GET https://platform.duetmachines.com/api/production-conversations/<its uid> with the same header. Report the turns, whether the config arrived (instructions, tool count, variables), which tool calls carry duration_seconds, and anything missing with the reason. If nothing arrived within 30 seconds, show me the logged error.

6. Finish. List the files you changed, how to run them, what I must set where the agent is deployed (DUET_API_KEY), and what was not recorded and why. Then tell me the next step: real conversations go without kind and each is judged when it ends, within the judged conversations the workspace's plan allows a month; https://platform.duetmachines.com/agents/<agent uid>/traffic shows them as they arrive, and https://platform.duetmachines.com/agents/<agent uid>/tests runs a stress test before launch.

claude "$(curl -fsSL https://platform.duetmachines.com/setup-prompt/<agent uid>)"
codex "$(curl -fsSL https://platform.duetmachines.com/setup-prompt/<agent uid>)"

curl -X POST https://platform.duetmachines.com/api/ingest/conversations \
  -H "Authorization: Bearer $DUET_API_KEY" -H "Content-Type: application/json" \
  -d '{
    "agent": "<agent uid>",
    "external_id": "conv_8f2a",
    "kind": "conversation",
    "user_id": "user_193",
    "agent_version": "a1b2c3d",
    "started_at": "2026-10-11T14:02:11Z",
    "turns": [
      {"role": "customer", "text": "Can I move my payment to Friday?", "at": "2026-10-11T14:02:11Z"},
      {"role": "tool", "text": "", "at": "2026-10-11T14:02:12Z",
       "tool": {"name": "get_account", "arguments": {"id": "193"}, "status": "executed", "result": "{\"due\": \"2026-10-14\"}"}},
      {"role": "agent", "text": "Yes, I can move it to Friday the 16th.", "at": "2026-10-11T14:02:13Z"}
    ],
    "ended": false,
    "metadata": {"channel": "web"},
    "config": {
      "instructions": "You are Riley, a billing assistant. Never move a due date more than 7 days.",
      "tools": [{"name": "get_account", "description": "Look up an account.",
                 "parameters": {"type": "object", "properties": {"id": {"type": "string"}}}}]
    }
  }'

import json, os, urllib.request


def send_to_duet(conversation: dict) -> None:
    """Send the whole conversation so far; Duet Machines replaces what it had under the same external_id."""
    request = urllib.request.Request(
        "https://platform.duetmachines.com/api/ingest/conversations",
        data=json.dumps(conversation).encode(),
        headers={"Authorization": f"Bearer {os.environ['DUET_API_KEY']}", "Content-Type": "application/json"},
    )
    urllib.request.urlopen(request, timeout=10)


# After every turn, and once more with "ended": True when the conversation ends.
send_to_duet({
    "agent": "<agent uid>",
    "external_id": conversation_id,
    "user_id": user_id,
    "agent_version": os.environ.get("GIT_SHA"),
    "started_at": started_at.isoformat(),
    "turns": [{"role": "customer" if m["role"] == "user" else "agent", "text": m["content"]} for m in messages],
    "ended": False,
    # What the agent ran with, so findings cite only rules it had; the same config is stored once.
    "config": {"instructions": SYSTEM_PROMPT, "tools": TOOLS},
})

# Up to 100 conversations per request, each answered on its own:
curl -X POST https://platform.duetmachines.com/api/ingest/conversations:batch \
  -H "Authorization: Bearer $DUET_API_KEY" -H "Content-Type: application/json" \
  -d '{"conversations": [{"agent": "<agent uid>", "external_id": "conv_8f2a", ...}, ...]}'
# [{"status": 202, "conversation": "<uid>", "error": null},
#  {"status": 422, "conversation": null, "error": "started_at: Field required"}]

{
  "type": "function",
  "function": {
    "name": "report_trouble",
    "description": "Report trouble you are having in this conversation, when it happens: you cannot do what the customer asked, a tool failed you, or you are unsure of your answer. The customer never sees it.",
    "parameters": {
      "type": "object",
      "properties": {
        "kind": {"type": "string", "enum": ["cannot_do", "tool_failed", "unsure"]},
        "detail": {"type": "string", "description": "What happened, in one sentence."}
      },
      "required": ["kind", "detail"],
      "additionalProperties": false
    },
    "strict": true
  }
}

from duetmachines import REPORT_TROUBLE_TOOL  # TypeScript: import { REPORT_TROUBLE_TOOL } from "@duetmachines/sdk"

tools = [*TOOLS, REPORT_TROUBLE_TOOL]  # what the agent may call; answer report_trouble with "reported"

# When the agent calls it, record the call like any tool call, or with the helper:
conversation.report_trouble("tool_failed", "The payment service timed out twice.")  # reportTrouble(kind, detail)