Datasets and training

#Datasets

Add production conversations, simulated conversations and tests to a dataset from a conversation, an issue or a production search; Datasets in Models adds every test. The tests a dataset trains on are its own and those of the issues its conversations show. Build tests reproduces each untested issue its production conversations show. A production conversation leaves its datasets when retention deletes it.

#Exports

DPO

Together's preference JSONL: for each failed conversation, the first reply the test's verifier already fails as rejected, and a reply written by the proven fix's patched prompt, or the verifier's reference answer, that passes on the same prefix as chosen.

SFT

Conversational JSONL of the passed replays and each pair's chosen reply, under the live instructions; for a distilled build, the teacher replies the signals passed.

RL environments

One JSON object per hardened test: the agent, the customer simulator, the hidden world, the situation and trap, and the verifier whose verdict is the reward.

Production conversations train only when the workspace's training consent is on (Settings > General, off by default); everything else is simulated.

#Distill from production

Distill from production samples prefixes of the dataset's production conversations, each ending right before an agent reply, and has an open teacher Together serves write several candidate replies on each. The agent's judged signals of its own failures grade every candidate: a reply none of them hits becomes an SFT line, and a passing reply beside a failing one on the same prefix becomes a preference pair. The build states how many prefixes it sampled, how many replies passed and how many each signal rejected. OpenAI, Google and Anthropic models are never teachers: their terms forbid training models that compete with theirs on their output. It needs the workspace's training consent.

#Train, deploy and verify

For hosted prompt agents, choose the base (Together's list of trainable bases, read daily), the method and every hyperparameter (epochs, learning rate and its schedule, warmup, weight decay, gradient clipping, sequence length, batch size, packing, and LoRA rank, alpha, dropout and modules; leave packing off for a small dataset), and optionally an earlier trained model to continue from. Together prices exactly that fine-tune first; nothing is billed until an owner or admin accepts the estimate.

Once trained, Deploy and verify serves the trained model and its base on Together dedicated inference, each on its own private endpoint with an idle timeout, replays every test on both once they are ready, pooled as a fix proof (proven, suggestive, not improved or inconclusive, with the regressed tests), and tears down what was deployed only to verify. Keep it as a candidate to leave the trained model serving. Each served model shows its state, its price per replica-hour and what it has cost so far; Tear down stops it and deletes its deployment and endpoint.