Review studio

#Queues

A queue lists the agent's ended production conversations nobody has reviewed yet. Random is a stable shuffle. Grader fails first puts the conversations the published graders failed first, then those they passed, then those they did not grade. Disagreements keeps only conversations where the published graders and the production judge disagree. The last two need a published grader.

The address names the queue and the conversation on screen, so a link or a reload lands on the same one. Review this conversation on a conversation's record, or on its Traffic row, opens the studio on it, then continues the queue. Moving away keeps what you wrote: a violet dot on the counter and the step buttons marks another conversation with an unsaved draft, and saving clears it.

j / k

Next, previous conversation

p / f

Pass, fail

c

Write the critique

Enter

Save (Shift+Enter is a new line in the critique)

#Reviews

Each review needs a verdict and a critique, whatever the verdict. Each member keeps one review per conversation, and every change is in the audit log. A fail can be mapped to the agent's current failure categories; left unmapped, it waits for the next Group failures.

Reviewed conversations split by a hash of the conversation into the reviews that build the graders and those held out to measure them, so a conversation never moves between them. Build critiques become graders' examples; held-out reviews are never shown to them.