Pass or fail hides the exact thing you needed to know
A support agent can route a ticket correctly and still write a reply that's unsafe or useless. A single score can't tell those two failures apart — a rubric can.
Picture a support agent that routes a billing question to the right queue every single time, but writes replies that are confusing or occasionally just wrong. A pass/fail score can't hold both of those facts at once. It collapses "routing works" and "the reply is bad" into one bit of information, and you lose the thing you actually needed to know.
A rubric names what actually matters, separately
Versalist rubrics split quality into dimensions — routing accuracy, factual correctness, tone, whether the response is actually actionable, whatever applies to the task. Each dimension can be scored as binary (did it or didn't it) or on an ordinal scale, depending on what makes sense.
Not every dimension deserves equal weight
Getting the tone slightly off is not the same category of problem as giving a factually wrong answer that costs someone money. Authors can weight dimensions to reflect that. Scale definitions spell out what a given score actually means, and Gold Items give you real test cases with known-correct answers to calibrate against.
The breakdown is what tells you what to fix next
The total score tells you how the run went overall. The dimension breakdown tells you whether the problem is the instructions, the workflow, or something structural — which is the difference between guessing at a fix and actually making one.
Write the rubric before you start comparing versions
If you're about to A/B a couple of prompts or agent versions, define the criteria first. Otherwise you'll end up picking the "better" one based on vibes, which is exactly the problem rubrics exist to solve.
A skill can't get better if you can't see why it missed
An Episode runs a skill against a challenge and hands back a repeatable record: what it did, how it scored, and where exactly it fell short.
You can improve a workflow without ever seeing inside the model's head
We don't have access to what a model is "actually thinking," and we're not going to pretend we do. What we can measure is execution metadata and output quality — and that's enough to work with.
Two people ran the same challenge and got two different scores. Here's why that shouldn't surprise you
A run contract pins down the model, who paid for the run, and who judged it — before the run starts, not after someone asks.