Back to blog
Versalist blog

A skill can't get better if you can't see why it missed

An Episode runs a skill against a challenge and hands back a repeatable record: what it did, how it scored, and where exactly it fell short.

Teal line graph with a small hand-drawn person connecting a skill to a score
EpisodesEvaluation

You can't improve a skill by staring at it. You improve it by running it, watching where it breaks, and changing the one thing that's causing the break. That sounds obvious, but most people skip straight from "the output feels off" to "let me rewrite the prompt," without ever pinning down which part actually failed. An Episode is our attempt to force that middle step.

The challenge sets the test, the skill sets the attempt

A challenge supplies the task, the constraints, the test cases, and the criteria it'll be judged against. Your skill supplies the instructions that guide the agent through it. Keeping those two things separate is what makes an Episode repeatable — you can swap the skill and rerun the exact same test.

The score comes with a breakdown, not just a number

A single total score tells you whether things are getting better or worse overall. The rubric dimensions underneath it tell you which part to actually go fix. That second part is the one that matters if you're trying to improve something rather than just track it.

Baselines give you something to measure against

We record the baseline model and the date it was run, so when your skill scores higher, you know it's actually your skill doing the work — not a stronger model that shipped last month making everything look better.

The loop is just: run, read, edit, repeat

Run the skill. Read what the dimension scores are telling you. Edit the one thing that's clearly weak. Run it again. It's not complicated — it just requires the record to actually exist, which is the part most workflows skip.

Related posts