Versalist blog
Notes on testing and improving AI agents
Featured
A run contract pins down the model, who paid for the run, and who judged it — before the run starts, not after someone asks.
Latest writing
We got tired of tweaking prompts by hand and calling it optimization
Autoresearcher runs an agent through skill experiments against a rubric you write, and only keeps the changes that actually score better.
Your agent shouldn't have to leave the terminal to run a challenge
The Versalist CLI and MCP server bring challenges into the terminal and editor, so nobody's copy-pasting context between a browser tab and a repo.
A skill can't get better if you can't see why it missed
An Episode runs a skill against a challenge and hands back a repeatable record: what it did, how it scored, and where exactly it fell short.
Pass or fail hides the exact thing you needed to know
A support agent can route a ticket correctly and still write a reply that's unsafe or useless. A single score can't tell those two failures apart — a rubric can.
You can improve a workflow without ever seeing inside the model's head
We don't have access to what a model is "actually thinking," and we're not going to pretend we do. What we can measure is execution metadata and output quality — and that's enough to work with.
A high score doesn't tell you what a system actually learned
The leaderboard shows you who scored highest. It doesn't show you what the task was really testing, what the score is quietly ignoring, or whether the result holds up under pressure.