You can improve a workflow without ever seeing inside the model's head
We don't have access to what a model is "actually thinking," and we're not going to pretend we do. What we can measure is execution metadata and output quality — and that's enough to work with.
A workflow can produce a fine answer today and be nearly impossible to improve tomorrow, because nobody kept a record of what happened during the run. Teams end up changing prompts on instinct, without any real way to know which change actually helped and which one just happened to coincide with a good day.
Start with what the runtime actually gives you
We record bounded execution metadata — which stage ran, its status, whatever timing data is available. That kind of trace is genuinely useful for tracking down where a call failed. It is not a replay of the run and it's not a window into hidden model reasoning, and we're careful not to describe it as either, because it isn't.
Decide what "good" means before you start optimizing
Schemas, business rules, and rubrics — pick whichever combination fits the workflow — need to exist before you start comparing outputs. Optimizing against a fuzzy sense of quality just means you're optimizing against your own mood that day.
Use the comparison to pick the next change, not to declare victory
Run different versions against the same test cases and keep whichever one actually moves the criteria you defined. Anything else is just guessing with extra steps.
This is exactly how we built Episodes
Versalist applies structured evaluation to every challenge Episode. There's also a trace prototype that records limited model and judge call metadata — it's off by default, because we'd rather ship something narrow and honest than something that overpromises what it can see.
Pass or fail hides the exact thing you needed to know
A support agent can route a ticket correctly and still write a reply that's unsafe or useless. A single score can't tell those two failures apart — a rubric can.
We got tired of tweaking prompts by hand and calling it optimization
Autoresearcher runs an agent through skill experiments against a rubric you write, and only keeps the changes that actually score better.
Two people ran the same challenge and got two different scores. Here's why that shouldn't surprise you
A run contract pins down the model, who paid for the run, and who judged it — before the run starts, not after someone asks.