Skills as Accuracy Primitives

When capability lives in one long system prompt, an error has no address: you can feel that accuracy dropped, but there is nothing to inspect, version, or roll back. A skill on Versalist gives accuracy a location. Skills are named, versioned bundles with a recorded life: every run writes an observation, observations become test cases, and a proposed amendment is evaluated against the skill's own history before it can replace the current version. This note describes the loop as built, the thresholds that decide promotion, and the limit we want understood up front: the evaluation is a judged comparison, not a re-execution.

The common failure of agent systems is not that they lack instructions, it is that the instructions form a single undifferentiated mass. When something goes wrong inside a two-thousand-line system prompt, the fix is an edit made on intuition, and the regression it causes lands somewhere else in the mass. Nothing carries a version. Nothing can be rolled back.

A skill, in our design, is the unit that fixes this: a named bundle of instructions and tool access, carrying a semantic version, owned, forkable, with lineage from parent to fork. The registry treats a skill the way a package index treats a package, including a verification tier that requires a hostile review report before a bundle can be marked verified.

Naming the unit is not enough; the unit has to remember its own use. Every run of a skill records an observation: the task it was asked to do, the task category, the outcome, a score when one exists, the error type on failure, execution time, which tools were called and which tool calls failed, and any feedback the user gave.

The observation stream is what turns improvement from an aesthetic exercise into a testable one. A claim that a new version is better has to mean better on these, the tasks this skill actually faced.

A proposed change to a skill, whether generated by the system or written by hand, is an amendment, and an amendment starts with no authority. To be promoted it is evaluated against test cases drawn from the skill's recorded history, and the selection is deliberately failure-weighted, up to seven past failures against three past successes. The successes are there to catch regressions; the failures are what the amendment exists to fix.

• 01The evaluation scores the original and the amended version per test case and per dimension, and counts cases improved, regressed, and unchanged.
• 02The verdict rules are explicit. More regressions than improvements is regressed, full stop. Improved requires the score delta to clear 0.05 with more cases improved than regressed. A delta inside 0.05 with balanced cases is neutral.
• 03A skill with no usable history is inconclusive: it cannot be evaluated, so it cannot improve. An unused skill is not a free option, it is a dead end, and the system treats it as one.
• 04Automatic promotion happens only on an improved verdict. Promotion bumps the version; rollback exists and is expected to be used.

The evaluation is performed by a model reasoning about which version would have handled each test case better, at low temperature, in a fixed output schema. It is a judged comparison, not a re-execution of the original tasks, and a judge inherits a judge's biases. We would rather say that plainly than let the promotion record imply a rerun that never happened.

We also do not claim a controlled study showing that skills beat long prompts across deployments. That study has not been run. The claim we make is narrower and inspectable: every promoted version carries the scored comparison that justified promoting it, and you can go read it.

• 01A skill should answer a question the agent would otherwise ask itself. If no run ever routes to it, it is not scope, it is clutter in the routing decision.
• 02Two skills that always activate together are one skill. Boundaries follow the shape of the work, not the shape of the team that wrote them.
• 03The observation stream is the asset. A skill whose runs are not observed is back to being a prompt, whatever we call it.