Skills as Accuracy Primitives

A named, versioned skill gives an error a location. Clients can record observations that become test cases. A proposed amendment uses this recorded history before it can replace the current version. The evaluation is a judged comparison, not a re-execution.

The common failure of agent systems is not that they lack instructions, it is that the instructions form a single undifferentiated mass. When something goes wrong inside a two-thousand-line system prompt, the fix is an edit made on intuition, and the regression it causes lands somewhere else in the mass. Nothing carries a version. Nothing can be rolled back.

A skill, in our design, is the unit that fixes this: a named bundle of instructions and tool access, carrying a semantic version, owned, forkable, with lineage from parent to fork. The registry treats a skill the way a package index treats a package, including a verification tier that requires a hostile review report before a bundle can be marked verified.

Clients can record a task, category, outcome, score, error, duration, tool metadata, and user feedback as an observation.

The current runtime does not record every skill run or tool call automatically.

The observation stream is what turns improvement from an aesthetic exercise into a testable one. A claim that a new version is better has to mean better on these, the tasks this skill actually faced.

A proposed change to a skill, whether generated by the system or written by hand, is an amendment, and an amendment starts with no authority. To be promoted it is evaluated against test cases drawn from the skill's recorded history, and the selection is deliberately failure-weighted, up to seven past failures against three past successes. The successes are there to catch regressions; the failures are what the amendment exists to fix.

• 01The evaluation scores the original and the amended version per test case and per dimension, and counts cases improved, regressed, and unchanged.
• 02The verdict rules are explicit. More regressions than improvements is regressed, full stop. Improved requires the score delta to clear 0.05 with more cases improved than regressed. A delta inside 0.05 with balanced cases is neutral.
• 03A skill with no usable history is inconclusive: it cannot be evaluated, so it cannot improve. An unused skill is not a free option, it is a dead end, and the system treats it as one.
• 04Automatic promotion happens only on an improved verdict. Promotion bumps the version; rollback exists and is expected to be used.

The evaluation is performed by a model reasoning about which version would have handled each test case better, at low temperature, in a fixed output schema. It is a judged comparison, not a re-execution of the original tasks, and a judge inherits a judge's biases. We would rather say that plainly than let the promotion record imply a rerun that never happened.

We also do not claim a controlled study showing that skills beat long prompts across deployments. That study has not been run. The claim we make is narrower and inspectable: every promoted version carries the scored comparison that justified promoting it, and you can go read it.

• 01A skill should answer a question the agent would otherwise ask itself. If no run ever routes to it, it is not scope, it is clutter in the routing decision.
• 02Two skills that always activate together are one skill. Boundaries follow the shape of the work, not the shape of the team that wrote them.
• 03The observation stream is the asset. A skill whose runs are not observed is back to being a prompt, whatever we call it.