An Inspectable Substrate for AI Skill Workflows
When capture is enabled, an episode trace exposes bounded event metadata behind a skill score. Trace rows do not duplicate raw model payloads.
Abstract
When a platform reports an AI skill score, users need the supporting evidence. Versalist can provide a trace when capture is enabled. The trace is a read-only projection of ordered episode events.
Trace event rows contain bounded metadata and hashes. They do not duplicate raw prompts or model completions. Episode step rows can retain public-case agent and evaluator outputs. Authorized consumers can identify incomplete capture and compare event shapes.
The problem
Scores are useful only when they can be challenged. A rubric breakdown can explain what was judged, but it does not show how the system moved through the work. The score still arrives as a verdict from a pipeline the evaluator has to trust.
Trace inspection changes the artifact under review. The evaluator can walk through the episode step by step: what call fired, whether it completed or failed, which model handled it, how long it took, and how many tokens moved through the call.
Architecture
Externally, the system reduces to two nouns: episode and trace. Run is a verb and a classifier of episode type, not an object. Trajectory is a forward-looking view across episodes rather than a stored entity. The glossary below states each precisely.
- EpisodePrimitive
- One scored execution of a skill bundle against a challenge. It owns identity, ownership, step scores, outcome, and reproducibility hashes.
- TraceProjection
- A read-only view assembled over one episode. Its identity is the episode identifier. There is no separate trace table.
- RunVerb only
- Useful as a verb and as a classification of episode type, but not a durable object. External language should use episode rather than invent a run entity.
- TrajectoryConcept only
- A future-facing way to describe movement across episodes and events. A view to render at the application layer, not a stored entity.
A trace does not determine how a consumer must use it. Authorized consumers can inspect ordered events, status, hashes, and dropped-event counts. The current trace does not include optimizer, tool, sandbox, approval, or subagent events.
Trace events
An episode trace is an ordered list of captured events. Each event records its type, status, and latency. Model events can include model and token data. This metadata shows the scoring call structure. It does not reconstruct the underlying conversation.
| Seq | Event type | Status | Model | Latency | Tokens in / out |
|---|---|---|---|---|---|
| 01 | agent_turn | completed | — | 1,168 ms | — |
| 02 | model_call | completed | gpt-4o | 842 ms | 1,820 / 420 |
| 03 | judge_call | completed | rubric evaluator | 304 ms | 620 / 110 |
Claim boundaries
The following limits define the current trace surface.
- 01Behavior inspection only: the trace shows what the system did, not everything it saw.
- 02Trace event rows do not duplicate raw prompts, model completions, or full transcripts.
- 03Episode steps can retain agent and evaluator outputs for public test cases.
- 04Service-run private test cases do not retain those outputs.
- 05Hashes support comparison without storing the hashed prompt in trace event rows.
- 06Dropped-event counts identify known incomplete capture.
- 07This is not a tamper-proof audit log, a payload replay system, or a full reward-integrity claim.
Versalist Research (2026). Inspectable Scores: Episode Traces for AI Skill Evaluation.
More from the research notebook
Evaluation Generation as a Post-Selection Step
Most generated challenges never ship, and evaluations written for them are wasted work, so the pipeline defers evaluation generation until promotion. What that buys, what regenerate-not-edit means in practice, and what the generated module is not.
Read note →Skills as Accuracy PrimitivesSkills as Accuracy Primitives
On Versalist, a skill is a named and versioned instruction bundle. Clients can record observations, and proposed changes use recorded history for scored comparisons.
Read note →Agent Training MethodologyThe Adversary Agent: Hostile Review as a Pipeline Step
A second agent, prompted to assume the code is broken until proven otherwise, reviews changes it did not write. We describe the prompt shape, how the step is enforced, and what happened the day it returned nothing.
Read note →