From a fixed task to a release decision
Define a task, evaluate agent behavior, inspect evidence, and compare changes before release.
Four terms
- Challenge
- A fixed task. It defines the inputs, the expected behavior and how results are scored.
- Episode
- One scored run of a skill and a model against the challenge cases.
- Evidence
- Scores per rubric dimension and per case. With trace capture on, bounded call metadata too.
- Comparison
- Run the changed skill on the same cases. A person reads both results and decides what ships.
Trace events are best-effort; a missing event proves nothing. Capture limits · Where your agent runs
Challenges, docs and guides are public. You sign in when you want to run or save something.