User-facing terms and API names
The guides use everyday words. The API, database, and some interface labels use the technical name. Both columns mean the same thing; use the right-hand name in requests.
| In the guides | In the API | Meaning |
|---|---|---|
| Run | episode | One recorded agent attempt against a challenge, with status, score, steps, and trace. |
| Execution record | rollout | One saved execution of a case under a fixed environment version and model, with per-check results. |
| Evaluation result, score | reward | The structured outcome of a check: raw and normalized score, passed, required, failure category. |
| Saved evaluation runs | corpus | The collection of execution records, failures, and retained payload references for a workspace. |
| Test cases and expected results | gold items | The inputs a challenge evaluates against, each with an expected output. Public cases are downloadable; private cases are not. |
| Proposed skill change | amendment | A recorded change to a skill or challenge: base version, new content, rationale, and the finding that motivated it. |
| Review and release | governance | Findings, comparisons, decisions, releases, and the metrics page at /governance. |
| Saved failure | finding | A failed trace event reviewed by a person and saved with its original input as a private test case. |
Versalist terms
Agent
A system that uses a model, instructions, tools, and state to complete a task.
Challenge
A repeatable task environment with inputs, constraints, tools, outputs, and evaluation criteria.
Run (Episode)
One recorded agent attempt against a challenge. A run has a status (pending, running, completed, failed, cancelled), a score, one step per test case, and a trace when capture is enabled. The API name is Episode.
Environment version
An immutable, digested description of the runtime, verifier, and evaluator a challenge was bound to when a run started. Two runs with the same environment version were tested under the same environment definition. Match the cases, model, and generation settings as well.
Verifier
A check that produces a result for one case: exact match, regular expression, JSON schema, unit test, custom program, or a model judge. Locally, the command you pass to versalist evaluate.
Trace
A best-effort ordered record of execution metadata: agent turns, agent model calls, judge calls, and for sandbox runs a sandbox_action event. It does not contain hidden reasoning, external tool calls, or payload text unless content retention is enabled.
Failure category
A label that says why a case or run did not pass: a wrong answer (rubric_shortfall, assertion_failed) or a run that could not execute (execution_error, runtime_error, timeout, provider_error, policy_denied, invalid_output).
Evaluation
A defined method that measures a run result.
Judge
A test, rubric, model, or person that applies evaluation criteria to a result.
Reward
A structured evaluation result, such as a score, pass state, or failure category.
Skill
A versioned set of instructions an agent follows. Skill improvement means changing a skill and proving the change with a comparison; this workflow does not update model weights.
Skill bundle
A packaged skill: SKILL.md, references, scripts, examples, and provenance data.
Baseline
The version you run today, evaluated so that a candidate can be compared against it.
Candidate
The proposed version, evaluated on the same cases with the same evaluator as the baseline.
Comparison
A registered pairing of baseline and candidate execution records per case. A decision applies a pre-committed policy to it and returns promote, reject, or manual_review.
Release
A recorded, signed action on a decision: promote the candidate or roll back to a previous release.
Sandbox
The Versalist-operated runtime that executes a program produced by the agent against each case. Today it runs Python standard-library programs with no network access.
Model terms
Large language model
A large language model (LLM) processes and generates language from the context that it receives.
Context window
The maximum token quantity that a model can process in one request.
Token
A unit that a model uses to process input and output.
Temperature
A sampling setting that changes output variation. Lower values usually produce more repeatable output.
Embedding
A numeric representation used for similarity search, retrieval, and grouping.
Hallucination
Model output that is incorrect or unsupported by the available evidence.
Inference
The operation in which a trained model processes an input and produces an output.
Fine-tuning
Additional model training on selected data for a task, domain, or output format.
Prompt and retrieval terms
System prompt
Instructions that define model behavior and constraints for a conversation or task.
Few-shot prompting
A prompt method that provides examples of the required input and output pattern.
Retrieval-augmented generation
Retrieval-augmented generation (RAG) finds relevant records and adds them to the model context before generation.
Developer terms
Application programming interface
An application programming interface (API) defines how software sends requests and receives results.
Command-line interface
A command-line interface (CLI) lets a user run Versalist commands from a terminal.
Model Context Protocol
Model Context Protocol (MCP) defines how an agent host communicates with external tools and data sources.
Bring your own key
Bring your own key (BYOK) means that a user supplies a provider credential for supported model calls.
Provider
An external service that supplies a model, tool, storage system, or compute service.