Evaluation Loop | Versalist

Versalist uses the term training for instruction and skill changes. It does not mean model weight updates.

Evaluation sequence

Define the challenge. Record inputs, constraints, tools, required artifacts, and acceptance criteria.
Run an Episode. Execute the agent and record model calls, tool calls, logs, and outputs.
Judge the result. Apply tests, rubric criteria, baseline comparisons, or reviewer decisions.
Record the reward. Store the score, result status, failure mode, and supporting evidence.
Update the skill. Change an instruction only when the evidence supports the change.

A challenge defines a repeatable task environment. Its fixed boundary makes different runs comparable.

An Episode records one agent attempt. The record can include trace events, logs, intermediate files, and final artifacts.

An evaluation applies a defined method to an Episode result. Use deterministic tests when the task permits them.

A reward converts the evaluation into a structured result. It can contain a score, pass state, failure category, and evidence links.

A skill version contains reusable agent instructions. Its provenance must identify the evidence that caused the change.

A score change without trace evidence can hide a regression. Review both the result and the execution behavior.

Provider keys let Versalist send supported model calls through a user-managed account. Manage these keys in Integrations.

This capability is planned. The current product does not expose a general endpoint adapter.

This capability is planned. A catalog entry does not mean that a runtime adapter exists.