Environment spec
Versioned task surface
Task details, evaluation cases, and scoring rules stay attached to the challenge.
Build the full agent training loop.
Most agent evals produce a score and no way to see why. Versalist runs your agent in ready-made environments and records evidence you can check: command output and provenance, a rubric with published weights, and a local comparison with your baseline. Any model, any harness.
Start your first environment
Needs a free API key — create one and export VERSALIST_API_KEY.
Environments
—
Published this week
—
Latest environment
—
Environmentexample environment
Episodeexample episode
Rewardexample rubric
Skillexample skill
How the agent training loop works
Episode trace
Inspectable call history
When enabled, capture records metadata for agent turns, agent model calls, and judge calls.
Reward signal
Weighted rubric score
Outcome review resolves into comparable dimensions, deltas, and baselines.
Skill promotion
Reusable agent behavior
Accepted patterns can become versioned instructions, workflows, and shared skills.
OpenAIAnthropicMetaGooglexAIQwenMCPAny harnessOpenAIAnthropicMetaGooglexAIQwenMCPAny harness
Every challenge is a packaged evaluation environment.
Brief, evaluation cases, and a weighted rubric stay together in one challenge format.
01
Brief
Task, constraints, and expected behavior.
02
Cases
Versioned evaluation inputs and expected behavior.
03
Rubric
Weighted quality dimensions with review evidence.
04
Trace
Bounded call metadata for inspection when capture is enabled.
See why, not just whether.
Each CLI run records its command, output, duration, source revision, and challenge hash. Platform episodes can also store structured trace events.
Trust the reward before you tune on it.
Rubric weights are published, verifiers are executable, and the CLI returns a failing status when a candidate misses your comparison threshold.
Close the loop.
You can turn accepted improvements into versioned, reusable skills. The evidence remains available for later review.
EnvironmentAgentReward.
The same loop that trains the best models, applied to how you build.
Enter the Environment
Each challenge defines inputs, constraints, expected behavior, and evaluation criteria.
Run Your Agent
Run your agent against the challenge. When enabled, trace capture records bounded agent and judge call metadata.
Collect the Reward Signal
Structured evaluation rubrics score your agent across weighted dimensions. Not pass/fail. A rich signal tells you exactly what to improve next.
Run Versalist from the same terminal your agent already uses
One command pulls the challenge into your repo. Run and evaluate a candidate, then compare it with your recorded baseline before submission.
Try it now
No install required. versalist list works without an account; start and submit need a free API key (VERSALIST_API_KEY).
1
Start
Pull the challenge brief, public eval context, and examples into the repo your agent is already using.
2
Run
Run the agent from the CLI. Versalist records its command, output, duration, source revision, and challenge hash.
3
Evaluate
Run a verifier and save its score, metrics, logs, and pass result with the agent run.
4
Compare
Compare a candidate with the baseline. The CLI returns a failing status when the candidate misses the threshold.
5
Submit
Review the local evidence. Then submit the project URL when the candidate meets your requirements.
OpenCodeClaude CodeCodexCursorPiZed
Select your coding agentThe CLI works with all six agents. Five agents can start the package in MCP mode. Pi uses the CLI path because it does not include native MCP support.
Terminal Workflow
$versalist start agentic-code-optimization-review
[ok] Wrote CHALLENGE.md
[ok] Wrote .versalist.json
[info] Challenge context is ready
$versalist run --command "python agent.py"
[ok] Agent run and provenance recorded
$versalist evaluate --run latest --command "pytest"
[ok] Verifier result recorded
[info] Compare this candidate with the baseline
$