Environment spec
Versioned task surface
Tools, observations, budgets, and termination rules ship with the challenge.
Build the full agent training loop.
Most agent evals produce a score and no way to see why. Versalist runs your agent in ready-made environments and records evidence you can check: command output and provenance, a rubric with published weights, and a local comparison with your baseline. Any model, any harness.
Start your first environment
Needs a free API key — create one and export VERSALIST_API_KEY.
Environments
—
Published this week
—
Latest environment
—
Environmentexample environment
Episodeexample episode
Rewardexample rubric
Skillexample skill
How the agent training loop works
Episode trace
Replayable run history
Recorded model, tool, and sandbox events remain available for inspection.
Reward signal
Weighted rubric score
Outcome review resolves into comparable dimensions, deltas, and baselines.
Skill promotion
Reusable agent behavior
Accepted patterns can become versioned instructions, workflows, and shared skills.
OpenAIAnthropicMetaGooglexAIQwenMCPAny harnessOpenAIAnthropicMetaGooglexAIQwenMCPAny harness
The Environment Format
Every challenge is a packaged evaluation environment.
Brief, tools, rubric, trace — the pieces teams otherwise invent from scratch, shipped as a versioned format.
01
Brief
Task, constraints, and expected behavior.
02
Tools
Allowed actions, APIs, files, and runtime bounds.
03
Rubric
Weighted quality dimensions with review evidence.
04
Trace
Replayable decisions that become improvement signal.
See why, not just whether.
Each CLI run records its command, output, duration, source revision, and challenge hash. Platform episodes can also store structured trace events.
Trust the reward before you tune on it.
Rubric weights are published, verifiers are executable, and the CLI returns a failing status when a candidate misses your comparison threshold.
Close the loop.
You can turn accepted improvements into versioned, reusable skills. The evidence remains available for later review.
The Learning Loop
EnvironmentAgentReward.
The same loop that trains the best models, applied to how you build.
Enter the Environment
Each challenge defines a learning environment: the sandbox your agent runs in, the tools it can use, and the constraints it must respect.
Run Your Agent
Deploy your agent against the environment. Every action, tool call, and decision is captured as a trajectory you can inspect and learn from.
Collect the Reward Signal
Structured evaluation rubrics score your agent across weighted dimensions. Not pass/fail. A rich signal tells you exactly what to improve next.
For Agents
Run Versalist from the same terminal your agent already uses
One command pulls the challenge into your repo. Run and evaluate a candidate, then compare it with your recorded baseline before submission.
Try it now
No install required. versalist list works without an account; start and submit need a free API key (VERSALIST_API_KEY).
1
Start
Pull the challenge brief, public eval context, and examples into the repo your agent is already using.
2
Run
Run the agent from the CLI. Versalist records its command, output, duration, source revision, and challenge hash.
3
Evaluate
Run a verifier and save its score, metrics, logs, and pass result with the agent run.
4
Compare
Compare a candidate with the baseline. The CLI returns a failing status when the candidate misses the threshold.
5
Submit
Review the local evidence. Then submit the project URL when the candidate meets your requirements.
Claude CodeCursorWindsurfGitHub CopilotContinueClineOpenAIGeminiZed
See the CLI flowThe same package also runs in MCP mode inside agent-native tools. The CLI records local verifier evidence. The in-app Run flow records platform episodes.
Terminal Workflow
$versalist start agentic-code-optimization-review
[ok] Wrote CHALLENGE.md
[ok] Wrote .versalist.json
[info] Challenge context is ready
$versalist run --command "python agent.py"
[ok] Agent run and provenance recorded
$versalist evaluate --run latest --command "pytest"
[ok] Verifier result recorded
[info] Compare this candidate with the baseline
$
Versalist is a research and training platform for improving AI agents. Each challenge is an evaluation environment with a defined task surface, tools, input data, and a weighted rubric. Teams run agents through it, inspect full trajectories, and turn outcomes into signal they can iterate on.