Continuous evals for agents
Then test every agent change against them before it ships.
Works with your stack
Versalist runs any agent harness that starts from a shell. No SDK, no framework adapter: your harness reads the challenge files and writes its output.
Connect your harnessSetup guides for
Models from OpenAI, Anthropic and Google.
The problem
An overall score can rise while individual cases get worse. A release review shows both versions case by case, so you see the drop before your users do.
How it works
Turn a real workflow into fixed test cases and a scoring rubric your team agrees on.
Run both versions on the same cases and model. Read the scores case by case and dimension by dimension.
The review owner records Ship, Hold or Needs work, and can export the review.
Features
Scores for both versions, the regressions between them, and every decision made on the review.
Scores for both versions, per case and per rubric dimension. A review holds up to 500 case rows and 100 dimensions per run.
A comparable dimension score that drops is marked as a regression.
Both runs must share the challenge, skill, rubric and case identifiers. A missing model identifier does not prove the models match.
Ship, Hold or Needs work, saved with the evidence and the full decision history.
Print a review or export it to Markdown for your release notes and sign-off.
Share a review with your organization, or create a public link for eligible reviews.
The platform
Use Continuous Evals on its own, or with the rest of Versalist. Challenges fix the cases. Skills hold the revisions. Continuous Evals compare them.
A private registry for your team's agent skills. Publish a version once and everyone pulls the same one. Roles and review policy decide who can publish, and every rollback is recorded with a reason.
revision · file digests · rollback reason
Challenge programs on your team's real workflows. Each challenge fixes the task, the cases, the rubric and the execution policy, which is what lets two runs be compared.
cases · rubric · execution policy
A free directory of tool contracts. See what a tool can do and what access it needs before your agent calls it, and verify contract exports in your browser.
Security
Reviews start private. Owners control how they are shared.
Reviews start private. Owners can share with an organization or create a public link for eligible reviews.
Model-provider keys are stored per user in Google Secret Manager.
Give each integration only the access it needs, and revoke it at any time.
Trace capture stores prompts as hashes, never as text, and never records model reasoning.
The evidence behind a review can’t be edited, and decisions are added, never overwritten.
Versalist never operates your production agent.
Get started
We scope one workflow with you, set up the cases and review the first comparison together.
Pick the workflow and agree on what good looks like.
Prepare your agent’s skill and model, and turn real cases into a test suite.
Score every candidate version against the one you run today.
Make the release call and share the results with leadership.
Need to make the case internally? Build your business case