Case-by-case scores
Compare available case and dimension scores. Reviews include up to 500 case rows and 100 dimensions per run; some scores may be unavailable.
Catch regressions before your agent ships.
Run the current and proposed skill revisions on the same test cases, rubric and model. Compare available scores and record a release decision.
Built for
AI and platform teams reviewing agent changes
What you get
Side-by-side release reviews and decisions on record
Get started
Guided pilot
We scope one workflow with you and agree the fee up front.
Why it matters
An average score can hide a regression.
An overall score can improve even when some test cases get worse. Compare the available scores before deciding whether to release.
Without Assurance
- One overall score that hides the cases that got worse
- Runs on different data, rubrics or models compared as if they matched
- Release calls made in chat, with nothing to point back to
With Assurance
- Available case scores shown side by side
- Matching challenge, skill, rubric and case identifiers required
- Ship, Hold or Needs work, saved with the evidence behind it
How it works
From change to decision in four steps.
Scope
Pick one workflow, its test cases and the person who owns the release call. We agree scope and fee before work begins.
Set up
Your engineer sets up the agent’s skill and model on Versalist and turns your cases and scoring rubric into a test suite. The current version runs as the baseline.
Compare
Run the candidate on the same cases, rubric and model, then open a release review against the baseline.
Decide
The review owner reads the results and records Ship, Hold or Needs work. Every decision stays on record.
Features
What a release review includes.
Like-for-like comparisons
Versalist checks matching challenge, skill, rubric and case identifiers. Model identifiers are compared when present; missing identifiers do not prove a model match.
Decisions on record
Ship, Hold or Needs work, timestamped and saved next to an evidence snapshot that can’t be edited.
Controlled sharing
Keep a review private, share it with your organization, or create a public link for eligible reviews. Revoke sharing at any time.
Benefits
Review a change before release.
Find score regressions
See decreases in available dimension scores before release.
Keep the decision with the results
Record the decision alongside the evidence used to make it.
Share the review
Give stakeholders access to the evidence and decision history.
Integrations
Supported model providers
- Model providers
- OpenAI
- Anthropic
- Your keys
- Your own provider API keys, stored in Google Secret Manager
- Share results
- Your organization
- Public link for eligible reviews
- Markdown
- Print and PDF
Resources
Examples and documentation
A sample comparison
Walk through a release review of an FDA safety-reporting agent, case by case.
See a sample comparisonDeveloper docs
How your engineers connect an agent to Versalist and set up the runs a review compares.
Read the developer docsApproval brief
A one-page brief your sponsor can take to budget approval.
Build your business case
Security
Review access and data handling
Reviews start private, and you decide who sees them, from the first run to the release decision.
Private by default
A review starts private. The owner can share it with an organization or create a public link for eligible reviews.
Locked evidence
The evidence snapshot can’t be edited, and decisions are added, never overwritten.
Secure credential storage
Model-provider keys are stored per user in Google Secret Manager. API keys are scoped and revocable.
No prompt text in traces
Trace capture stores prompts as hashes, never as text, and never records model reasoning.
Pricing
Start with a pilot.
Paid pilot, fee agreed up front.
Scope, duration, seats and run capacity are agreed with you before work begins. Invoiced, with no automatic charges.
FAQ
Frequently asked questions
How long does a pilot take?
Plan on one to two days of setup for your engineer, then under an hour for each candidate you compare. We agree the full timeline with you during scoping.
What do we need to provide?
A pilot owner for about an hour a week, one engineer to set up the agent’s skill and write the test cases, and one security review session.
What kinds of change can we compare?
Compare skill revisions on the same challenge, cases and rubric. Use the same model. Missing model identifiers do not prove the models match.
Who sees the results?
Reviews start private. The owner can share them with the organization or create a public link for eligible reviews.
Do you run our agent in production?
No. Versalist works before release, alongside the observability you already run in production.