Versalist CLI | Versalist Documentation

The versalist command loads challenge files and runs local commands. It stores run, evaluation, and comparison records under .versalist/.

list works with no key. start, submit, MCP tools, and local challenge runs need VERSALIST_API_KEY.

Requirements

Node.js 18 or later
An npm installation
A Versalist API key for start, submit, and MCP

Create an API key in API key settings. Store the key in an environment variable.

export VERSALIST_API_KEY=vk_live_...
export VERSALIST_BASE_URL=https://versalist.com

VERSALIST_BASE_URL is optional and defaults to https://versalist.com. Set it when you target staging or a self-hosted deployment. Authenticated HTTP calls time out after 20 seconds.

Install the CLI

Install the package globally if you use the CLI regularly.

npm install -g @versalist/cli
versalist --help

Use npx if you do not want a global installation.

npx -y @versalist/cli list
npx -y @versalist/cli start agentic-code-optimization-review

Use this form if your shell cannot resolve the package executable.

npm exec --yes --package @versalist/cli versalist list

Quick start

List the published challenges.
Select a challenge slug.
Load the challenge into your repository.
Run your agent command.
Run a verifier against the recorded run.
Compare a candidate with a baseline.
Submit the project URL.

versalist list --search "agentic code optimization"
versalist start agentic-code-optimization-review

versalist run --command "python agent.py" --label baseline
versalist evaluate --run <baseline-run-id> --command "pytest"

versalist run --command "python agent.py" --label candidate
versalist evaluate --run <candidate-run-id> --command "pytest"

versalist compare --baseline <baseline-run-id> --candidate <candidate-run-id>
versalist submit --url https://github.com/you/solution --title "Candidate run"

The quickstart walks through this sequence with an annotated failed check and comparison.

Commands

versalist list

List published challenges. This command does not require an API key for public data.

versalist list
versalist list --search reranker
versalist list --category evaluation --difficulty advanced
versalist list --page 2 --limit 12
versalist list --json

versalist start <slug-or-id>

Load one challenge into the current working directory. This command does not run an agent.

versalist start agentic-code-optimization-review

The command writes these files:

CHALLENGE.md contains the challenge brief.
.versalist.json contains the challenge identifier and local metadata.
eval/examples.json contains public examples when they are available.

The command also adds .versalist/ to .gitignore.

versalist run

Run a shell command in the current working directory. The command requires an existing .versalist.json file.

versalist run --command "python agent.py" --label baseline
versalist run --command "python agent.py" --timeout 1200
versalist run --command "python agent.py" --json

The run record contains:

The command and exit status
The start time, completion time, and duration
The current Git revision and dirty-worktree state
Hashes for the challenge files
Paths to stdout.log and stderr.log

The default timeout is 900 seconds. A failed command returns a nonzero exit status.

versalist evaluate

Run a verifier against a recorded run. The verifier is a local shell command.

versalist evaluate --run latest --command "pytest"
versalist evaluate --run <run-id> --command "pytest" --score 82
versalist evaluate --run <run-id> --command "pytest" --metrics results.json

passed comes from the verifier: exit status 0 without a timeout is a pass. Without --score, a pass records a score of 100 and a failure records 0. The CLI does not parse the verifier output; it stores it in stdout.log and stderr.log next to the record.

Use --score to record a score from 0 through 100. The value is self-declared: the CLI stores what you pass and does not check it against the verifier. Use --metrics to attach a JSON object. Use --timeout for the verifier process; the default is 900 seconds (max 86400).

versalist compare

Compare the latest evaluation for a baseline run and a candidate run.

versalist compare --baseline <baseline-run-id> --candidate <candidate-run-id>
versalist compare --baseline <baseline-run-id> --candidate <candidate-run-id> --min-delta 5
versalist compare --baseline <baseline-run-id> --candidate <candidate-run-id> --json

The command passes when the candidate evaluation passes and meets the minimum score change. It returns exit status 1 when the comparison fails. The record's decision is improved, unchanged, below_threshold, or regressed; a candidate that fails its verifier or scores lower is regressed.

The comparison is a local record. It does not change the baseline or prevent a later submit command.

versalist submit

Create a project submission for a challenge.

versalist submit \
  --url https://github.com/you/solution \
  --title "Candidate run"

versalist submit \
  --url https://demo.example.com \
  --github-url https://github.com/you/solution \
  --title "Hosted demonstration"

Use these optional flags:

--challenge selects a challenge instead of using .versalist.json.
--url / --project-url — same flag. Required. Must be http(s).
--description (alias --desc) adds implementation information.
--model records the model name.
--toolchain records the framework or toolchain name.

The command sends project metadata and URLs. It does not upload the contents of .versalist/.

The command does not require a passed comparison.

versalist mcp

Start the Model Context Protocol (MCP) server over standard input and output. Use this mode with an MCP-compatible editor or agent host.

versalist mcp
npx -y @versalist/cli mcp
npm exec --yes --package @versalist/cli versalist mcp

The server exposes eight tools. See the MCP tool reference. Host-specific config is on the coding agent guides.

API key scopes

read:challenges — challenge details and files.
submit:solutions — project submission and local model runs.
read:submissions — MCP get_my_submissions.
read:runs / execute:runs — accepted by the run protocol behind versalist challenge run as an alternative to submit:solutions.
read:governance / write:governance — comparisons and release decisions over HTTP, not this CLI. See Test a change before you release it.
read:skills / write:skills — Skill Exchange via vskill, not this CLI.

Scope descriptions and the key creation steps are on API keys.

Local records

The CLI stores run evidence in .versalist/runs/.

The CLI stores comparison records in .versalist/comparisons/.

These records are separate from hosted runs. The CLI does not upload them automatically, and the platform does not verify them. Where your agent runs compares the two.

Security

The CLI uses your current environment variables when it runs a command. Run only commands that you trust.

Command output can contain credentials or private data. Review all evidence files before you share them.

Do not commit .versalist/ to source control.

Automation

Use --json when another program must read a CLI result.

The run, evaluate, and compare commands return nonzero exit statuses when their checks fail.

Troubleshooting

The CLI reports an unknown command

Check the installed package version and update the package.

npm view @versalist/cli version
npm install -g @versalist/cli@latest
versalist --help

The CLI cannot find .versalist.json

Run versalist start <slug-or-id> in the repository first.

The API rejects the API key

Confirm that the environment variable exists in the current shell.

test -n "$VERSALIST_API_KEY" && echo "API key is set"

401 — key missing, invalid, or revoked. Create a new key.
403 — key is valid but missing a scope. Recreate it with the scopes above.
404 on start — wrong slug. Run versalist list.
409 on submit — this account already submitted to the challenge.

Authenticated commands time out after 20 seconds. If a request hangs past that, check VERSALIST_BASE_URL and your network, not the local run timeout.