Documentation / CLI

Versalist CLI

Use the command line to work with Versalist challenges and record evidence in your repository.

The versalist command loads challenge files and runs local commands. It stores run, evaluation, and comparison records under .versalist/.

Requirements

  • Node.js 18 or later
  • An npm installation
  • A Versalist API key for start and submit

Create an API key in API key settings. Store the key in an environment variable.

export VERSALIST_API_KEY=vk_live_...

Install the CLI

Install the package globally if you use the CLI regularly.

npm install -g @versalist/cli
versalist --help

Use npx if you do not want a global installation.

npx -y @versalist/cli list
npx -y @versalist/cli start agentic-code-optimization-review

Use this form if your shell cannot resolve the package executable.

npm exec --yes --package @versalist/cli versalist list

Quick start

  1. List the published challenges.
  2. Select a challenge slug.
  3. Load the challenge into your repository.
  4. Run your agent command.
  5. Run a verifier against the recorded run.
  6. Compare a candidate with a baseline.
  7. Submit the project URL.
versalist list --search "agentic code optimization"
versalist start agentic-code-optimization-review

versalist run --command "python agent.py" --label baseline
versalist evaluate --run <baseline-run-id> --command "pytest" --score 70

versalist run --command "python agent.py" --label candidate
versalist evaluate --run <candidate-run-id> --command "pytest" --score 84

versalist compare --baseline <baseline-run-id> --candidate <candidate-run-id>
versalist submit --url https://github.com/you/solution --title "Candidate run"

Commands

versalist list

List published challenges. This command does not require an API key for public data.

versalist list
versalist list --search reranker
versalist list --category evaluation --difficulty advanced
versalist list --page 2 --limit 12
versalist list --json

versalist start <slug-or-id>

Load one challenge into the current working directory. This command does not run an agent.

versalist start agentic-code-optimization-review

The command writes these files:

  • CHALLENGE.md contains the challenge brief.
  • .versalist.json contains the challenge identifier and local metadata.
  • eval/examples.json contains public examples when they are available.

The command also adds .versalist/ to .gitignore.

versalist run

Run a shell command in the current working directory. The command requires an existing .versalist.json file.

versalist run --command "python agent.py" --label baseline
versalist run --command "python agent.py" --timeout 1200
versalist run --command "python agent.py" --json

The run record contains:

  • The command and exit status
  • The start time, completion time, and duration
  • The current Git revision and dirty-worktree state
  • Hashes for the challenge files
  • Paths to stdout.log and stderr.log

The default timeout is 900 seconds. A failed command returns a nonzero exit status.

versalist evaluate

Run a verifier against a recorded run. The verifier is a local shell command.

versalist evaluate --run latest --command "pytest"
versalist evaluate --run <run-id> --command "pytest" --score 82
versalist evaluate --run <run-id> --command "pytest" --metrics results.json

A successful verifier receives a default score of 100. A failed verifier receives a default score of 0.

Use --score to provide a score from 0 through 100. Use --metrics to attach a JSON object.

versalist compare

Compare the latest evaluation for a baseline run and a candidate run.

versalist compare --baseline <baseline-run-id> --candidate <candidate-run-id>
versalist compare --baseline <baseline-run-id> --candidate <candidate-run-id> --min-delta 5
versalist compare --baseline <baseline-run-id> --candidate <candidate-run-id> --json

The command passes when the candidate evaluation passes and meets the minimum score change. It returns exit status 1 when the comparison fails.

The comparison is a local record. It does not change the baseline or prevent a later submit command.

versalist submit

Create a project submission for a challenge.

versalist submit   --url https://github.com/you/solution   --title "Candidate run"

versalist submit   --url https://demo.example.com   --github-url https://github.com/you/solution   --title "Hosted demonstration"

Use these optional flags:

  • --challenge selects a challenge instead of using .versalist.json.
  • --description adds implementation information.
  • --model records the model name.
  • --toolchain records the framework or toolchain name.

The command sends project metadata and URLs. It does not upload the contents of .versalist/.

The command does not require a passed comparison.

versalist mcp

Start the Model Context Protocol (MCP) server over standard input and output. Use this mode with an MCP-compatible editor or agent host.

versalist mcp
npx -y @versalist/cli mcp
npm exec --yes --package @versalist/cli versalist mcp

API key scopes

  • read:challenges permits access to challenge details and files.
  • submit:solutions permits project submission.
  • read:submissions permits the MCP server to list your submissions.

Local records

The CLI stores run evidence in .versalist/runs/.

The CLI stores comparison records in .versalist/comparisons/.

These records are separate from platform-managed Episodes. The CLI does not upload them automatically.

Security

The CLI uses your current environment variables when it runs a command. Run only commands that you trust.

Command output can contain credentials or private data. Review all evidence files before you share them.

Do not commit .versalist/ to source control.

Automation

Use --json when another program must read a CLI result.

The run, evaluate, and compare commands return nonzero exit statuses when their checks fail.

Troubleshooting

The CLI reports an unknown command

Check the installed package version and update the package.

npm view @versalist/cli version
npm install -g @versalist/cli@latest
versalist --help

The CLI cannot find .versalist.json

Run versalist start <slug-or-id> in the repository first.

The API rejects the API key

Confirm that the environment variable exists in the current shell.

test -n "$VERSALIST_API_KEY" && echo "API key is set"

Create a new key if the current key is invalid or revoked.

Was this page helpful?