Documentation / Tool catalog

Tool catalog

Find tools and compare them against task, infrastructure, and cost requirements.

The public tool catalog contains model providers, frameworks, agents, evaluation tools, and infrastructure products. Catalog entries do not grant provider access.

Find a tool

  1. Define the task before you select a tool.
  2. Open the tool catalog.
  3. Filter by category or capability.
  4. Open a tool detail page.
  5. Review related challenges, prompts, and guides.

Compare tools

Use the same requirements for each candidate.

  • Task support: Confirm that the tool performs the required action.
  • Evidence: Identify the logs, outputs, or metrics that the tool produces.
  • Hosting: Record where the tool processes and stores data.
  • Access: Check authentication, provider keys, and organization policy.
  • Cost: Compare cost at the expected request and token volume.
  • Latency: Measure response time under the expected workload.

Test candidates against one challenge. Do not compare tools with different tasks or evaluators.

Model providers

The catalog includes providers such as OpenAI, Anthropic, Google, Azure, and Amazon Bedrock. It also includes specialized providers.

A catalog entry describes a provider. It does not confirm that Versalist can execute a model through that provider.

Provider credentials

Add a credential in Integrations only when a Versalist workflow must call that provider.

  1. Select the provider that the workflow uses.
  2. Store the provider credential.
  3. Enable the provider.
  4. Run a small test request.
  5. Disable or rotate the credential when the environment changes.

Provider credentials are different from Versalist API keys. Read API keys before you configure CLI authentication.

Use a tool in an evaluation

  1. Record the tool and version.
  2. Record the model and model version when applicable.
  3. Keep the challenge and evaluator unchanged.
  4. Run the baseline.
  5. Replace one tool or configuration value.
  6. Run the candidate.
  7. Compare results and trace behavior.
  • Challenges explains task environments and evaluation rules.
  • Integrations explains provider credentials and current capability limits.
  • Prompt library contains prompt examples linked to tools and challenges.
Was this page helpful?