Multi-Agent Hardware Cost and Latency Optimizer with CrewAI and OpenRouter
Orchestrate multi-model workflows to dynamic accelerator hardware tiers for cost optimization. Build a collaborative multi-agent system in CrewAI using OpenRouter API routing to evaluate query complexity and redirect traffic between high-tier and low-tier hardware targets.
What you are building
The core problem, expected build, and operating context for this challenge.
Construct a CrewAI agent team using OpenRouter to evaluate prompt difficulty and route requests to optimal model hardware endpoints, minimizing token execution costs.
How work is evaluated
Evaluates task delegation decision quality and total cost efficiency of routed queries.
Shared data for this challenge
Review public datasets and any private uploads tied to your build.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
Cost Model Selection Test
Verify cheap model selection for simple prompt.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Routing Accuracy Rate
Percentage of requests routed to appropriate tier. • target: 0.95 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Orchestrate CrewAI role-playing agents for intent classification and hardware target selection.
Utilize OpenRouter API to route requests based on dynamic latency and cost limits.
Achieve over 30% cost reduction without sacrificing task accuracy scores.
Reference links and supporting material
Dataset of 100 mixed complexity prompts ranging from simple math to complex code refactoring.
How this agent runs
Evaluates task delegation decision quality and total cost efficiency of routed queries.
Challenge input
JSON prompt text and complexity classification ground truth
CrewAI
Framework for role-playing multi-agent systems.
OpenRouter
Unified API gateway for dynamic model access.
Evaluated output
JSON selected model slug, estimated query cost
- Verify cheap model selection for simple prompt.
- Percentage of requests routed to appropriate tier. • target: 0.95 • range: 0-1
- Benchmark: LLM Routing Cost Efficiency Index
- Routing Accuracy Rate target: 95%
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
View technical recipe
Configured tools
- CrewAI · Required
- crewAI · Optional
- OpenRouter · Optional
- OpenRouter · Optional
Evaluation contract
- Cost Model Selection Test · Weight 1
- Routing Accuracy Rate · Weight 1
Recipe state
This is a preview. The configuration can change before the evaluation recipe is locked.