Back to evaluations
Draft evaluation
Multi-Agent Hardware Cost and Latency Optimizer with CrewAI and OpenRouter — evaluation
Evaluates task delegation decision quality and total cost efficiency of routed queries.
Evaluation type
task based
Challenge
Multi-Agent Hardware Cost and Latency Optimizer with CrewAI and OpenRouter
Difficulty
Intermediate
Rigor
Not declared
The author has not specified a rigor level.
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Tasks
1
Benchmarks
0
Criteria
0
Task templates
Inputs and expected outputs.
Task 1
Routing Efficiency Task
Tests if simple prompts are routed to budget models and complex tasks to frontier models.
Input format
JSON prompt text and complexity classification ground truth
Output format
JSON selected model slug, estimated query cost