Back to evaluations
Public evaluation
Routing Efficiency Task
Evaluates task delegation decision quality and total cost efficiency of routed queries.
Evaluation type
task based
Challenge
Multi-Agent Hardware Cost and Latency Optimizer with CrewAI and OpenRouter
Difficulty
Intermediate
Rigor
Unspecified
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Tasks
1
Benchmarks
0
Criteria
0
Task templates
Inputs and expected outputs.
Task 1
Routing Efficiency Task
Tests if simple prompts are routed to budget models and complex tasks to frontier models.
Input format
JSON prompt text and complexity classification ground truth
Output format
JSON selected model slug, estimated query cost