Back to evaluations
Public evaluation
Hardware Aware Routing Task
Evaluates cost savings and SLA adherence for hardware request routing.
Evaluation type
task based
Challenge
OpenAI Agents SDK Hardware Serving Profiler & AWS Neuron Router
Difficulty
Intermediate
Rigor
Unspecified
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Tasks
1
Benchmarks
0
Criteria
0
Task templates
Inputs and expected outputs.
Task 1
Hardware Aware Routing Task
Evaluates routing agent decision for 1,000 mixed-length prompts between Trainium2 and H100 clusters under a 100ms P99 latency limit.
Input format
JSON request batch with prompt token counts, output target lengths, and SLA latency bounds.
Output format
JSON routing breakdown (Neuron % vs CUDA %), average P99 latency, and overall cost savings %.