Back to evaluations
Public evaluation
Accelerator Recommendation Task
Evaluates the precision of hardware recommendation profiles against known benchmark workloads.
Evaluation type
task based
Challenge
Model Hardware Profiler: AI SDK & OpenAI o3 Accelerator Selection Engine
Difficulty
Advanced
Rigor
Unspecified
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Tasks
1
Benchmarks
0
Criteria
0
Task templates
Inputs and expected outputs.
Task 1
Accelerator Recommendation Task
Evaluates the AI SDK agent's hardware selection for a 128k context coding workload under strict latency constraints.
Input format
JSON object with request rate, prompt length, output length, and latency budget.
Output format
JSON object containing selected accelerator, estimated latency, calculated cost, and architectural justification.