Back to evaluations
Draft evaluation
Model Hardware Profiler: AI SDK & OpenAI o3 Accelerator Selection Engine — evaluation
Evaluates the precision of hardware recommendation profiles against known benchmark workloads.
Evaluation type
task based
Challenge
Model Hardware Profiler: AI SDK & OpenAI o3 Accelerator Selection Engine
Difficulty
Advanced
Rigor
Not declared
The author has not specified a rigor level.
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Tasks
1
Benchmarks
0
Criteria
0
Task templates
Inputs and expected outputs.
Task 1
Accelerator Recommendation Task
Evaluates the AI SDK agent's hardware selection for a 128k context coding workload under strict latency constraints.
Input format
JSON object with request rate, prompt length, output length, and latency budget.
Output format
JSON object containing selected accelerator, estimated latency, calculated cost, and architectural justification.