Back to evaluations
Public evaluation
hardware_allocation_task
Evaluates the AutoGen profiler's accuracy in hardware accelerator selection and MHS compliance generation.
Evaluation type
task based
Challenge
AutoGen MHS Workload Profiler & Accelerator Allocation Engine
Difficulty
Intermediate
Rigor
Unspecified
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Tasks
1
Benchmarks
0
Criteria
0
Task templates
Inputs and expected outputs.
Task 1
hardware_allocation_task
Evaluates hardware recommendation for given model batch size, context length, and target SLA.
Input format
JSON containing model parameters (params_b, context_len, batch_size, SLA_ms)
Output format
JSON containing selected_accelerator, estimated_latency_ms, estimated_cost_usd, and MHS_profile