AutoGen MHS Workload Profiler & Accelerator Allocation Engine
Selecting optimal hardware for model workloads requires evaluating memory bandwidth, latency, and operational cost across diverse accelerators. Build an AutoGen multi-agent system powered by DeepSeek R1 to profile Model Hardware Standard (MHS) specifications and output verifiable hardware allocation plans under target latency constraints.
What you are building
The core problem, expected build, and operating context for this challenge.
Build an AutoGen multi-agent workflow that analyzes hardware specifications and selects optimal GPU, TPU, and AWS Neuron accelerators for MHS agent workloads.
How work is evaluated
Evaluates the AutoGen profiler's accuracy in hardware accelerator selection and MHS compliance generation.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
sla_compliance
Ensures selected hardware meets target SLA without exceeding cost budgets.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
allocation_accuracy
Accuracy of latency and hardware fitting calculation • target: 0.95 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Implement multi-agent hardware analysis using AutoGen and DeepSeek R1
Parse and validate Model Hardware Standard (MHS) device specifications
Calculate memory bandwidth, compute TFLOPS, and FP8/BF16 latency bounds
Generate cost and energy-optimized accelerator selection matrix
Reference links and supporting material
Hardware specs database containing memory bandwidth, FP8/BF16 TFLOPS, interconnect bandwidth, and cost for H100, TPU v5p, and Trainium2.
Challenge setup
Evaluates the AutoGen profiler's accuracy in hardware accelerator selection and MHS compliance generation.
Challenge input
JSON containing model parameters (params_b, context_len, batch_size, SLA_ms)
AutoGen
Framework for multi-agent negotiation on hardware constraints
DeepSeek R1
High-tier mathematical reasoning for roofline analysis
Evaluated output
JSON containing selected_accelerator, estimated_latency_ms, estimated_cost_usd, and MHS_profile
- Ensures selected hardware meets target SLA without exceeding cost budgets.
- Accuracy of latency and hardware fitting calculation • target: 0.95 • range: 0-1
- Benchmark: MLPerf Inference v4.0
- Allocation Accuracy target: 0.95
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- AutoGen · Required
- DeepSeek · Optional
- Anthropic · Optional
Evaluation contract
- sla_compliance · Weight 1
- allocation_accuracy · Weight 1
Metrics
- allocation_accuracy · Target 0.95
Benchmarks
- MLPerf Inference v4.0
- general
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON containing model parameters (params_b, context_len, batch_size, SLA_ms)
Output: JSON containing selected_accelerator, estimated_latency_ms, estimated_cost_usd, and MHS_profile
Execution harness: Python
Evaluation tools
- AutoGen · Agent Orchestration
- DeepSeek R1 · Reasoning LLM
Recipe freeze marker
No recipe freeze time is recorded.