Gemini 2.5 Pro & AutoGen for AI Hardware Optimization Agents
This challenge focuses on building an advanced multi-agent system designed to optimize AI model deployments for specific hardware architectures, such as AMD GPUs. The goal is to create an autonomous 'AI Operations' team that can analyze model requirements, profile hardware capabilities, propose optimal deployment configurations, and even generate code snippets for performance enhancements. Developers will use AutoGen to orchestrate a conversational multi-agent workflow, integrating Gemini 2.5 Pro for its powerful code generation, optimization, and deep technical reasoning. The system will feature MCP-enabled tool integration, allowing agents to interact with simulated enterprise hardware profiling APIs and cloud resource management systems, ensuring practical applicability for AI infrastructure management.
What you are building
The core problem, expected build, and operating context for this challenge.
Build an AutoGen multi-agent system with Gemini 2.5 Pro for optimizing AI model deployments on specific hardware, featuring MCP-enabled tool integration and deep reasoning.
How work is evaluated
The system will be evaluated on its ability to generate optimized deployment configurations and code, demonstrate effective agent collaboration, and utilize MCP-enabled tools to achieve specific performance and cost targets for AI model inference on target hardware.
Challenge datasets
Public datasets, plus the private datasets you have access to.
What you should walk away with
Implement MCP-enabled tool integration to connect agents with simulated hardware profiling and cloud APIs.
Integrate Gemini 2.5 Pro for advanced code generation, performance analysis, and technical solutioning.
Design agents for deep reasoning, enabling complex problem-solving in hardware-software co-optimization.
Orchestrate a conversational multi-agent workflow using AutoGen for AI deployment optimization.
Reference links and supporting material
Sample data for 2 tasks
Challenge setup
The system will be evaluated on its ability to generate optimized deployment configurations and code, demonstrate effective agent collaboration, and utilize MCP-enabled tools to achieve specific performance and cost t...
Challenge input
{'ai_model_spec': {'name': 'string', 'parameters_count': 'int', 'inference_latency_target_ms': 'int'}, 'available_hardware_profiles': [{'id': 'stri...
Agent execution
No agent tool is listed for this challenge.
Evaluated output
{'recommended_hardware_id': 'string', 'deployment_strategy': 'string', 'estimated_latency_ms': 'int', 'estimated_cost_per_hour_usd': 'float', 'opti...
- Recommended hardware and strategy must be technically plausible and meet latency targets (within a margin).
- Generated code snippet must be syntactically correct for the specified language and contain relevant keywor...
- Verify that agents successfully called MCP-enabled tools for hardware profiling and cost estimation (requir...
- OptimizationScore target: 0.85
- Python execution harness
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- Recommended hardware and strategy must be technically plausible and meet latency targets (within a margin).
- Generated code snippet must be syntactically correct for the specified language and contain relevant keywords for optimization (e.g., 'hip' for AMD GPUs).
- Verify that agents successfully called MCP-enabled tools for hardware profiling and cost estimation (requires logging analysis).
Metrics
- OptimizationScore · Target 0.85
- CodeQualityScore · Target 0.75
- DeepReasoningDepth · Target 0.8
Workflow tasks
- Task 1
Input: {'ai_model_spec': {'name': 'string', 'parameters_count': 'int', 'inference_latency_target_ms': 'int'}, 'available_hardware_profiles': [{'id': 'string', 'type': 'string', 'compute_gflops': 'int', 'memory_gb': 'int', 'cost_per_hour_usd': 'float'}]}
Output: {'recommended_hardware_id': 'string', 'deployment_strategy': 'string', 'estimated_latency_ms': 'int', 'estimated_cost_per_hour_usd': 'float', 'optimization_notes': ['string']}
Execution harness: Python
- Task 2
Input: {'bottleneck_description': 'string', 'programming_language': 'string', 'target_hardware_spec': 'dict'}
Output: {'generated_code_snippet': 'string', 'explanation': 'string', 'performance_gain_estimate_percent': 'float'}
Execution harness: Python
Recipe freeze marker
No recipe freeze time is recorded.