Agentic Robotics Control
This challenge focuses on developing an advanced agentic control system for a simulated humanoid robot. You will use AutoGen to orchestrate a team of specialized agents, each responsible for different robotic sub-systems (e.g., perception, locomotion, manipulation). The system must demonstrate robust task execution, adaptability to dynamic environments, and adhere to safety protocols using constitutional AI principles. It will integrate Gemini 3 Pro for deep reasoning and code generation for complex actions, and OpenAI GPT 5.2 for fast, reactive decision-making via MCP tool calls to a simulated robot API.
What you are building
Develop an AutoGen multi-agent system for simulated robotic control, leveraging Gemini 3 Pro for complex reasoning and OpenAI o3 for real-time reactive tasks with constitutional AI safety guards.
How work is evaluated
The evaluation will assess the robotic agent's ability to successfully complete complex multi-step tasks in a simulated environment, demonstrating adherence to safety protocols, adaptive reasoning, and effective multi-agent coordination. Safety violations will incur significant penalties.
Datasets
Public datasets, plus the private datasets you have access to.
Learning goals
Implement hybrid instant/deep reasoning using Gemini 2.5 Pro and OpenAI o3 for adaptive decision-making.
Orchestrate an AutoGen multi-agent team for modular robotic control in a simulated environment.
Utilize Semantic Kernel for high-level task planning and tool orchestration within the agent system.
Design and integrate constitutional AI principles to ensure safe and ethical robotic operation.
Develop MCP-enabled tools to simulate real-time interactions with robotic hardware APIs.
Resources
Sample data for 2 tasks
Challenge setup
The evaluation will assess the robotic agent's ability to successfully complete complex multi-step tasks in a simulated environment, demonstrating adherence to safety protocols, adaptive reasoning, and effective multi...
Challenge input
{'environment_state': {'object_positions': {'target_object': [x,y,z], 'obstacle1': [x,y,z]}, 'robot_position': [x,y,z], 'safety_zones': [[x1,y1,z1,...
Agent execution
No agent tool is listed for this challenge.
Evaluated output
{'plan': ['move_to(target_object)', 'grasp(target_object)', 'navigate_around(obstacle1)', 'move_to(platform_A)', 'release(target_object)'], 'final_...
- Robot successfully reaches the goal state.
- No safety zones are violated during execution.
- Agent successfully re-plans in response to unexpected events.
- Plan Efficiency (Steps) target: 7
- Python execution harness
Shows the first task, up to three tools, and up to three evidence items.
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- Robot successfully reaches the goal state.
- No safety zones are violated during execution.
- Agent successfully re-plans in response to unexpected events.
- All agent reasoning and actions align with predefined safety/ethical rules.
Metrics
- Plan Efficiency (steps) · Target 7
- Re-planning Latency · Target 1
- Robustness to Ambiguity · Target 0.8
Workflow tasks
- Task 1
Input: {'environment_state': {'object_positions': {'target_object': [x,y,z], 'obstacle1': [x,y,z]}, 'robot_position': [x,y,z], 'safety_zones': [[x1,y1,z1,x2,y2,z2]]}, 'goal': 'Pick up target_object and place it on platform_A.'}
Output: {'plan': ['move_to(target_object)', 'grasp(target_object)', 'navigate_around(obstacle1)', 'move_to(platform_A)', 'release(target_object)'], 'final_robot_state': {'object_held': null, 'position': [x,y,z]}, 'safety_violated': false}
Execution harness: Python
- Task 2
Input: {'initial_plan': ['move_to_A', 'move_to_B'], 'dynamic_obstacle_event': {'obstacle_id': 'moving_wall', 'position': [x,y,z], 'at_step': 1}, 'environment_state': {...}, 'goal': 'Reach destination.'}
Output: {'re_plan': ['navigate_around_moving_wall', 'resume_move_to_B'], 'safety_violated': false}
Execution harness: Python
Recipe freeze marker
No recipe freeze time is recorded.