Agentic Robotics Control
This challenge focuses on developing an advanced agentic control system for a simulated humanoid robot. You will use AutoGen to orchestrate a team of specialized agents, each responsible for different robotic sub-systems (e.g., perception, locomotion, manipulation). The system must demonstrate robust task execution, adaptability to dynamic environments, and adhere to safety protocols using constitutional AI principles. It will integrate Gemini 3 Pro for deep reasoning and code generation for complex actions, and OpenAI GPT 5.2 for fast, reactive decision-making via MCP tool calls to a simulated robot API.
What you are building
The core problem, expected build, and operating context for this challenge.
Develop an AutoGen multi-agent system for simulated robotic control, leveraging Gemini 3 Pro for complex reasoning and OpenAI o3 for real-time reactive tasks with constitutional AI safety guards.
How work is evaluated
The evaluation will assess the robotic agent's ability to successfully complete complex multi-step tasks in a simulated environment, demonstrating adherence to safety protocols, adaptive reasoning, and effective multi-agent coordination. Safety violations will incur significant penalties.
Challenge datasets
Public datasets, plus the private datasets you have access to.
What you should walk away with
Implement hybrid instant/deep reasoning using Gemini 2.5 Pro and OpenAI o3 for adaptive decision-making.
Orchestrate an AutoGen multi-agent team for modular robotic control in a simulated environment.
Utilize Semantic Kernel for high-level task planning and tool orchestration within the agent system.
Design and integrate constitutional AI principles to ensure safe and ethical robotic operation.
Develop MCP-enabled tools to simulate real-time interactions with robotic hardware APIs.
Reference links and supporting material
Sample data for 2 tasks
Challenge setup
The evaluation will assess the robotic agent's ability to successfully complete complex multi-step tasks in a simulated environment, demonstrating adherence to safety protocols, adaptive reasoning, and effective multi...
Challenge input
{'environment_state': {'object_positions': {'target_object': [x,y,z], 'obstacle1': [x,y,z]}, 'robot_position': [x,y,z], 'safety_zones': [[x1,y1,z1,...
Agent execution
No agent tool is listed for this challenge.
Evaluated output
{'plan': ['move_to(target_object)', 'grasp(target_object)', 'navigate_around(obstacle1)', 'move_to(platform_A)', 'release(target_object)'], 'final_...
- Robot successfully reaches the goal state.
- No safety zones are violated during execution.
- Agent successfully re-plans in response to unexpected events.
- Plan Efficiency (Steps) target: 7
- Python execution harness
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- Robot successfully reaches the goal state.
- No safety zones are violated during execution.
- Agent successfully re-plans in response to unexpected events.
- All agent reasoning and actions align with predefined safety/ethical rules.
Metrics
- Plan Efficiency (steps) · Target 7
- Re-planning Latency · Target 1
- Robustness to Ambiguity · Target 0.8
Workflow tasks
- Task 1
Input: {'environment_state': {'object_positions': {'target_object': [x,y,z], 'obstacle1': [x,y,z]}, 'robot_position': [x,y,z], 'safety_zones': [[x1,y1,z1,x2,y2,z2]]}, 'goal': 'Pick up target_object and place it on platform_A.'}
Output: {'plan': ['move_to(target_object)', 'grasp(target_object)', 'navigate_around(obstacle1)', 'move_to(platform_A)', 'release(target_object)'], 'final_robot_state': {'object_held': null, 'position': [x,y,z]}, 'safety_violated': false}
Execution harness: Python
- Task 2
Input: {'initial_plan': ['move_to_A', 'move_to_B'], 'dynamic_obstacle_event': {'obstacle_id': 'moving_wall', 'position': [x,y,z], 'at_step': 1}, 'environment_state': {...}, 'goal': 'Reach destination.'}
Output: {'re_plan': ['navigate_around_moving_wall', 'resume_move_to_B'], 'safety_violated': false}
Execution harness: Python
Recipe freeze marker
No recipe freeze time is recorded.