Home Robot Agent
This challenge focuses on designing and implementing an advanced agentic system for an autonomous home robot, simulating tasks like complex multi-step operations (e.g., making espresso or loading a dishwasher). Participants will leverage GPT-5 for high-level, extended thinking and planning, enabling the robot agent to break down abstract commands into concrete, executable actions. The system will integrate with a simulated smart home environment using the MCP for seamless tool integration. Langroid will be used to ground the GPT-5 agent in the simulated physical environment, allowing it to perceive its state, execute actions, and handle dynamic changes. This challenge emphasizes robust task decomposition, adaptive planning, and reliable tool use in a simulated real-world context, showcasing how cutting-edge generative AI can drive future robotics.
What you are building
The core problem, expected build, and operating context for this challenge.
Develop an autonomous home robot agent using GPT-5, Langroid, and Model Context Protocol for complex task orchestration in a simulated environment.
How work is evaluated
The evaluation will assess the robot agent's ability to successfully complete complex, multi-step household tasks in a simulated environment, demonstrating robust planning, tool invocation via MCP, error handling, and adaptation.
Challenge datasets
Public datasets, plus the private datasets you have access to.
What you should walk away with
Develop a system that can adapt to unexpected changes in the environment and recover from execution failures.
Design an agent architecture capable of hierarchical task planning using GPT-5's extended thinking capabilities.
Implement Model Context Protocol (Model Context Protocol) for standardized tool integration with simulated robot and smart home APIs.
Utilize Langroid to create a robust agent that can interact with a simulated environment and execute physical actions.
Reference links and supporting material
Sample data for 1 tasks
Challenge setup
The evaluation will assess the robot agent's ability to successfully complete complex, multi-step household tasks in a simulated environment, demonstrating robust planning, tool invocation via MCP, error handling, and...
Challenge input
{ "command": "string", "initial_env_state": { "coffee_machine_status": "off", "water_level": "full", "beans_level": "full", "cup_available": true } }
Agent execution
No agent tool is listed for this challenge.
Evaluated output
{ "final_env_state": { "coffee_machine_status": "on", "water_level": "low", "beans_level": "low", "cup_available": false, "espresso_made": true },...
- Verify that the `espresso_made` flag in the final environment state is true.
- Ensure all necessary MCP-enabled tool calls were successfully made (e.g., power_on_coffee_machine, brew_esp...
- If an initial action fails (simulated), verify the agent attempts to re-plan or correct.
- ActionEfficiency target: 0
- Python execution harness
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- Verify that the `espresso_made` flag in the final environment state is true.
- Ensure all necessary MCP-enabled tool calls were successfully made (e.g., power_on_coffee_machine, brew_espresso).
- If an initial action fails (simulated), verify the agent attempts to re-plan or correct.
Metrics
- ActionEfficiency · Target 0
- MCPToolAdherence · Target 95
- RobustnessScore · Target 4
Workflow tasks
- Task 1
Input: { "command": "string", "initial_env_state": { "coffee_machine_status": "off", "water_level": "full", "beans_level": "full", "cup_available": true } }
Output: { "final_env_state": { "coffee_machine_status": "on", "water_level": "low", "beans_level": "low", "cup_available": false, "espresso_made": true }, "action_log": ["action1", "action2", ...], "success_message": "string" }
Execution harness: Python
Recipe freeze marker
No recipe freeze time is recorded.