Multi-Modal Image Editing Agent
This challenge involves building an advanced, multi-modal AI agent system capable of interpreting complex text and voice prompts to perform sophisticated image editing operations. The system will leverage a graph-based workflow to break down high-level requests into a sequence of atomic image manipulation tasks, executing them via integrated tools. Developers will focus on enabling 'extended thinking' within the agent to handle nuanced instructions, ensuring iterative refinement and adaptive problem-solving during the editing process. This project requires designing a robust LangGraph state machine that manages the image editing pipeline. The core agent, powered by Gemini 2.5 Pro, will interpret multi-modal input, generate intermediate editing plans, and dynamically select and invoke image manipulation tools. The challenge emphasizes creating a seamless user experience where text or voice commands translate into visually stunning, refined image outputs through intelligent agent orchestration and tool integration.
What you are building
The core problem, expected build, and operating context for this challenge.
Build a graph-based multi-modal image editing agent using LangGraph and Gemini 2.5 Pro with extended thinking and tool integration.
How work is evaluated
The evaluation assesses the agent's ability to interpret multi-modal prompts, generate accurate editing plans, successfully execute tools, and produce high-quality image edits.
Challenge datasets
Public datasets, plus the private datasets you have access to.
What you should walk away with
Design and implement a graph-based agent workflow using LangGraph for sequential and parallel image editing tasks.
Create a user interface for multi-modal interaction and real-time visualization of agent progress.
Integrate Gemini 2.5 Pro for multi-modal input understanding (text/voice) and complex reasoning to generate editing plans.
Implement 'extended thinking' within the agent to iteratively refine editing plans and adapt to unforeseen challenges.
Develop and integrate custom tools/APIs for various image manipulation operations (e.g., resizing, filtering, object removal, style transfer).
Reference links and supporting material
Sample data for 2 tasks
Challenge setup
The evaluation assesses the agent's ability to interpret multi-modal prompts, generate accurate editing plans, successfully execute tools, and produce high-quality image edits.
Challenge input
{'prompt': 'string', 'image_path': 'string'}
Agent execution
No agent tool is listed for this challenge.
Evaluated output
{'edited_image_path': 'string', 'final_plan': 'list of strings'}
- Checks if an output image file is successfully generated for each task.
- Verifies that the generated plan aligns logically with the input prompt.
- VisualQualityScore target: 8
- Python execution harness
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- Checks if an output image file is successfully generated for each task.
- Verifies that the generated plan aligns logically with the input prompt.
Metrics
- VisualQualityScore · Target 8
- ToolExecutionAccuracy · Target 90
Workflow tasks
- Task 1
Input: {'prompt': 'string', 'image_path': 'string'}
Output: {'edited_image_path': 'string', 'final_plan': 'list of strings'}
Execution harness: Python
- Task 2
Input: {'initial_prompt': 'string', 'initial_image_path': 'string', 'failure_scenario': 'string', 'adaptation_prompt': 'string'}
Output: {'final_edited_image_path': 'string', 'intermediate_decisions': 'list of strings'}
Execution harness: Python
Recipe freeze marker
No recipe freeze time is recorded.