Back to evaluations
Public evaluation

MultiModalEditExecution

The evaluation assesses the agent's ability to interpret multi-modal prompts, generate accurate editing plans, successfully execute tools, and produce high-quality image edits.

Evaluation type
task based
Challenge
Multi-Modal Image Editing Agent
Difficulty
Advanced
Rigor
Unspecified
Open linked challenge

Evaluation overview

How the linked challenge is judged: tasks, benchmarks, and criteria count.

Tasks
2
Benchmarks
0
Criteria
0

Task templates

Inputs and expected outputs.

Task 1

MultiModalEditExecution

Evaluates the agent's end-to-end capability to perform a complex image edit from a text prompt.

Input format

{'prompt': 'string', 'image_path': 'string'}

Output format

{'edited_image_path': 'string', 'final_plan': 'list of strings'}

Task 2

AdaptiveReasoningTest

Assesses the agent's ability to correct a failed tool execution or adapt to a new constraint.

Input format

{'initial_prompt': 'string', 'initial_image_path': 'string', 'failure_scenario': 'string', 'adaptation_prompt': 'string'}

Output format

{'final_edited_image_path': 'string', 'intermediate_decisions': 'list of strings'}