Skip to main content
Back to challenges
Challenge

Build a Resilient UI Automation Agent

The 'ChatGPT Atlas' headline highlights the potential and limitations of LLMs in UI interaction. This challenge aims to overcome these limitations by building a robust, autonomous agent capable of navigating complex enterprise web applications to complete multi-step tasks. The agent will leverage the advanced reasoning capabilities of GPT-5 Pro for dynamic decision-making and adaptive path planning, orchestrated via LangGraph's stateful DAGs. Critical to its success will be MCP-enabled tool integration for reliable UI interaction (e.g., clicking, typing, extracting data) and an extended thinking pattern with adaptive reasoning budgets to handle unexpected UI changes or session expirations gracefully. This system will demonstrate how cutting-edge agentic AI can automate complex human-computer interaction workflows, improving efficiency and reducing manual errors in enterprise settings. The focus is on creating a self-correcting, intelligent agent that can adapt to dynamic web environments, making it suitable for tasks like data entry, report generation, or system configuration across various web portals.

Workflow AutomationHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Build a graph-based multi-agent system using LangGraph with GPT-5 Pro for autonomous, resilient UI automation in enterprise applications, leveraging MCP for robust tool integration.

Delivery guide

How work is evaluated

Evaluation

The evaluation will focus on the agent's ability to autonomously complete a multi-step task on a simulated enterprise web application, demonstrating resilience, accurate task execution, and efficient resource management. The solution will be run in a Dockerized environment to ensure consistent execution.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Learning goals

What you should walk away with

  • Orchestrate advanced LLMs like GPT-5 Pro for dynamic decision-making and error handling in agent systems.

  • Develop extended thinking patterns with adaptive reasoning budgets for resilient task completion.

  • Integrate MCP-enabled tools for reliable and dynamic interaction with web UIs.

  • Design and implement stateful, graph-based agent workflows using LangGraph for complex UI automation.

Resources and assets

Reference links and supporting material

Dataset notes

Sample data for 1 tasks

Challenge setup

The evaluation will focus on the agent's ability to autonomously complete a multi-step task on a simulated enterprise web application, demonstrating resilience, accurate task execution, and efficient resource manageme...

Challenge input

{'customer_name': 'str', 'email': 'str', 'phone': 'str', 'company': 'str', 'notes': 'str'}

Agent execution

No agent tool is listed for this challenge.

Evaluated output

{'status': 'success' | 'failure', 'record_id': 'str' | 'null', 'final_page_url': 'str', 'error_message': 'str' | 'null'}

Evaluation checks
  • Agent successfully completes the CRM entry and returns 'status': 'success'.
  • The returned 'record_id' matches the one generated by the simulated CRM upon successful entry.
  • Agent demonstrates effective recovery from a simulated session timeout, e.g., re-login or resuming task, wi...
Configured evaluation evidence
  • Completion Time Seconds target: 90
Configured runtime
  • Docker execution harness

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

No tool records are attached.

Evaluation contract

  • Agent successfully completes the CRM entry and returns 'status': 'success'.
  • The returned 'record_id' matches the one generated by the simulated CRM upon successful entry.
  • Agent demonstrates effective recovery from a simulated session timeout, e.g., re-login or resuming task, without critical failure.
  • Agent successfully dismisses or navigates around an unexpected modal popup, continuing the task.

Metrics

  • Completion_Time_Seconds · Target 90
  • API_Call_Efficiency · Target 15
  • Tool_Usage_Accuracy · Target 95

Workflow tasks

  1. Task 1

    Input: {'customer_name': 'str', 'email': 'str', 'phone': 'str', 'company': 'str', 'notes': 'str'}

    Output: {'status': 'success' | 'failure', 'record_id': 'str' | 'null', 'final_page_url': 'str', 'error_message': 'str' | 'null'}

    Execution harness: Docker

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Build a Resilient UI Automation Agent