Skip to main content
Back to challenges
Challenge

Multi-Agent User Behavior Simulation

This challenge involves building a sophisticated multi-agent system to model and predict user interactions on a digital platform. Participants will use OpenAI o3 for advanced reasoning capabilities and AutoGen to orchestrate a team of autonomous agents that interact via an A2A protocol. Each agent will embody a distinct user persona, capable of extended thinking and decision-making within a simulated environment. The system will leverage MCP-enabled tools to interact with the simulated platform's API, generating realistic behavioral data for product testing and market analysis.

Special Purpose AgentsHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Build an A2A protocol multi-agent system using AutoGen and OpenAI o3 to simulate diverse user behaviors on a digital platform with MCP tool integration.

Delivery guide

How work is evaluated

Evaluation

The evaluation will assess the realism and diversity of simulated user behaviors, the efficiency of A2A communication, and the agents' ability to interact accurately with the simulated platform via MCP-enabled tools. Key metrics include task completion rates, behavioral variety, and logical consistency of agent actions.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Learning goals

What you should walk away with

  • Integrate MCP-enabled tools to allow agents to interact with a simulated platform API

  • Design diverse user personas with OpenAI o3 for realistic behavioral patterns

  • Implement the A2A protocol for seamless communication between simulated user agents

  • Orchestrate a multi-agent system using AutoGen for user behavior simulation

Resources and assets

Reference links and supporting material

Dataset notes

Sample data for 2 tasks

Challenge setup

The evaluation will assess the realism and diversity of simulated user behaviors, the efficiency of A2A communication, and the agents' ability to interact accurately with the simulated platform via MCP-enabled tools....

Challenge input

{'personas': [{'name': 'string', 'goal': 'string', 'preferences': 'dict'}], 'simulated_env_api_schema': 'dict'}

Agent execution

No agent tool is listed for this challenge.

Evaluated output

{'agent_logs': [{'agent_id': 'string', 'action': 'string', 'timestamp': 'string'}], 'completion_status': 'boolean', 'behavioral_diversity_score': '...

Evaluation checks
  • All agents successfully complete their assigned tasks or the shared goal is achieved.
  • Agents must successfully call the simulated environment's API tools without errors.
Configured evaluation evidence
  • BehavioralDiversityScore target: 0.8
Configured runtime
  • Python execution harness

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

No tool records are attached.

Evaluation contract

  • All agents successfully complete their assigned tasks or the shared goal is achieved.
  • Agents must successfully call the simulated environment's API tools without errors.

Metrics

  • BehavioralDiversityScore · Target 0.8
  • A2AMessageEfficiency · Target 5

Workflow tasks

  1. Task 1

    Input: {'personas': [{'name': 'string', 'goal': 'string', 'preferences': 'dict'}], 'simulated_env_api_schema': 'dict'}

    Output: {'agent_logs': [{'agent_id': 'string', 'action': 'string', 'timestamp': 'string'}], 'completion_status': 'boolean', 'behavioral_diversity_score': 'float'}

    Execution harness: Python

  2. Task 2

    Input: {'scenario': 'string', 'shared_goal': 'string', 'agent_roles': ['string']}

    Output: {'communication_log': ['string'], 'goal_achieved': 'boolean', 'a2a_message_count': 'integer'}

    Execution harness: Python

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Multi-Agent User Behavior Simulation