Back to evaluations
Public evaluation

PolicyChangeDetectionAndInterpretation

The evaluation will assess the AutoGen system's ability to accurately identify policy changes, interpret their implications, quantify stakeholder impact, and generate high-quality, actionable reports using A2A and MCP tools.

Evaluation type
task based
Challenge
Multi-Agent Policy Impact Analysis
Difficulty
Advanced
Rigor
Unspecified
Open linked challenge

Evaluation overview

How the linked challenge is judged: tasks, benchmarks, and criteria count.

Tasks
2
Benchmarks
0
Criteria
0

Task templates

Inputs and expected outputs.

Task 1

PolicyChangeDetectionAndInterpretation

Provide raw news articles and official government press releases. The agent team should identify a new policy change, summarize it, and interpret its core intent using GPT-5.

Input format

{"news_articles": "list_of_strings", "official_releases": "list_of_strings"}

Output format

{"policy_identified": "boolean", "policy_summary": "string", "core_intent_interpretation": "string"}

Task 2

StakeholderImpactAssessment

Based on a detected policy change and its interpretation, assess the impact on a specific stakeholder group (e.g., 'journalists', 'tech companies').

Input format

{"policy_summary": "string", "stakeholder_group": "string", "core_intent": "string"}

Output format

{"impact_level": "string (low|medium|high)", "impact_details": "string", "recommended_actions": "list_of_strings"}