Back to evaluations
Public evaluation

Strategic Recommendation Generation

The AutoGen system will be evaluated on the quality and coherence of its strategic recommendations, the effectiveness of agent collaboration, and the appropriate use of tools and data.

Evaluation type
task based
Challenge
AutoGen Multi-Agent System for Media Strategic Resource Planning
Difficulty
Intermediate
Rigor
Unspecified
Open linked challenge

Evaluation overview

How the linked challenge is judged: tasks, benchmarks, and criteria count.

Tasks
2
Benchmarks
0
Criteria
0

Task templates

Inputs and expected outputs.

Task 1

Strategic Recommendation Generation

Generate strategic recommendations for a media company facing a resource allocation challenge.

Input format

{'scenario': 'string', 'financial_data_snapshot': 'dict', 'audience_metrics_snapshot': 'dict'}

Output format

{'decision_area': 'string', 'recommended_action': 'string', 'justification': 'string', 'pros': ['string'], 'cons': ['string'], 'metrics_considered': ['string']}

Task 2

Collaborative Discussion Quality

Assess the quality and relevance of the multi-agent discussion leading to a recommendation.

Input format

{'discussion_topic': 'string'}

Output format

{'agent_dialogue_summary': 'string', 'key_points_discussed': ['string'], 'tool_usage_log': ['string']}