PolicyViolationDetection
The evaluation will assess the system's ability to accurately identify policy violations, its adaptability in reasoning, and the efficiency of its agent communication within the LangGraph framework. Emphasis will be placed on the correct implementation of MCP for tool calls and dynamic budget allocation.
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Task templates
Inputs and expected outputs.
PolicyViolationDetection
Evaluates the system's ability to identify specific policy violations from synthetic social media posts.
{ 'social_media_posts': [{'id': 'str', 'text': 'str', 'author_id': 'str'}], 'company_policies': [{'id': 'str', 'text': 'str'}] }
{ 'alerts': [{'post_id': 'str', 'violation_type': 'str', 'reasoning': 'str', 'severity': 'low|medium|high', 'adaptive_budget_used': 'float'}] }