Public evaluation

ThreatAnalysisTask

Evaluation will assess the agent system's ability to accurately analyze cybersecurity alerts, propose relevant remediations, and demonstrate effective A2A communication within the LangGraph workflow.

Evaluation type

task based

Challenge

Multi-Agent Cyber Threat Triage

Difficulty

Advanced

Rigor

Unspecified

Open linked challenge

Evaluation overview

How the linked challenge is judged: tasks, benchmarks, and criteria count.

Tasks

Benchmarks

Criteria

Task templates

Inputs and expected outputs.

Task 1

ThreatAnalysisTask

Analyze a simulated cybersecurity alert and identify its type, severity, and potential impact.

Input format

{'alert_id': 'string', 'timestamp': 'string', 'source_ip': 'string', 'destination_ip': 'string', 'event_type': 'string', 'raw_log': 'string'}

Output format

{'analysis': 'string', 'severity': 'low|medium|high|critical', 'threat_category': 'string', 'confidence_score': 'float'}

Task 2

RemediationPlanningTask

Based on a threat analysis, propose concrete remediation steps and update simulated SIEM/SOAR system via MCP.

Input format

{'alert_id': 'string', 'analysis': 'string', 'severity': 'string', 'threat_category': 'string', 'confidence_score': 'float'}

Output format

{'remediation_steps': ['string'], 'siem_update_status': 'success|failure', 'siem_action_details': 'string'}