ThreatDetectionAndClassification
The evaluation module will assess the LangGraph agent system's ability to accurately detect, classify, and orchestrate remediation for simulated cyber threats. Performance metrics will focus on accuracy, response time, and the quality of generated remediation plans.
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Task templates
Inputs and expected outputs.
ThreatDetectionAndClassification
Evaluate the agent's ability to detect a given threat pattern and classify its severity.
{"log_entry": "string", "context": "string"}
{"threat_detected": "boolean", "threat_type": "string", "severity": "string"}
RemediationPlanGeneration
Evaluate the agent's ability to generate a coherent and actionable remediation plan for a classified threat.
{"threat_type": "string", "severity": "string", "affected_systems": ["string"]}
{"remediation_plan": "list of strings", "estimated_impact": "string"}