Build an Automated Cyber Incident Triage System with AutoGen and Pydantic AI
Financial institutions face persistent advanced threat tactics from state-sponsored cyber campaigns targeting core banking infrastructure. Build an AutoGen multi-agent system using Pydantic AI to ingest threat feeds, map IOCs against MITRE ATT&CK patterns, and generate actionable remediation playbooks. Achieve automated incident triage with under two-minute latency and high classification accuracy.
What you are building
Develop an AutoGen multi-agent conversational framework paired with Pydantic AI validation to process threat intelligence feeds and map attacks against banking assets.
How work is evaluated
Evaluates the incident response agent's capacity to accurately map threat indicators to MITRE tactics and output structured playbooks.
Datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
valid_mitre_code_test
Checks if returned technique ID is a valid MITRE code
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
triage_precision
Precision in identifying correct threat severity and technique • target: 0.9 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Learning goals
Construct multi-agent AutoGen conversations for security incident orchestration
Integrate Pydantic AI for type-safe alert structured parsing and validation
Automate mapping of raw log indicators (IOCs) to MITRE ATT&CK enterprise tactics
Generate context-aware remediation playbooks for SOC tier-2 analysts
Resources
Dataset of 200 synthetic SOC log alerts simulating credential dumping and lateral movement targeting core banking servers.
Challenge setup
Evaluates the incident response agent's capacity to accurately map threat indicators to MITRE tactics and output structured playbooks.
Challenge input
Unstructured text payload containing firewall and system execution logs
AutoGen
Allows back-and-forth negotiation between analyst and supervisor agents.
Pydantic AI
Ensures output structured data adheres strict cybersecurity alert schemas.
Evaluated output
JSON object with threat_actor_class, mitre_technique_id, severity, and remediation_action
- Checks if returned technique ID is a valid MITRE code
- Precision in identifying correct threat severity and technique • target: 0.9 • range: 0-1
- Benchmark: SOC Cyber Automation Benchmark
- Triage Precision target: 0.9
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Shows the first task, up to three tools, and up to three evidence items.
View technical recipe
Configured tools
- AutoGen · Required
- Pydantic AI · Optional
- Microsoft AutoGen · Optional
- Pydantic AI · Optional
Evaluation contract
- valid_mitre_code_test · Weight 1
- triage_precision · Weight 1
Metrics
- triage_precision · Target 0.9
Benchmarks
- SOC Cyber Automation Benchmark
- security
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: Unstructured text payload containing firewall and system execution logs
Output: JSON object with threat_actor_class, mitre_technique_id, severity, and remediation_action
Execution harness: Python
Evaluation tools
- AutoGen · Multi-agent Conversation Framework
- Pydantic AI · Type Validation Engine
Recipe freeze marker
No recipe freeze time is recorded.