Build an Automated Cyber Incident Triage System with AutoGen and Pydantic AI
Financial institutions face persistent advanced threat tactics from state-sponsored cyber campaigns targeting core banking infrastructure. Build an AutoGen multi-agent system using Pydantic AI to ingest threat feeds, map IOCs against MITRE ATT&CK patterns, and generate actionable remediation playbooks. Achieve automated incident triage with under two-minute latency and high classification accuracy.
What you are building
The core problem, expected build, and operating context for this challenge.
Develop an AutoGen multi-agent conversational framework paired with Pydantic AI validation to process threat intelligence feeds and map attacks against banking assets.
How work is evaluated
Evaluates the incident response agent's capacity to accurately map threat indicators to MITRE tactics and output structured playbooks.
Shared data for this challenge
Review public datasets and any private uploads tied to your build.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
valid_mitre_code_test
Checks if returned technique ID is a valid MITRE code
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
triage_precision
Precision in identifying correct threat severity and technique • target: 0.9 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Construct multi-agent AutoGen conversations for security incident orchestration
Integrate Pydantic AI for type-safe alert structured parsing and validation
Automate mapping of raw log indicators (IOCs) to MITRE ATT&CK enterprise tactics
Generate context-aware remediation playbooks for SOC tier-2 analysts
Reference links and supporting material
Dataset of 200 synthetic SOC log alerts simulating credential dumping and lateral movement targeting core banking servers.
How this agent runs
Evaluates the incident response agent's capacity to accurately map threat indicators to MITRE tactics and output structured playbooks.
Challenge input
Unstructured text payload containing firewall and system execution logs
AutoGen
Allows back-and-forth negotiation between analyst and supervisor agents.
Pydantic AI
Ensures output structured data adheres strict cybersecurity alert schemas.
Evaluated output
JSON object with threat_actor_class, mitre_technique_id, severity, and remediation_action
- Checks if returned technique ID is a valid MITRE code
- Precision in identifying correct threat severity and technique • target: 0.9 • range: 0-1
- Benchmark: SOC Cyber Automation Benchmark
- Triage Precision target: 0.9
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
View technical recipe
Configured tools
- AutoGen · Required
- Pydantic AI · Optional
- Microsoft AutoGen · Optional
- Pydantic AI · Optional
Evaluation contract
- valid_mitre_code_test · Weight 1
- triage_precision · Weight 1
Recipe state
This is a preview. The configuration can change before the evaluation recipe is locked.