Cyberthreat Orchestrator Agent
This challenge requires building an autonomous cyber threat detection and remediation system using the LangChain framework, specifically leveraging LangGraph for complex, stateful multi-agent workflows. Developers will design a team of specialized agents that work together to identify threats from simulated log data, analyze their severity, formulate a remediation plan, and orchestrate protective actions. The system must be capable of dynamic decision-making and adapting its response based on the evolving threat landscape. The focus is on robust agent collaboration patterns, sophisticated tool integration, and continuous evaluation of the agent system's effectiveness.
What you are building
The core problem, expected build, and operating context for this challenge.
Build a graph-based multi-agent system using LangGraph and Gemini 2.5 Pro for autonomous cyber threat detection and remediation orchestration.
How work is evaluated
The evaluation module will assess the LangGraph agent system's ability to accurately detect, classify, and orchestrate remediation for simulated cyber threats. Performance metrics will focus on accuracy, response time, and the quality of generated remediation plans.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
ThreatAccuracy
Agent correctly identifies and classifies threats based on provided logs.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
PlanActionability
Remediation plan contains at least 3 actionable steps for high-severity threats.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
PlanCoherenceScore
Expert-rated score for the coherence and completeness of the remediation plan (1-5). • target: 4 • range: 1-5
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
ResponseLatency_ms
Time taken for the agent to process a threat and propose a plan in milliseconds. • target: 1000 • range: 0-5000
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Master LangGraph for building stateful, cyclic agent workflows, including defining nodes, edges, and conditional routing based on threat context.
Implement robust tool invocation patterns within LangChain agents, enabling interaction with mock security APIs for threat scanning and system lockdown.
Design prompts for Gemini 2.5 Pro to perform sophisticated threat intelligence analysis, identifying zero-day potential and recommending remediation strategies.
Build custom data parsers and aggregators to normalize threat data inputs from various simulated security tools.
Integrate Evidently AI to establish continuous evaluation metrics for agent response time, accuracy of threat identification, and efficacy of proposed remediation steps.
Orchestrate the deployment of specialized Qwen 2-powered sub-agents via Together AI for high-throughput anomaly detection in network traffic logs.
Develop fault-tolerant agent execution strategies within LangGraph to handle partial failures during critical incident response scenarios.
Reference links and supporting material
Sample data for 2 tasks
Challenge setup
The evaluation module will assess the LangGraph agent system's ability to accurately detect, classify, and orchestrate remediation for simulated cyber threats. Performance metrics will focus on accuracy, response time...
Challenge input
{"log_entry": "string", "context": "string"}
Agent execution
No agent tool is listed for this challenge.
Evaluated output
{"threat_detected": "boolean", "threat_type": "string", "severity": "string"}
- Agent correctly identifies and classifies threats based on provided logs.
- Remediation plan contains at least 3 actionable steps for high-severity threats.
- Expert-rated score for the coherence and completeness of the remediation plan (1-5). • target: 4 • range: 1-5
- PlanCoherenceScore target: 4
- 2 public reference cases
- Python execution harness
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- ThreatAccuracy · Weight 1
- PlanActionability · Weight 1
- PlanCoherenceScore · Weight 1
- ResponseLatency_ms · Weight 1
Metrics
- PlanCoherenceScore · Target 4
- ResponseLatency_ms · Target 1000
Reference cases
- 2 public reference cases
Workflow tasks
- Task 1
Input: {"log_entry": "string", "context": "string"}
Output: {"threat_detected": "boolean", "threat_type": "string", "severity": "string"}
Execution harness: Python
- Task 2
Input: {"threat_type": "string", "severity": "string", "affected_systems": ["string"]}
Output: {"remediation_plan": "list of strings", "estimated_impact": "string"}
Execution harness: Python
Recipe freeze marker
No recipe freeze time is recorded.