Public evaluation

Threat Classification Test

Ensure agents accurately classify threats and adhere to safety policies.

Evaluation type

task based

Challenge

Develop Secure Multi-Agent Security Operations with LangChain and Claude Opus 4.6

Difficulty

Advanced

Rigor

Unspecified

Open linked challenge

Evaluation overview

How the linked challenge is judged: tasks, benchmarks, and criteria count.

Tasks

Benchmarks

Criteria

Task templates

Inputs and expected outputs.

Task 1

Threat Classification Test

Evaluate agent capability to flag unauthorized access attempts

Input format

Text log snippet

Output format

Categorization + Risk Score

Linked challenge

Develop Secure Multi-Agent Security Operations with LangChain and Claude Opus 4.6

Create a multi-agent defense swarm to monitor and categorize potential cybersecurity incidents in critical infrastructure. Using LangChain and LangGraph, you will design a hierarchical team where specialists assess vulnerabilities and manage defensive responses. This challenge focuses on secure agent practices using Protect AI to guard against model prompt injection and systemic risks. The system utilizes All Hands AI to facilitate human-in-the-loop interactions for complex edge cases. Mem0 is integrated to maintain long-term memory of specific infrastructure configurations and past threat profiles, enabling the agents to evolve their defensive posture without relying on static databases.

Open challenge