Multi-Agent Pen-Testing
This challenge tasks you with building a multi-agent penetration testing system. You will orchestrate a team of specialized agents using CrewAI, each with distinct roles (e.g., 'Reconnaissance Agent', 'Vulnerability Scanner Agent', 'Report Generator Agent'). These agents must communicate securely via an A2A protocol and leverage Claude Opus 4.1 for advanced reasoning to identify and exploit vulnerabilities in a simulated web application. The system needs to integrate MCP-enabled tools for interacting with the target environment and dynamically adapt its thinking budget based on the complexity of findings.
What you are building
The core problem, expected build, and operating context for this challenge.
Develop an A2A multi-agent system using CrewAI with Claude Opus 4.5 to autonomously conduct penetration tests and report vulnerabilities in a simulated environment.
How work is evaluated
The evaluation will assess the multi-agent system's ability to autonomously discover vulnerabilities in a simulated environment, communicate findings between agents, and generate a professional penetration test report.
Challenge datasets
Public datasets, plus the private datasets you have access to.
What you should walk away with
Conduct autonomous vulnerability discovery and generate comprehensive reports.
Orchestrate a team of role-based agents for security tasks using CrewAI.
Integrate MCP-enabled tools for simulating security scanner interactions.
Implement secure agent-to-agent (A2A) communication protocols for collaboration.
Reference links and supporting material
Sample data for 3 tasks
Challenge setup
The evaluation will assess the multi-agent system's ability to autonomously discover vulnerabilities in a simulated environment, communicate findings between agents, and generate a professional penetration test report.
Challenge input
{"target_url": "URL of the simulated vulnerable web application"}
Agent execution
No agent tool is listed for this challenge.
Evaluated output
{"vulnerabilities_found": [{"type": "string", "location": "string", "severity": "string", "exploit_path_summary": "string"}, ...]}
- Agent team identifies at least two distinct vulnerabilities in the target application.
- Communication logs clearly show agents sharing information relevant to their tasks.
- A penetration test report is generated with clear sections (summary, findings, recommendations).
- Vulnerability Coverage target: 75
- Python execution harness
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- Agent team identifies at least two distinct vulnerabilities in the target application.
- Communication logs clearly show agents sharing information relevant to their tasks.
- A penetration test report is generated with clear sections (summary, findings, recommendations).
Metrics
- Vulnerability Coverage · Target 75
- Report Quality Score · Target 85
Workflow tasks
- Task 1
Input: {"target_url": "URL of the simulated vulnerable web application"}
Output: {"vulnerabilities_found": [{"type": "string", "location": "string", "severity": "string", "exploit_path_summary": "string"}, ...]}
Execution harness: Python
- Task 2
Input: {"agent_interaction_log_request": "Retrieve logs from the multi-agent run"}
Output: {"communication_log": [{"sender": "string", "receiver": "string", "message": "string", "timestamp": "datetime"}, ...]}
Execution harness: Python
- Task 3
Input: {"vulnerability_findings": [...], "agent_actions": "string"}
Output: {"report_title": "string", "summary": "string", "findings_section": [{"vulnerability_type": "string", "details": "string", "recommendations": "string"}, ...], "conclusion": "string"}
Execution harness: Python
Recipe freeze marker
No recipe freeze time is recorded.