IncidentDetectionAndResolution
The evaluation will assess the system's ability to autonomously detect, diagnose, and propose/execute remediation for simulated cloud incidents. Key metrics will include incident resolution time, accuracy of root cause analysis, and effective use of A2A communication and MCP tools.
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Task templates
Inputs and expected outputs.
IncidentDetectionAndResolution
Simulate a cloud incident and evaluate if the agents detect it, perform root cause analysis, and suggest/execute remediation.
JSON object with 'incident_type', 'error_logs', 'metrics_data'
JSON object with 'incident_id', 'status', 'root_cause_analysis', 'remediation_plan', 'executed_actions'
A2ACommunicationFlow
Verify that agents correctly communicate and pass information between themselves using the A2A protocol.
JSON object representing the incident context.
Log of A2A messages exchanged between agents.