Back to evaluations
Public evaluation
switch_failure_rca
Evaluates root cause detection and operator guidance accuracy during network switch outages.
Evaluation type
task based
Challenge
Claude Agents SDK Situational Awareness Diagnostic with Qwen 3
Difficulty
Advanced
Rigor
Unspecified
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Tasks
1
Benchmarks
0
Criteria
0
Task templates
Inputs and expected outputs.
Task 1
switch_failure_rca
Evaluates agent analysis of concurrent SNMP logs and SCADA alarms.
Input format
JSON object with snmp_traps and scada_alarms arrays
Output format
JSON object with root_cause_component, primary_impact, and recommended_action