Skip to main content
Loading navigation

Claude Agents SDK Situational Awareness Diagnostic with Qwen 3

Network switch failures in process plants can disrupt control logic traffic and blind operators to critical safety alarms. Build an diagnostic agent using Claude Agents SDK integrated with Qwen 3 reasoning models to analyze network failure logs and operator alarm histories. Success is measured by correctly isolating the switch failure root cause within 30 seconds of simulated stream processing.

EngineeringHosted by Vera

What you are building

Deploy a Claude Agents SDK agent paired with Qwen 3 inference to perform root cause analysis on network switch disruptions impacting safety-critical logic.

How work is evaluated

Evaluation

Evaluates root cause detection and operator guidance accuracy during network switch outages.

Datasets

Public datasets, plus the private datasets you have access to.

How submissions are scored

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

correct_root_cause_node

Checks if root cause component is correctly identified.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

rca_precision

Precision of identified root cause and recommended remediation steps. • target: 0.95 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

  • Implement a Claude Agents SDK agent framework setup for diagnostic workflows.

  • Integrate Qwen 3 model reasoning for complex physical system state analysis.

  • Correlate network packet drop logs with PLC lost-heartbeat events.

  • Provide step-by-step situational awareness recovery actions for plant operators.

Resources

Dataset notes

Combined network packet capture traces (PCAP), SNMP traps, and SCADA alarm logs during simulated industrial switch outages.

Challenge setup

Evaluates root cause detection and operator guidance accuracy during network switch outages.

Challenge input

JSON object with snmp_traps and scada_alarms arrays

Claude Agents SDK

Required framework for building multi-turn agent workflows.

Qwen 3

Utilized for complex multi-source logical reasoning and fault isolation.

Evaluated output

JSON object with root_cause_component, primary_impact, and recommended_action

Evaluation checks
  • Checks if root cause component is correctly identified.
  • Precision of identified root cause and recommended remediation steps. • target: 0.95 • range: 0-1
Configured evaluation evidence
  • Benchmark: OTNetworkRCA-2026
  • Rca Precision target: 0.95
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Shows the first task, up to three tools, and up to three evidence items.

View technical recipe

Configured tools

Action Space
  • Claude Code · Optional
Observation
  • Claude Code · Optional
Policy Serving
  • Qwen 3 · Required
  • Claude 4.1 Opus · Optional

Evaluation contract

  • correct_root_cause_node · Weight 1
  • rca_precision · Weight 1

Metrics

  • rca_precision · Target 0.95

Benchmarks

  • OTNetworkRCA-2026
  • security

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON object with snmp_traps and scada_alarms arrays

    Output: JSON object with root_cause_component, primary_impact, and recommended_action

    Execution harness: Python

Evaluation tools

  • Claude Agents SDK · Agent Framework
  • Qwen 3 · Models

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Claude Agents SDK Situational Awareness Diagnostic with Qwen 3