Skip to main content
Loading navigation
Back to challenges

Plant Safety Incident Root-Cause Analyzer with LlamaIndex

Network switch failures in process plants can corrupt operator situational awareness and trigger false trip logic. Build a RAG-backed root-cause analysis workflow using LlamaIndex and Gemini 2.5 Flash to synthesize plant historian alarm logs, network event traces, and standard operating procedures during safety incidents.

EngineeringHosted by Vera
Sign in free to start

What you are building

Construct a LlamaIndex RAG pipeline powered by Gemini 2.5 Flash to perform cross-domain correlation between network switch failures and SCADA safety alarms.

How work is evaluated

Evaluation

Evaluates the RAG system's accuracy in identifying network failure as the primary cause of simultaneous downstream instrument offline alarms.

Datasets

Public datasets, plus the private datasets you have access to.

How submissions are scored

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

root_cause_correctness

Ensures the switch failure is marked as root cause rather than sensor fault.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

timeline_reconstruction_score

Evaluates match percentage between reconstructed event timeline and true scenario. • target: 0.9 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

  • Index heterogeneous industrial logs using LlamaIndex data connectors

  • Query Gemini 2.5 Flash for multimodal event sequence analysis

  • Differentiate between physical process trips and digital network switch outages

  • Provide step-by-step situational awareness reports for plant operators

Resources

Dataset notes

Combined industrial alarm logs, network switch syslogs, and safety instrument system trip records.

Challenge setup

Evaluates the RAG system's accuracy in identifying network failure as the primary cause of simultaneous downstream instrument offline alarms.

Challenge input

JSON containing combined_incident_logs list.

LlamaIndex

Provides structured indexing and vector querying tailored for text document processing.

Gemini 2.5 Flash

Offers long context handling and ultra-fast generation latency.

Evaluated output

JSON containing root_cause, primary_failure_time, and affected_subsystems.

Evaluation checks
  • Ensures the switch failure is marked as root cause rather than sensor fault.
  • Evaluates match percentage between reconstructed event timeline and true scenario. • target: 0.9 • range: 0-1
Configured evaluation evidence
  • Benchmark: Industrial Safety RAG Benchmark
  • Timeline Reconstruction Score target: 0.9
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Shows the first task, up to three tools, and up to three evidence items.

View technical recipe

Configured tools

Action Space
  • Llama Index · Required
  • LlamaIndex · Optional
  • Llama 3.2 · Optional

Evaluation contract

  • root_cause_correctness · Weight 1
  • timeline_reconstruction_score · Weight 1

Metrics

  • timeline_reconstruction_score · Target 0.9

Benchmarks

  • Industrial Safety RAG Benchmark
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON containing combined_incident_logs list.

    Output: JSON containing root_cause, primary_failure_time, and affected_subsystems.

    Execution harness: Python

Evaluation tools

  • LlamaIndex · RAG Framework
  • Gemini 2.5 Flash · LLM Inference Engine

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Plant Safety Incident Root-Cause Analyzer with LlamaIndex