Skip to main content
Back to challenges
Challenge

Purdue Model OT/IT Industrial Knowledge Graph RAG with LlamaIndex

Plant security and control data structured across Level 0 (sensors) to Level 4 (ERP) under the Purdue Model is difficult to query safely without crossing network boundary segments. Build a context-aware RAG pipeline using LlamaIndex to query cross-layer OT/IT architecture documents with 100% compliance with security level access rules.

EngineeringHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Implement a Purdue-model aware RAG system using LlamaIndex to allow safe cross-layer queries across OT operational zones and IT enterprise levels.

Delivery guide

How work is evaluated

Evaluation

Evaluates security boundary filtering compliance and query answer precision across Purdue model layers.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

Boundary Block Verification

Confirms that Level 4 user attempting Level 1 direct access is blocked.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

authorized_query_precision

Accuracy score on valid, allowed cross-level queries. • target: 0.95 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Construct a hierarchical Property Graph Index in LlamaIndex representing Purdue Enterprise levels (0 through 4)

  • Implement role-based metadata filtering to prevent unauthorized data movement across Level 2 OT / Level 4 IT boundaries

  • Build dynamic SQL/Vector hybrid retrieval tools targeting plant historian and ERP schemas

Resources and assets

Reference links and supporting material

Dataset notes

Documents describing manufacturing execution systems (MES), SCADA topologies, and enterprise network architectures classified by Purdue model layers.

Challenge setup

Evaluates security boundary filtering compliance and query answer precision across Purdue model layers.

Challenge input

JSON containing user_role, user_purdue_level, and query_text

LlamaIndex

Offers property graph indices and customizable node postprocessor pipelines.

Evaluated output

JSON with response, boundary_violation_blocked, and retrieved_nodes_count

Evaluation checks
  • Confirms that Level 4 user attempting Level 1 direct access is blocked.
  • Accuracy score on valid, allowed cross-level queries. • target: 0.95 • range: 0-1
Configured evaluation evidence
  • Benchmark: OT Security Context RAG Benchmark
  • Authorized Query Precision target: 0.95
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Llama Index · Required
  • LlamaIndex · Optional
  • Llama 3.2 · Optional

Evaluation contract

  • Boundary Block Verification · Weight 1
  • authorized_query_precision · Weight 1

Metrics

  • authorized_query_precision · Target 0.95

Benchmarks

  • OT Security Context RAG Benchmark
  • security

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON containing user_role, user_purdue_level, and query_text

    Output: JSON with response, boundary_violation_blocked, and retrieved_nodes_count

    Execution harness: Python

Evaluation tools

  • LlamaIndex · Data Framework & RAG Engine

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Purdue Model OT/IT Industrial Knowledge Graph RAG with LlamaIndex