Develop an OT Incident Analysis System with LlamaIndex and Mem0 Memory
Industrial control integrators face persistent knowledge gaps when troubleshooting modern manufacturing plant shutdowns across complex Purdue Model layers (Level 0 field devices up to Level 3 MES systems). In this challenge, you will build an enterprise OT Troubleshooting Agent utilizing LlamaIndex for hierarchical RAG and Mem0 for persistent operational memory. Your solution will index plant architecture manuals, network failure logs, and control logic diagrams. Using Mem0, the agent will maintain stateful memory across multiple operator interactions, remembering past equipment maintenance history and plant-specific safety constraints to accelerate downtime resolution.
What you are building
The core problem, expected build, and operating context for this challenge.
Build an OT knowledge retrieval and troubleshooting assistant using LlamaIndex RAG and Mem0 persistent memory.
How work is evaluated
Evaluates OT incident resolution context accuracy and memory retention across multi-turn operator troubleshooting sessions.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
mem0_persistence_test
Verifies facts stored in session 1 are successfully retrieved in session 2 via Mem0.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
retrieval_precision_at_k
Precision of retrieved document chunks matching correct Purdue Level SOP • target: 0.85 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Construct a LlamaIndex vector and summary index over Purdue Model OT documentation.
Integrate Mem0 to track continuous maintenance history across user interaction sessions.
Implement multi-document retrieval algorithms tailored for industrial incident resolution.
Evaluate context retrieval precision and operator response accuracy.
Reference links and supporting material
Purdue Model architecture documents, industrial switch network crash logs, and safety shutdown standard operating procedures (SOPs).
Challenge setup
Evaluates OT incident resolution context accuracy and memory retention across multi-turn operator troubleshooting sessions.
Challenge input
Operator turn query and user ID for Mem0 context retrieval
LlamaIndex
Provides hierarchical document indexing over technical OT manuals.
Mem0
Maintains persistent operator state and past incident notes across sessions.
Evaluated output
Root-cause answer with Purdue layer identification and citation
- Verifies facts stored in session 1 are successfully retrieved in session 2 via Mem0.
- Precision of retrieved document chunks matching correct Purdue Level SOP • target: 0.85 • range: 0-1
- Benchmark: Enterprise RAG Architecture Benchmark
- Retrieval Precision At K target: 0.85
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- Llama Index · Required
- LlamaIndex · Optional
- Llama 3.2 · Optional
Evaluation contract
- mem0_persistence_test · Weight 1
- retrieval_precision_at_k · Weight 1
Metrics
- retrieval_precision_at_k · Target 0.85
Benchmarks
- Enterprise RAG Architecture Benchmark
- general
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: Operator turn query and user ID for Mem0 context retrieval
Output: Root-cause answer with Purdue layer identification and citation
Execution harness: Python
Evaluation tools
- LlamaIndex · rag_framework
- Mem0 · memory_store
Recipe freeze marker
No recipe freeze time is recorded.