Manufacturing Knowledge Graph RAG System with LlamaIndex
Factory managers struggle to query disparate MES logs, maintenance records, and quality documents simultaneously. Build an enterprise manufacturing RAG pipeline using LlamaIndex to query structured asset data and unstructured downtime reports. Reach an F1 score above 0.88 on complex operational query benchmarks.
What you are building
The core problem, expected build, and operating context for this challenge.
Build an enterprise manufacturing search engine with LlamaIndex connecting structured SQL tables and unstructured equipment manuals.
How work is evaluated
Evaluate retrieval precision and answer correctness across complex plant operational queries.
Shared data for this challenge
Review public datasets and any private uploads tied to your build.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
citation_correctness
Cites correct source document and table
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
f1_score
F1 evaluation score on factual operational QA • target: 0.88 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Construct a LlamaIndex PropertyGraphIndex combining unstructured manual PDFs and structured SQL MES logs
Implement hybrid retrieval (vector search + graph traversal) for manufacturing operations
Evaluate response accuracy against factory operational query ground truths
Optimize indexing speed and query latency for large manual sets
Reference links and supporting material
Corpus of factory downtime records, work orders, and PDF technical service manuals.
How this agent runs
Evaluate retrieval precision and answer correctness across complex plant operational queries.
Challenge input
JSON object with query string
LlamaIndex
Framework for data indexing and graph rag operations.
Evaluated output
JSON object with answer, source_nodes, and metric_citations
- Cites correct source document and table
- F1 evaluation score on factual operational QA • target: 0.88 • range: 0-1
- Benchmark: Enterprise RAG Industrial Benchmark
- F1 Score target: 0.88
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
View technical recipe
Configured tools
- Llama Index · Required
- LlamaIndex · Optional
- Llama 3.2 · Optional
Evaluation contract
- citation_correctness · Weight 1
- f1_score · Weight 1
Recipe state
This is a preview. The configuration can change before the evaluation recipe is locked.