Manufacturing Knowledge Graph RAG System with LlamaIndex
Factory managers struggle to query disparate MES logs, maintenance records, and quality documents simultaneously. Build an enterprise manufacturing RAG pipeline using LlamaIndex to query structured asset data and unstructured downtime reports. Reach an F1 score above 0.88 on complex operational query benchmarks.
What you are building
Build an enterprise manufacturing search engine with LlamaIndex connecting structured SQL tables and unstructured equipment manuals.
How work is evaluated
Evaluate retrieval precision and answer correctness across complex plant operational queries.
Datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
citation_correctness
Cites correct source document and table
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
f1_score
F1 evaluation score on factual operational QA • target: 0.88 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Learning goals
Construct a LlamaIndex PropertyGraphIndex combining unstructured manual PDFs and structured SQL MES logs
Implement hybrid retrieval (vector search + graph traversal) for manufacturing operations
Evaluate response accuracy against factory operational query ground truths
Optimize indexing speed and query latency for large manual sets
Resources
Corpus of factory downtime records, work orders, and PDF technical service manuals.
Challenge setup
Evaluate retrieval precision and answer correctness across complex plant operational queries.
Challenge input
JSON object with query string
LlamaIndex
Framework for data indexing and graph rag operations.
Evaluated output
JSON object with answer, source_nodes, and metric_citations
- Cites correct source document and table
- F1 evaluation score on factual operational QA • target: 0.88 • range: 0-1
- Benchmark: Enterprise RAG Industrial Benchmark
- F1 Score target: 0.88
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Shows the first task, up to three tools, and up to three evidence items.
View technical recipe
Configured tools
- Llama Index · Required
- LlamaIndex · Optional
- Llama 3.2 · Optional
Evaluation contract
- citation_correctness · Weight 1
- f1_score · Weight 1
Metrics
- f1_score · Target 0.88
Benchmarks
- Enterprise RAG Industrial Benchmark
- general
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON object with query string
Output: JSON object with answer, source_nodes, and metric_citations
Execution harness: Python
Evaluation tools
- LlamaIndex · RAG Framework
Recipe freeze marker
No recipe freeze time is recorded.