LlamaIndex Research Intelligence RAG Engine for Digital Infrastructure
Policy researchers and digital infrastructure architects require instant synthesis across extensive government research reports, whitepapers, and regulatory updates. Build a knowledge synthesis RAG engine using LlamaIndex.
What you are building
Develop an advanced research paper synthesis engine using LlamaIndex RAG capabilities.
How work is evaluated
Evaluates policy document query synthesis accuracy and document citations.
Datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
citations_present
Verifies presence of source document citations.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
citation_precision
Accuracy of citation attribution • target: 0.95 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Learning goals
Implement LlamaIndex DocumentSummaryIndex for large policy PDF collections
Build custom metadata extraction strategies for technical research reports
Synthesize cross-report insights with citation back-links
Resources
DPI research reports and AI strategy policy whitepapers.
Challenge setup
Evaluates policy document query synthesis accuracy and document citations.
Challenge input
JSON query input
LlamaIndex
Mandatory framework requirement
Evaluated output
JSON answer output with source references
- Verifies presence of source document citations.
- Accuracy of citation attribution • target: 0.95 • range: 0-1
- Benchmark: PolicyRAG-Bench
- Citation Precision target: 0.95
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Shows the first task, up to three tools, and up to three evidence items.
View technical recipe
Configured tools
- Llama Index · Required
- LangWatch · Optional
- ElevenLabs · Optional
Evaluation contract
- citations_present · Weight 1
- citation_precision · Weight 1
Metrics
- citation_precision · Target 0.95
Benchmarks
- PolicyRAG-Bench
- general
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON query input
Output: JSON answer output with source references
Execution harness: Python
Evaluation tools
- LlamaIndex · Data Framework
Recipe freeze marker
No recipe freeze time is recorded.