LlamaIndex Research Intelligence RAG Engine for Digital Infrastructure
Policy researchers and digital infrastructure architects require instant synthesis across extensive government research reports, whitepapers, and regulatory updates. Build a knowledge synthesis RAG engine using LlamaIndex.
What you are building
The core problem, expected build, and operating context for this challenge.
Develop an advanced research paper synthesis engine using LlamaIndex RAG capabilities.
How work is evaluated
Evaluates policy document query synthesis accuracy and document citations.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
citations_present
Verifies presence of source document citations.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
citation_precision
Accuracy of citation attribution • target: 0.95 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Implement LlamaIndex DocumentSummaryIndex for large policy PDF collections
Build custom metadata extraction strategies for technical research reports
Synthesize cross-report insights with citation back-links
Reference links and supporting material
DPI research reports and AI strategy policy whitepapers.
Challenge setup
Evaluates policy document query synthesis accuracy and document citations.
Challenge input
JSON query input
LlamaIndex
Mandatory framework requirement
Evaluated output
JSON answer output with source references
- Verifies presence of source document citations.
- Accuracy of citation attribution • target: 0.95 • range: 0-1
- Benchmark: PolicyRAG-Bench
- Citation Precision target: 0.95
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- Llama Index · Required
- LangWatch · Optional
- ElevenLabs · Optional
Evaluation contract
- citations_present · Weight 1
- citation_precision · Weight 1
Metrics
- citation_precision · Target 0.95
Benchmarks
- PolicyRAG-Bench
- general
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON query input
Output: JSON answer output with source references
Execution harness: Python
Evaluation tools
- LlamaIndex · Data Framework
Recipe freeze marker
No recipe freeze time is recorded.