Back to challenges
Challenge

Build an Inflation & Macroeconomic Analysis RAG Agent with LlamaIndex

Economic analysts require verifiable, multi-document search across monthly MAS consumer price index reports and statistical annexes. Construct an enterprise RAG agent using LlamaIndex with custom table parsers to extract inflation drivers and produce verifiable monetary analysis.

Workflow AutomationHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Construct an enterprise RAG pipeline using LlamaIndex to query MAS economic reports and output structured monetary risk forecasts.

Delivery guide

How work is evaluated

Evaluation

Evaluates retrieval accuracy and metric extraction accuracy from MAS CPI press releases.

Datasets

Shared data for this challenge

Review public datasets and any private uploads tied to your build.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

citation_presence

Ensures retrieved_citations contains valid report reference string

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

fact_extraction_accuracy

Match score of reported inflation percentage numbers against ground truth • target: 0.95 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Build advanced RAG pipelines with LlamaIndex and custom data connectors

  • Parse complex tabular price indexes and PDF statistical annexes accurately

  • Implement citation back-linking to ensure verifiable macroeconomic reporting

  • Expose query engine abstractions for monetary policy downstream tools

Resources and assets

Reference links and supporting material

Dataset notes

Singapore MAS monthly Consumer Price Index reports and statistical appendixes.

How this agent runs

Evaluates retrieval accuracy and metric extraction accuracy from MAS CPI press releases.

Challenge input

JSON object with query_string and period

LlamaIndex

Provides chunking, indexing, and verifiable retrieval pipelines

Evaluated output

JSON object with headline_cpi, mas_core_cpi, key_drivers, and retrieved_citations

Checks for
  • Ensures retrieved_citations contains valid report reference string
  • Match score of reported inflation percentage numbers against ground truth • target: 0.95 • range: 0-1
Proof of success
  • Benchmark: FinanceRAG-Bench
  • Fact Extraction Accuracy target: 0.95
  • 1 public reference case
Runtime evidence
  • Python execution harness
  • Python sandbox (unavailable on Versalist)
View technical recipe

Configured tools

Action Space
  • Llama Index · Required
  • LangWatch · Optional
  • Aider · Optional
Observation
  • Aider · Optional

Evaluation contract

  • citation_presence · Weight 1
  • fact_extraction_accuracy · Weight 1

Recipe state

This is a preview. The configuration can change before the evaluation recipe is locked.

Frequently Asked Questions about Build an Inflation & Macroeconomic Analysis RAG Agent with LlamaIndex