Skip to main content
Loading navigation
Back to challenges

Build an Inflation & Macroeconomic Analysis RAG Agent with LlamaIndex

Economic analysts require verifiable, multi-document search across monthly MAS consumer price index reports and statistical annexes. Construct an enterprise RAG agent using LlamaIndex with custom table parsers to extract inflation drivers and produce verifiable monetary analysis.

Workflow AutomationHosted by Vera
Sign in free to start

What you are building

Construct an enterprise RAG pipeline using LlamaIndex to query MAS economic reports and output structured monetary risk forecasts.

How work is evaluated

Evaluation

Evaluates retrieval accuracy and metric extraction accuracy from MAS CPI press releases.

Datasets

Public datasets, plus the private datasets you have access to.

How submissions are scored

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

citation_presence

Ensures retrieved_citations contains valid report reference string

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

fact_extraction_accuracy

Match score of reported inflation percentage numbers against ground truth • target: 0.95 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

  • Build advanced RAG pipelines with LlamaIndex and custom data connectors

  • Parse complex tabular price indexes and PDF statistical annexes accurately

  • Implement citation back-linking to ensure verifiable macroeconomic reporting

  • Expose query engine abstractions for monetary policy downstream tools

Resources

Dataset notes

Singapore MAS monthly Consumer Price Index reports and statistical appendixes.

Challenge setup

Evaluates retrieval accuracy and metric extraction accuracy from MAS CPI press releases.

Challenge input

JSON object with query_string and period

LlamaIndex

Provides chunking, indexing, and verifiable retrieval pipelines

Evaluated output

JSON object with headline_cpi, mas_core_cpi, key_drivers, and retrieved_citations

Evaluation checks
  • Ensures retrieved_citations contains valid report reference string
  • Match score of reported inflation percentage numbers against ground truth • target: 0.95 • range: 0-1
Configured evaluation evidence
  • Benchmark: FinanceRAG-Bench
  • Fact Extraction Accuracy target: 0.95
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Shows the first task, up to three tools, and up to three evidence items.

View technical recipe

Configured tools

Action Space
  • Llama Index · Required
  • LangWatch · Optional
  • Aider · Optional
Observation
  • Aider · Optional

Evaluation contract

  • citation_presence · Weight 1
  • fact_extraction_accuracy · Weight 1

Metrics

  • fact_extraction_accuracy · Target 0.95

Benchmarks

  • FinanceRAG-Bench
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON object with query_string and period

    Output: JSON object with headline_cpi, mas_core_cpi, key_drivers, and retrieved_citations

    Execution harness: Python

Evaluation tools

  • LlamaIndex · Data & RAG Framework

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Build an Inflation & Macroeconomic Analysis RAG Agent with LlamaIndex