Skip to main content
Back to challenges
Challenge

Build an Inflation & Macroeconomic Analysis RAG Agent with LlamaIndex

Economic analysts require verifiable, multi-document search across monthly MAS consumer price index reports and statistical annexes. Construct an enterprise RAG agent using LlamaIndex with custom table parsers to extract inflation drivers and produce verifiable monetary analysis.

Workflow AutomationHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Construct an enterprise RAG pipeline using LlamaIndex to query MAS economic reports and output structured monetary risk forecasts.

Delivery guide

How work is evaluated

Evaluation

Evaluates retrieval accuracy and metric extraction accuracy from MAS CPI press releases.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

citation_presence

Ensures retrieved_citations contains valid report reference string

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

fact_extraction_accuracy

Match score of reported inflation percentage numbers against ground truth • target: 0.95 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Build advanced RAG pipelines with LlamaIndex and custom data connectors

  • Parse complex tabular price indexes and PDF statistical annexes accurately

  • Implement citation back-linking to ensure verifiable macroeconomic reporting

  • Expose query engine abstractions for monetary policy downstream tools

Resources and assets

Reference links and supporting material

Dataset notes

Singapore MAS monthly Consumer Price Index reports and statistical appendixes.

Challenge setup

Evaluates retrieval accuracy and metric extraction accuracy from MAS CPI press releases.

Challenge input

JSON object with query_string and period

LlamaIndex

Provides chunking, indexing, and verifiable retrieval pipelines

Evaluated output

JSON object with headline_cpi, mas_core_cpi, key_drivers, and retrieved_citations

Evaluation checks
  • Ensures retrieved_citations contains valid report reference string
  • Match score of reported inflation percentage numbers against ground truth • target: 0.95 • range: 0-1
Configured evaluation evidence
  • Benchmark: FinanceRAG-Bench
  • Fact Extraction Accuracy target: 0.95
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Llama Index · Required
  • LangWatch · Optional
  • Aider · Optional
Observation
  • Aider · Optional

Evaluation contract

  • citation_presence · Weight 1
  • fact_extraction_accuracy · Weight 1

Metrics

  • fact_extraction_accuracy · Target 0.95

Benchmarks

  • FinanceRAG-Bench
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON object with query_string and period

    Output: JSON object with headline_cpi, mas_core_cpi, key_drivers, and retrieved_citations

    Execution harness: Python

Evaluation tools

  • LlamaIndex · Data & RAG Framework

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Build an Inflation & Macroeconomic Analysis RAG Agent with LlamaIndex