Skip to main content
Back to challenges
Challenge

IPO Document RAG Pipeline for Startup Governance with LlamaIndex

Parsing multi-hundred page regulatory financial filings (DRHP) for IPO readiness requires deep document intelligence and precise indexing. Build an enterprise RAG query agent using LlamaIndex to extract financial metrics, risk factors, and governance compliance records from public startup prospectus filings.

Business OperationsHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Develop an advanced RAG and indexing pipeline using LlamaIndex to query complex financial disclosures and risk tables from regulatory DRHP documents.

Delivery guide

How work is evaluated

Evaluation

Evaluates accuracy and faithfulness of financial information retrieval from DRHP documents.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

anchor_amount_accuracy

Verifies correct financial value is retrieved from indexed context.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

rag_faithfulness_score

Faithfulness score of retrieved context relative to original text. • target: 0.95 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Implement advanced RAG indexing using LlamaIndex VectorStoreIndex and SummaryIndex.

  • Use LlamaIndex chunking and query engine transformations for complex PDF financial reports.

  • Build structured metadata extraction pipelines for financial risk assessment.

Resources and assets

Reference links and supporting material

Dataset notes

Parsed sections of Draft Red Herring Prospectus (DRHP) filings containing balance sheets, risk disclosures, and anchor investor schedules.

Challenge setup

Evaluates accuracy and faithfulness of financial information retrieval from DRHP documents.

Challenge input

JSON query string asking about specific DRHP document sections.

LlamaIndex

Advanced RAG and enterprise document indexing framework

Evaluated output

JSON object with extracted_metric_value, source_page_number, and context_snippet.

Evaluation checks
  • Verifies correct financial value is retrieved from indexed context.
  • Faithfulness score of retrieved context relative to original text. • target: 0.95 • range: 0-1
Configured evaluation evidence
  • Benchmark: Financial RAG Bench v2
  • Rag Faithfulness Score target: 0.95
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Llama Index · Required
  • Libretto · Optional
  • Cartesia · Optional

Evaluation contract

  • anchor_amount_accuracy · Weight 1
  • rag_faithfulness_score · Weight 1

Metrics

  • rag_faithfulness_score · Target 0.95

Benchmarks

  • Financial RAG Bench v2
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON query string asking about specific DRHP document sections.

    Output: JSON object with extracted_metric_value, source_page_number, and context_snippet.

    Execution harness: Python

Evaluation tools

  • LlamaIndex · agent_framework

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about IPO Document RAG Pipeline for Startup Governance with LlamaIndex