Back to challenges
Challenge

Develop an ESG Asset Management Auditor with LlamaIndex & GPT-5 Pro

Asset managers in Singapore must verify fund prospectus disclosures against MAS sustainability guidelines. Build a LlamaIndex RAG pipeline leveraging GPT-5 Pro to ingest complex fund documentation, detect greenwashing risks, and produce compliance scorecards with over 90% recall on regulatory violation benchmarks.

Workflow AutomationHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Construct an ESG compliance retrieval and audit workflow using LlamaIndex index structures and GPT-5 Pro to detect greenwashing risks.

Delivery guide

How work is evaluated

Evaluation

Evaluates the RAG pipeline's ability to detect unbacked ESG claims and provide accurate document citations.

Datasets

Shared data for this challenge

Review public datasets and any private uploads tied to your build.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

citation_presence

Verifies that every flagged claim includes direct contextual quotes.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

violation_recall

Recall metric of identified ESG non-compliance claims. • target: 0.92 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Master LlamaIndex advanced index structures (VectorStoreIndex, SummaryIndex) for dense financial documents

  • Leverage GPT-5 Pro reasoning capabilities for subtle ESG claim verification

  • Implement citation-backed claim validation tools to combat corporate greenwashing

  • Evaluate retrieval recall and regulatory alignment against official MAS guidelines

Resources and assets

Reference links and supporting material

Dataset notes

Synthetic corpus of 20 asset management fund prospectuses annotated with MAS ESG guideline violations.

How this agent runs

Evaluates the RAG pipeline's ability to detect unbacked ESG claims and provide accurate document citations.

Challenge input

PDF document or plain text prospectus text

LlamaIndex

Offers hierarchical indexing and RAG query engines for dense financial documents.

GPT-5 Pro

Provides state-of-the-art multi-step reasoning for compliance checking.

Evaluated output

JSON audit result listing flagged claims with source quotes and confidence

Checks for
  • Verifies that every flagged claim includes direct contextual quotes.
  • Recall metric of identified ESG non-compliance claims. • target: 0.92 • range: 0-1
Proof of success
  • Benchmark: ESGRagBench
  • Violation Recall target: 0.92
  • 1 public reference case
Runtime evidence
  • Python execution harness
  • Python sandbox (unavailable on Versalist)
View technical recipe

Configured tools

Action Space
  • Llama Index · Required
  • GPT-5 Pro · Optional
  • Yupp AI · Optional

Evaluation contract

  • citation_presence · Weight 1
  • violation_recall · Weight 1

Recipe state

This is a preview. The configuration can change before the evaluation recipe is locked.

Frequently Asked Questions about Develop an ESG Asset Management Auditor with LlamaIndex & GPT-5 Pro