Develop an ESG Asset Management Auditor with LlamaIndex & GPT-5 Pro
Asset managers in Singapore must verify fund prospectus disclosures against MAS sustainability guidelines. Build a LlamaIndex RAG pipeline leveraging GPT-5 Pro to ingest complex fund documentation, detect greenwashing risks, and produce compliance scorecards with over 90% recall on regulatory violation benchmarks.
What you are building
The core problem, expected build, and operating context for this challenge.
Construct an ESG compliance retrieval and audit workflow using LlamaIndex index structures and GPT-5 Pro to detect greenwashing risks.
How work is evaluated
Evaluates the RAG pipeline's ability to detect unbacked ESG claims and provide accurate document citations.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
citation_presence
Verifies that every flagged claim includes direct contextual quotes.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
violation_recall
Recall metric of identified ESG non-compliance claims. • target: 0.92 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Master LlamaIndex advanced index structures (VectorStoreIndex, SummaryIndex) for dense financial documents
Leverage GPT-5 Pro reasoning capabilities for subtle ESG claim verification
Implement citation-backed claim validation tools to combat corporate greenwashing
Evaluate retrieval recall and regulatory alignment against official MAS guidelines
Reference links and supporting material
Synthetic corpus of 20 asset management fund prospectuses annotated with MAS ESG guideline violations.
Challenge setup
Evaluates the RAG pipeline's ability to detect unbacked ESG claims and provide accurate document citations.
Challenge input
PDF document or plain text prospectus text
LlamaIndex
Offers hierarchical indexing and RAG query engines for dense financial documents.
GPT-5 Pro
Provides state-of-the-art multi-step reasoning for compliance checking.
Evaluated output
JSON audit result listing flagged claims with source quotes and confidence
- Verifies that every flagged claim includes direct contextual quotes.
- Recall metric of identified ESG non-compliance claims. • target: 0.92 • range: 0-1
- Benchmark: ESGRagBench
- Violation Recall target: 0.92
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- Llama Index · Required
- GPT-5 Pro · Optional
- Yupp AI · Optional
Evaluation contract
- citation_presence · Weight 1
- violation_recall · Weight 1
Metrics
- violation_recall · Target 0.92
Benchmarks
- ESGRagBench
- general
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: PDF document or plain text prospectus text
Output: JSON audit result listing flagged claims with source quotes and confidence
Execution harness: Python
Evaluation tools
- LlamaIndex · Data Framework
- GPT-5 Pro · Inference Model
Recipe freeze marker
No recipe freeze time is recorded.