Geospatial Burn Severity & Reinsurance RAG using LlamaIndex and Multimodal Embeddings
Reinsurance underwriters processing catastrophe claims after El Niño-driven wildfires must integrate complex multi-modal evidence: high-resolution Sentinel SAR radar imagery, post-fire drone burn severity index maps, and thousands of pages of field adjuster reports. Querying this heterogenous corpus requires specialized geospatial RAG architecture. In this challenge, you will implement a multi-modal spatial Retrieval-Augmented Generation (RAG) system using LlamaIndex. The system will index satellite imagery embeddings alongside unstructured PDF claims documents into a unified vector store. Your solution will execute sub-question query engines to retrieve relevant document sections and raster patches, providing transparent, evidence-backed reinsurance claim risk answers.
What you are building
Build a multi-modal RAG agent with LlamaIndex to query geospatial burn imagery and unstructured reinsurance claim documents.
How work is evaluated
Evaluates LlamaIndex RAG retrieval precision and synthesis accuracy on wildfire claim queries.
Datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
retrieval_relevance
Retrieved node IDs match gold reference documents
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
rag_answer_faithfulness
LlamaIndex FaithfulnessEvaluator score on generated answer • target: 0.92 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Learning goals
Build a multimodal RAG engine using LlamaIndex.
Index raster metadata and unstructured document collections into vector indexes.
Implement SubQuestionQueryEngine for multi-step reasoning across claims databases.
Evaluate retrieval recall and precision for geospatial climate claim queries.
Resources
Synthetic multi-modal corpus consisting of reinsurance PDF contract documents, claim assessment notes, and dNBR burn severity raster metadata.
Challenge setup
Evaluates LlamaIndex RAG retrieval precision and synthesis accuracy on wildfire claim queries.
Challenge input
JSON with query_str, focus_region_lat_lon
LlamaIndex
Mandatory framework for multimodal vector indexing and RAG.
Evaluated output
JSON with answer, retrieved_doc_ids, source_burn_index_dnbr
- Retrieved node IDs match gold reference documents
- LlamaIndex FaithfulnessEvaluator score on generated answer • target: 0.92 • range: 0-1
- Benchmark: GeoRAGBench
- Rag Answer Faithfulness target: 0.92
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Shows the first task, up to three tools, and up to three evidence items.
View technical recipe
Configured tools
- Modal · Required
- Llama Index · Optional
- Exa · Optional
- Modal · Required
Evaluation contract
- retrieval_relevance · Weight 1
- rag_answer_faithfulness · Weight 1
Metrics
- rag_answer_faithfulness · Target 0.92
Benchmarks
- GeoRAGBench
- security
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON with query_str, focus_region_lat_lon
Output: JSON with answer, retrieved_doc_ids, source_burn_index_dnbr
Execution harness: Python
Evaluation tools
- LlamaIndex · RAG Engine
Recipe freeze marker
No recipe freeze time is recorded.