Geospatial Burn Severity & Reinsurance RAG using LlamaIndex and Multimodal Embeddings
Reinsurance underwriters processing catastrophe claims after El Niño-driven wildfires must integrate complex multi-modal evidence: high-resolution Sentinel SAR radar imagery, post-fire drone burn severity index maps, and thousands of pages of field adjuster reports. Querying this heterogenous corpus requires specialized geospatial RAG architecture. In this challenge, you will implement a multi-modal spatial Retrieval-Augmented Generation (RAG) system using LlamaIndex. The system will index satellite imagery embeddings alongside unstructured PDF claims documents into a unified vector store. Your solution will execute sub-question query engines to retrieve relevant document sections and raster patches, providing transparent, evidence-backed reinsurance claim risk answers.
What you are building
The core problem, expected build, and operating context for this challenge.
Build a multi-modal RAG agent with LlamaIndex to query geospatial burn imagery and unstructured reinsurance claim documents.
How work is evaluated
Evaluates LlamaIndex RAG retrieval precision and synthesis accuracy on wildfire claim queries.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
retrieval_relevance
Retrieved node IDs match gold reference documents
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
rag_answer_faithfulness
LlamaIndex FaithfulnessEvaluator score on generated answer • target: 0.92 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Build a multimodal RAG engine using LlamaIndex.
Index raster metadata and unstructured document collections into vector indexes.
Implement SubQuestionQueryEngine for multi-step reasoning across claims databases.
Evaluate retrieval recall and precision for geospatial climate claim queries.
Reference links and supporting material
Synthetic multi-modal corpus consisting of reinsurance PDF contract documents, claim assessment notes, and dNBR burn severity raster metadata.
Challenge setup
Evaluates LlamaIndex RAG retrieval precision and synthesis accuracy on wildfire claim queries.
Challenge input
JSON with query_str, focus_region_lat_lon
LlamaIndex
Mandatory framework for multimodal vector indexing and RAG.
Evaluated output
JSON with answer, retrieved_doc_ids, source_burn_index_dnbr
- Retrieved node IDs match gold reference documents
- LlamaIndex FaithfulnessEvaluator score on generated answer • target: 0.92 • range: 0-1
- Benchmark: GeoRAGBench
- Rag Answer Faithfulness target: 0.92
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- Modal · Required
- Llama Index · Optional
- Exa · Optional
- Modal · Required
Evaluation contract
- retrieval_relevance · Weight 1
- rag_answer_faithfulness · Weight 1
Metrics
- rag_answer_faithfulness · Target 0.92
Benchmarks
- GeoRAGBench
- security
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON with query_str, focus_region_lat_lon
Output: JSON with answer, retrieved_doc_ids, source_burn_index_dnbr
Execution harness: Python
Evaluation tools
- LlamaIndex · RAG Engine
Recipe freeze marker
No recipe freeze time is recorded.