Hurricane Claims Fraud Detection RAG Agent with LlamaIndex and Claude 4.1 Opus
Post-disaster insurance claims spikes often enable systemic fraudulent claims that evade traditional manual review. You will build a multi-modal RAG agent using LlamaIndex and Claude 4.1 Opus to audit coastal property damage reports against meteorological radar and satellite feeds. The solution must achieve an F1-score of at least 0.88 in flag accuracy.
What you are building
The core problem, expected build, and operating context for this challenge.
Build an intelligent claims auditing agent using LlamaIndex vector indices and Claude 4.1 Opus reasoning to detect fraudulent storm damage claims by cross-referencing geospatial feeds.
How work is evaluated
Evaluate the fraud detection agent on synthetic Beryl property claims and satellite image vectors.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
valid_output_structure
Checks if output contains fraud_score, flag_status, and reasoning_summary.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
f1_score
Harmonic mean of precision and recall for fraud classification. • target: 0.88 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Implement multi-modal indexing of satellite imagery and claims text using LlamaIndex.
Integrate Claude 4.1 Opus for reasoning over complex wind damage vector fields.
Evaluate claims validity by cross-referencing temporal weather APIs and adjusters' notes.
Deploy automated fraud scoring pipelines with lower than 5% false positive rates.
Reference links and supporting material
Synthetic dataset of 500 post-hurricane property claims coupled with spatial radar wind field raster records and damage photo logs.
Challenge setup
Evaluate the fraud detection agent on synthetic Beryl property claims and satellite image vectors.
Challenge input
JSON object containing claim_id, text_report, damage_date, geolocation, and ground_truth_imagery_ref
LlamaIndex
Provides structured document indexing and tool querying capabilities for claims audit.
Claude 4.1 Opus
Offers high-capacity reasoning over complex, multi-modal geospatial damage feeds.
Evaluated output
JSON object with fraud_score (0.0 to 1.0), reasoning_summary, and flag_status (BOOLEAN)
- Checks if output contains fraud_score, flag_status, and reasoning_summary.
- Harmonic mean of precision and recall for fraud classification. • target: 0.88 • range: 0-1
- Benchmark: ClimateFraudEval-2026
- F1 Score target: 0.88
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- Llama Index · Optional
- Exa · Optional
- Claude 4.1 Opus · Required
Evaluation contract
- valid_output_structure · Weight 1
- f1_score · Weight 1
Metrics
- f1_score · Target 0.88
Benchmarks
- ClimateFraudEval-2026
- general
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON object containing claim_id, text_report, damage_date, geolocation, and ground_truth_imagery_ref
Output: JSON object with fraud_score (0.0 to 1.0), reasoning_summary, and flag_status (BOOLEAN)
Execution harness: Python
Evaluation tools
- LlamaIndex · Agent & RAG Framework
- Claude 4.1 Opus · LLM Inference Engine
Recipe freeze marker
No recipe freeze time is recorded.