Back to challenges
Challenge

Hurricane Claims Fraud Detection RAG Agent with LlamaIndex and Claude 4.1 Opus

Post-disaster insurance claims spikes often enable systemic fraudulent claims that evade traditional manual review. You will build a multi-modal RAG agent using LlamaIndex and Claude 4.1 Opus to audit coastal property damage reports against meteorological radar and satellite feeds. The solution must achieve an F1-score of at least 0.88 in flag accuracy.

Machine LearningHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Build an intelligent claims auditing agent using LlamaIndex vector indices and Claude 4.1 Opus reasoning to detect fraudulent storm damage claims by cross-referencing geospatial feeds.

Delivery guide

How work is evaluated

Evaluation

Evaluate the fraud detection agent on synthetic Beryl property claims and satellite image vectors.

Datasets

Shared data for this challenge

Review public datasets and any private uploads tied to your build.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

valid_output_structure

Checks if output contains fraud_score, flag_status, and reasoning_summary.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

f1_score

Harmonic mean of precision and recall for fraud classification. • target: 0.88 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Implement multi-modal indexing of satellite imagery and claims text using LlamaIndex.

  • Integrate Claude 4.1 Opus for reasoning over complex wind damage vector fields.

  • Evaluate claims validity by cross-referencing temporal weather APIs and adjusters' notes.

  • Deploy automated fraud scoring pipelines with lower than 5% false positive rates.

Resources and assets

Reference links and supporting material

Dataset notes

Synthetic dataset of 500 post-hurricane property claims coupled with spatial radar wind field raster records and damage photo logs.

How this agent runs

Evaluate the fraud detection agent on synthetic Beryl property claims and satellite image vectors.

Challenge input

JSON object containing claim_id, text_report, damage_date, geolocation, and ground_truth_imagery_ref

LlamaIndex

Provides structured document indexing and tool querying capabilities for claims audit.

Claude 4.1 Opus

Offers high-capacity reasoning over complex, multi-modal geospatial damage feeds.

Evaluated output

JSON object with fraud_score (0.0 to 1.0), reasoning_summary, and flag_status (BOOLEAN)

Checks for
  • Checks if output contains fraud_score, flag_status, and reasoning_summary.
  • Harmonic mean of precision and recall for fraud classification. • target: 0.88 • range: 0-1
Proof of success
  • Benchmark: ClimateFraudEval-2026
  • F1 Score target: 0.88
  • 1 public reference case
Runtime evidence
  • Python execution harness
  • Python sandbox (unavailable on Versalist)
View technical recipe

Configured tools

Action Space
  • Llama Index · Optional
  • Exa · Optional
Policy Serving
  • Claude 4.1 Opus · Required

Evaluation contract

  • valid_output_structure · Weight 1
  • f1_score · Weight 1

Recipe state

This is a preview. The configuration can change before the evaluation recipe is locked.

Frequently Asked Questions about Hurricane Claims Fraud Detection RAG Agent with LlamaIndex and Claude 4.1 Opus