Post-Hurricane Claims Fraud Detection with Pydantic AI and Qwen 3
Following major natural disasters, fraudulent property claims surge, leading to multi-million dollar losses. You will build a type-safe claims fraud analysis pipeline using Pydantic AI and Qwen 3 to evaluate damage reports against satellite imagery data. Achieve a fraud classification F1-score of 0.88 or higher on hurricane claim benchmarks.
What you are building
The core problem, expected build, and operating context for this challenge.
Integrate Pydantic AI schemas with Qwen 3 vision-language inference to validate post-disaster insurance claims.
How work is evaluated
Evaluate claim validation accuracy and fraud classification metrics against labeled test claims.
Shared data for this challenge
Review public datasets and any private uploads tied to your build.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
schema_conformance
Output strictly adheres to required FraudAssessment Pydantic model.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
f1_score
F1 score for binary fraud classification • target: 0.88 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Construct strictly typed claim validation models using Pydantic AI
Process multimodal imagery and text claims data using Qwen 3
Evaluate anomaly scores across claim histories and physical damage metrics
Deploy a structured fraud detection pipeline with reproducible evaluation metrics
Reference links and supporting material
Synthetic and anonymized post-hurricane insurance claim dataset containing text summaries, imagery metadata, and ground truth fraud labels.
How this agent runs
Evaluate claim validation accuracy and fraud classification metrics against labeled test claims.
Challenge input
JSON containing claim text, damage description, and image URL
Pydantic AI
Structured outputs and type-safe agent orchestrations.
Qwen 3
Multimodal reasoning over document text and claim images.
Evaluated output
JSON object matching FraudAssessment Pydantic schema
- Output strictly adheres to required FraudAssessment Pydantic model.
- F1 score for binary fraud classification • target: 0.88 • range: 0-1
- Benchmark: ClaimsFraudBench-2026
- F1 Score target: 0.88
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
View technical recipe
Configured tools
- Pydantic AI · Optional
- Aim · Optional
- Qwen 3 · Required
- Pydantic AI · Optional
Evaluation contract
- schema_conformance · Weight 1
- f1_score · Weight 1
Recipe state
This is a preview. The configuration can change before the evaluation recipe is locked.