Post-Hurricane Claims Fraud Detection with Pydantic AI and Qwen 3
Following major natural disasters, fraudulent property claims surge, leading to multi-million dollar losses. You will build a type-safe claims fraud analysis pipeline using Pydantic AI and Qwen 3 to evaluate damage reports against satellite imagery data. Achieve a fraud classification F1-score of 0.88 or higher on hurricane claim benchmarks.
What you are building
Integrate Pydantic AI schemas with Qwen 3 vision-language inference to validate post-disaster insurance claims.
How work is evaluated
Evaluate claim validation accuracy and fraud classification metrics against labeled test claims.
Datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
schema_conformance
Output strictly adheres to required FraudAssessment Pydantic model.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
f1_score
F1 score for binary fraud classification • target: 0.88 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Learning goals
Construct strictly typed claim validation models using Pydantic AI
Process multimodal imagery and text claims data using Qwen 3
Evaluate anomaly scores across claim histories and physical damage metrics
Deploy a structured fraud detection pipeline with reproducible evaluation metrics
Resources
Synthetic and anonymized post-hurricane insurance claim dataset containing text summaries, imagery metadata, and ground truth fraud labels.
Challenge setup
Evaluate claim validation accuracy and fraud classification metrics against labeled test claims.
Challenge input
JSON containing claim text, damage description, and image URL
Pydantic AI
Structured outputs and type-safe agent orchestrations.
Qwen 3
Multimodal reasoning over document text and claim images.
Evaluated output
JSON object matching FraudAssessment Pydantic schema
- Output strictly adheres to required FraudAssessment Pydantic model.
- F1 score for binary fraud classification • target: 0.88 • range: 0-1
- Benchmark: ClaimsFraudBench-2026
- F1 Score target: 0.88
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Shows the first task, up to three tools, and up to three evidence items.
View technical recipe
Configured tools
- Pydantic AI · Optional
- Aim · Optional
- Qwen 3 · Required
- Pydantic AI · Optional
Evaluation contract
- schema_conformance · Weight 1
- f1_score · Weight 1
Metrics
- f1_score · Target 0.88
Benchmarks
- ClaimsFraudBench-2026
- general
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON containing claim text, damage description, and image URL
Output: JSON object matching FraudAssessment Pydantic schema
Execution harness: Python
Evaluation tools
- Pydantic AI · Agent Framework
- Qwen 3 · LLM Provider
Recipe freeze marker
No recipe freeze time is recorded.