Build a Fraud-Detection Audit Agent with Pydantic AI and Claude 4 Opus for Catastrophe Claims
Post-disaster claims fraud creates massive financial risk and delays legitimate payouts after severe events like Hurricane Beryl. Build a type-safe fraud detection pipeline using Pydantic AI and Claude 4 Opus to flag suspicious damage claims. Achieve at least 90% accuracy in detecting inflated damage estimates across simulated property claims.
What you are building
The core problem, expected build, and operating context for this challenge.
Implement structured data models and validation dependencies in Pydantic AI to audit storm claims against geospatial impact data.
How work is evaluated
Evaluates Pydantic AI claims verification logic against flagged ground-truth hurricane claims.
Shared data for this challenge
Review public datasets and any private uploads tied to your build.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
schema_validation
Output strictly adheres to required Pydantic fraud classification schema.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
accuracy
Accuracy of binary fraud classification against ground truth • target: 0.9 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Implement strong type validation for insurance claim assessment using Pydantic AI models.
Integrate geospatial storm track data to verify loss claims automatically.
Engineer prompt pipelines that leverage Claude 4 Opus for structured anomaly reasoning.
Construct automated test suites to validate claim risk scoring accuracy.
Reference links and supporting material
Synthetic dataset of 500 post-hurricane property insurance claims matched with regional NOAA weather station data.
How this agent runs
Evaluates Pydantic AI claims verification logic against flagged ground-truth hurricane claims.
Challenge input
JSON claim record with metadata and repair line items
Pydantic AI
Enforces strong dynamic typing and output schemas.
Claude 4 Opus
Provides state-of-the-art reasoning for complex anomaly detection.
Evaluated output
JSON containing risk_score (0-1), flag_reason, and decision
- Output strictly adheres to required Pydantic fraud classification schema.
- Accuracy of binary fraud classification against ground truth • target: 0.9 • range: 0-1
- Benchmark: InsuranceFraudBench
- Accuracy target: 0.9
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
View technical recipe
Configured tools
- Pydantic AI · Optional
- Aim · Optional
- Claude 4 Opus · Required
- Pydantic AI · Optional
Evaluation contract
- schema_validation · Weight 1
- accuracy · Weight 1
Recipe state
This is a preview. The configuration can change before the evaluation recipe is locked.