Build a Fraud-Detection Audit Agent with Pydantic AI and Claude 4 Opus for Catastrophe Claims
Post-disaster claims fraud creates massive financial risk and delays legitimate payouts after severe events like Hurricane Beryl. Build a type-safe fraud detection pipeline using Pydantic AI and Claude 4 Opus to flag suspicious damage claims. Achieve at least 90% accuracy in detecting inflated damage estimates across simulated property claims.
What you are building
The core problem, expected build, and operating context for this challenge.
Implement structured data models and validation dependencies in Pydantic AI to audit storm claims against geospatial impact data.
How work is evaluated
Evaluates Pydantic AI claims verification logic against flagged ground-truth hurricane claims.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
schema_validation
Output strictly adheres to required Pydantic fraud classification schema.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
accuracy
Accuracy of binary fraud classification against ground truth • target: 0.9 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Implement strong type validation for insurance claim assessment using Pydantic AI models.
Integrate geospatial storm track data to verify loss claims automatically.
Engineer prompt pipelines that leverage Claude 4 Opus for structured anomaly reasoning.
Construct automated test suites to validate claim risk scoring accuracy.
Reference links and supporting material
Synthetic dataset of 500 post-hurricane property insurance claims matched with regional NOAA weather station data.
Challenge setup
Evaluates Pydantic AI claims verification logic against flagged ground-truth hurricane claims.
Challenge input
JSON claim record with metadata and repair line items
Pydantic AI
Enforces strong dynamic typing and output schemas.
Claude 4 Opus
Provides state-of-the-art reasoning for complex anomaly detection.
Evaluated output
JSON containing risk_score (0-1), flag_reason, and decision
- Output strictly adheres to required Pydantic fraud classification schema.
- Accuracy of binary fraud classification against ground truth • target: 0.9 • range: 0-1
- Benchmark: InsuranceFraudBench
- Accuracy target: 0.9
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- Pydantic AI · Optional
- Aim · Optional
- Claude 4 Opus · Required
- Pydantic AI · Optional
Evaluation contract
- schema_validation · Weight 1
- accuracy · Weight 1
Metrics
- accuracy · Target 0.9
Benchmarks
- InsuranceFraudBench
- security
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON claim record with metadata and repair line items
Output: JSON containing risk_score (0-1), flag_reason, and decision
Execution harness: Python
Evaluation tools
- Pydantic AI · Agent Framework
- Claude 4 Opus · LLM Provider
Recipe freeze marker
No recipe freeze time is recorded.