Build Post-Hurricane Claims Fraud Detection Agent with Claude Agents SDK
Catastrophe claims fraud spikes following severe storm events, requiring rapid document auditing. You will construct an automated fraud audit agent using the Claude Agents SDK and o3 reasoning models to analyze adjuster notes against storm data. The solution must achieve at least 88% precision in flagging suspicious claims.
What you are building
The core problem, expected build, and operating context for this challenge.
Build an intelligent claims auditing agent using Claude Agents SDK and o3 reasoning capabilities to detect inflated catastrophe loss estimates.
How work is evaluated
Evaluates the agent's ability to classify claim files as authentic or fraudulent and generate structured audit reports.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
valid_json_structure
Checks if the model returns a valid JSON matching required response schema.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
precision
Precision score of fraud detection classification • target: 0.88 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Implement autonomous claims audit workflows using the Claude Agents SDK.
Integrate o3 reasoning APIs to cross-examine claims notes with meteorological damage signatures.
Evaluate fraud detection performance using precision, recall, and cross-validation metrics.
Reference links and supporting material
Synthetic dataset containing 500 post-hurricane claim records with loss estimates, adjuster notes, and ground-truth fraud labels based on court filings.
Challenge setup
Evaluates the agent's ability to classify claim files as authentic or fraudulent and generate structured audit reports.
Challenge input
JSON object with claim_id, damage_narrative, line_items, and hurricane_intensity_vector
Claude Agents SDK
Primary agent orchestration SDK required for this challenge.
o3
Assigned reasoning engine for complex claims audit logic.
Evaluated output
JSON object with fraud_score (0.0-1.0), flag_reasons (array), and recommendation
- Checks if the model returns a valid JSON matching required response schema.
- Precision score of fraud detection classification • target: 0.88 • range: 0-1
- Benchmark: Financial Fraud Agent Benchmark
- Precision target: 0.88
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- Aim · Required
- AI21 Studio · Optional
- o3 · Optional
Evaluation contract
- valid_json_structure · Weight 1
- precision · Weight 1
Metrics
- precision · Target 0.88
Benchmarks
- Financial Fraud Agent Benchmark
- security
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON object with claim_id, damage_narrative, line_items, and hurricane_intensity_vector
Output: JSON object with fraud_score (0.0-1.0), flag_reasons (array), and recommendation
Execution harness: Python
Evaluation tools
- Claude Agents SDK · Agent Framework
- o3 · Reasoning Model API
Recipe freeze marker
No recipe freeze time is recorded.