TWIA Hurricane Claims Fraud Detector with OpenAI Agents SDK
Post-storm insurance fraud surges during catastrophic events like Hurricane Beryl, driving up litigation and carrier losses. Build a fraud detection workflow using OpenAI Agents SDK and GPT-5 Pro that ingests claims notes and building inspections to classify fraudulent damage patterns. Achieve 90% ROC-AUC on post-disaster claims evaluation.
What you are building
The core problem, expected build, and operating context for this challenge.
Build an agentic claims fraud detection pipeline using the OpenAI Agents SDK and GPT-5 Pro function calling to inspect hurricane claims documentation.
How work is evaluated
Evaluates claims fraud classification accuracy and output schema compliance.
Shared data for this challenge
Review public datasets and any private uploads tied to your build.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
schema_validity
Output must match expected JSON schema.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
roc_auc
Area under ROC curve for fraud detection • target: 0.9 • range: 0.5-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Implement multi-agent workflow routing using OpenAI Agents SDK
Integrate GPT-5 Pro function calling for claims note parsing
Extract structured fraud signals from unstructured catastrophe inspection reports
Evaluate classification accuracy using ROC-AUC on real-world post-storm datasets
Reference links and supporting material
Synthetic dataset of 500 post-hurricane property claims including adjuster text notes, photo descriptions, and contractor estimates.
How this agent runs
Evaluates claims fraud classification accuracy and output schema compliance.
Challenge input
JSON containing claim_id, damage_description, contractor_quote, and inspection_notes
OpenAI Agents SDK
Orchestrates stateful multi-agent communication and tool execution.
GPT-5 Pro
Provides high-reasoning accuracy for complex insurance contract and claim parsing.
Evaluated output
JSON with fraud_probability (float 0-1), flag_reason (string), and action (string)
- Output must match expected JSON schema.
- Area under ROC curve for fraud detection • target: 0.9 • range: 0.5-1
- Benchmark: ClaimsFraudBench-2026
- Roc Auc target: 0.9
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
View technical recipe
Configured tools
- OpenAI · Required
- Aim · Optional
- Seldon Core · Optional
Evaluation contract
- schema_validity · Weight 1
- roc_auc · Weight 1
Recipe state
This is a preview. The configuration can change before the evaluation recipe is locked.