AutoGen & OpenAI o3 Fraud Detection Agent Network for Hurricane Claims
Post-disaster insurance claims surge after major events like Hurricane Beryl, increasing vulnerability to systemic claims fraud. Build an AutoGen multi-agent system powered by OpenAI o3 to analyze claims, contractor estimates, and geospatial damage feeds, detecting fraudulent patterns with 90%+ accuracy.
What you are building
The core problem, expected build, and operating context for this challenge.
Implement a collaborative multi-agent workflow using AutoGen and OpenAI o3 to audit Texas windstorm claims for post-hurricane fraud.
How work is evaluated
Evaluates multi-agent claim audit precision, recall, and structured anomaly report schema.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
schema_conformance
Verifies output structure matches required JSON schema
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
fraud_detection_f1
F1 score on standard fraud benchmark set • target: 0.85 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Design an AutoGen agent network with conversational reasoning and validation roles
Integrate OpenAI o3 reasoning capabilities for cross-referencing repair bills against imagery data
Deploy geospatial anomaly detection for storm damage validation
Evaluate multi-agent claim audit precision and false positive rates
Reference links and supporting material
Synthetic Texas coastal hurricane claims dataset paired with public NOAA wind vector overlays.
Challenge setup
Evaluates multi-agent claim audit precision, recall, and structured anomaly report schema.
Challenge input
JSON array of claim files with contractor invoices and satellite imagery metadata.
AutoGen
Agent coordination and structured negotiation
OpenAI o3
Deep policy and document reasoning
Evaluated output
JSON report containing fraud scores, flags, and justification reasoning.
- Verifies output structure matches required JSON schema
- F1 score on standard fraud benchmark set • target: 0.85 • range: 0-1
- Benchmark: ClaimsFraudBench-v1
- Fraud Detection F1 target: 0.85
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- OpenAI o3 · Optional
- AutoGen · Optional
- o3 · Required
Evaluation contract
- schema_conformance · Weight 1
- fraud_detection_f1 · Weight 1
Metrics
- fraud_detection_f1 · Target 0.85
Benchmarks
- ClaimsFraudBench-v1
- general
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON array of claim files with contractor invoices and satellite imagery metadata.
Output: JSON report containing fraud scores, flags, and justification reasoning.
Execution harness: Python
Evaluation tools
- AutoGen · Multi-agent framework
- OpenAI o3 · Reasoning LLM
Recipe freeze marker
No recipe freeze time is recorded.