Skip to main content
Back to challenges
Challenge

Build an Automated Insurtech Claims Triaging System with OpenAI Agents SDK

Insurance providers handle high volumes of health and casualty claims requiring fast, accurate triaging and fraud scoring. Build an autonomous claims processing agent using the OpenAI Agents SDK to parse intake documents, run automated policy validation, and route clean claims directly to payout.

Workflow AutomationHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Implement an autonomous claims triaging system with OpenAI Agents SDK to parse insurance documentation and assign automated payout decisions.

Delivery guide

How work is evaluated

Evaluation

Evaluates OpenAI Agents SDK agent accuracy in triaging claims, identifying fraudulent billing indicators, and deciding settlement routing.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

approval_logic_pass

Ensures low-risk claim within policy limit is set to AUTO_APPROVED

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

fraud_score_accuracy

Precision of fraud risk rating against synthetic evaluation baseline • target: 0.95 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Develop autonomous multi-turn agents using the OpenAI Agents SDK

  • Implement native function calling for policy limit database queries

  • Apply automated fraud detection heuristics on medical invoice receipts

  • Output standardized payout authorization or human-adjuster escalation payloads

Resources and assets

Reference links and supporting material

Dataset notes

Synthetic insurance claims data and medical bill receipt records.

Challenge setup

Evaluates OpenAI Agents SDK agent accuracy in triaging claims, identifying fraudulent billing indicators, and deciding settlement routing.

Challenge input

JSON object with claim_id, policy_num, claimed_amount_sgd, diagnosis_code, hospital_bill_url

OpenAI Agents SDK

Provides multi-agent handoffs and robust function calling capabilities

Evaluated output

JSON object with claim_status, approved_payout_sgd, fraud_risk_score, escalation_reason

Evaluation checks
  • Ensures low-risk claim within policy limit is set to AUTO_APPROVED
  • Precision of fraud risk rating against synthetic evaluation baseline • target: 0.95 • range: 0-1
Configured evaluation evidence
  • Benchmark: InsurTriage-Bench
  • Fraud Score Accuracy target: 0.95
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Aim · Required
  • OpenAI · Optional
  • Factory AI · Optional

Evaluation contract

  • approval_logic_pass · Weight 1
  • fraud_score_accuracy · Weight 1

Metrics

  • fraud_score_accuracy · Target 0.95

Benchmarks

  • InsurTriage-Bench
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON object with claim_id, policy_num, claimed_amount_sgd, diagnosis_code, hospital_bill_url

    Output: JSON object with claim_status, approved_payout_sgd, fraud_risk_score, escalation_reason

    Execution harness: Python

Evaluation tools

  • OpenAI Agents SDK · Autonomous Agent Framework

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Build an Automated Insurtech Claims Triaging System with OpenAI Agents SDK