Skip to main content
Back to challenges
Challenge

CrewAI Multi-Agent GAIP Insurance Risk Underwriter

Orchestrate a team of CrewAI agents with distinct roles (risk assessor, policy analyst, auditor) to process commercial insurance claims and calculate risk scores. Achieve zero policy hallucination errors and automated audit logging.

Workflow AutomationHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Design a collaborative multi-agent system using CrewAI with defined roles and delegated tasks for underwriting regional commercial insurance.

Delivery guide

How work is evaluated

Evaluation

Evaluates multi-agent CrewAI task completion accuracy and calculated payout precision.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

Math Calculation Correctness

Verifies payout calculation accurately factors in deductibles

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

payout_accuracy

Accuracy of payout recommendation • target: 1 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Configure CrewAI agents with distinct backstories, goals, and role capabilities

  • Implement sequential and hierarchical agent delegation processes

  • Enforce compliance and coverage boundaries for regional commercial insurance

  • Generate structured claim settlement reports with full rationale

Resources and assets

Reference links and supporting material

Dataset notes

Synthetic commercial property and casualty claim files tailored to Asian insurance markets.

Challenge setup

Evaluates multi-agent CrewAI task completion accuracy and calculated payout precision.

Challenge input

JSON containing policy terms, claim loss details, and surveyor report

CrewAI

Framework for orchestrating role-playing collaborative agents

Evaluated output

JSON containing approved_amount, deduction_reasons, and crew_task_status

Evaluation checks
  • Verifies payout calculation accurately factors in deductibles
  • Accuracy of payout recommendation • target: 1 • range: 0-1
Configured evaluation evidence
  • Benchmark: InsurBench
  • Payout Accuracy target: 1
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • CrewAI · Required
  • crewAI · Optional
  • Aim · Optional

Evaluation contract

  • Math Calculation Correctness · Weight 1
  • payout_accuracy · Weight 1

Metrics

  • payout_accuracy · Target 1

Benchmarks

  • InsurBench
  • security

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON containing policy terms, claim loss details, and surveyor report

    Output: JSON containing approved_amount, deduction_reasons, and crew_task_status

    Execution harness: Python

Evaluation tools

  • CrewAI · Agent Framework

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about CrewAI Multi-Agent GAIP Insurance Risk Underwriter