Skip to main content
Back to challenges
Challenge

Automate Insurance Underwriting Compliance and Risk Audits with Mastra AI

Insurers must ensure underwriting algorithms comply with regulatory fairness and capital adequacy standards. Build a Mastra AI workflow that executes multi-turn compliance audits on parametric insurance models to flag bias and regulatory non-compliance.

CybersecurityHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Develop an automated audit workflow using Mastra AI to evaluate insurance underwriting logic against regulatory standards.

Delivery guide

How work is evaluated

Evaluation

Evaluates bias detection and regulatory compliance parsing in underwriting logic.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

Demographic Proxy Detection Test

Heavy weighting on demographic proxies must result in is_compliant == False.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

Compliance Audit Accuracy

Percentage of regulatory infractions identified. • target: 0.96 • range: 0.9-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Construct Mastra AI agent workflows that evaluate parametric insurance underwriting rules.

  • Audit model decision distributions for protected class demographic bias.

  • Validate solvency capital requirements against statutory formulas.

  • Output audit reports matching regulatory disclosure standards.

Resources and assets

Reference links and supporting material

Dataset notes

Synthetic insurance underwriting rule set dataset with embedded regulatory non-compliance traps.

Challenge setup

Evaluates bias detection and regulatory compliance parsing in underwriting logic.

Challenge input

JSON containing underwriting_rules, variable_weights, geographical_proxies

Mastra AI

Orchestrates step-by-step regulatory audit workflows in TypeScript.

Evaluated output

JSON with is_compliant (boolean), bias_flags, non_compliance_severity

Evaluation checks
  • Heavy weighting on demographic proxies must result in is_compliant == False.
  • Percentage of regulatory infractions identified. • target: 0.96 • range: 0.9-1
Configured evaluation evidence
  • Benchmark: InsuranceAuditBench
  • Compliance Audit Accuracy target: 0.96
  • 1 public reference case
Configured runtime
  • TypeScript execution harness
  • javascript sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Mastra AI · Required
Orchestration
  • Mastra AI · Required

Evaluation contract

  • Demographic Proxy Detection Test · Weight 1
  • Compliance Audit Accuracy · Weight 1

Metrics

  • Compliance Audit Accuracy · Target 0.96

Benchmarks

  • InsuranceAuditBench
  • security

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON containing underwriting_rules, variable_weights, geographical_proxies

    Output: JSON with is_compliant (boolean), bias_flags, non_compliance_severity

    Execution harness: TypeScript

Evaluation tools

  • Mastra AI · Agent Framework

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Automate Insurance Underwriting Compliance and Risk Audits with Mastra AI