Skip to main content
Back to challenges
Challenge

Data Breach Loss Exposure Simulator using Mastra AI and Novita AI

Massive cyber breach settlements demonstrate the critical need for precise quantified exposure modeling for enterprise insurance. You will implement a real-time loss simulation agent using Mastra AI and Novita AI inference pipelines to estimate potential settlement liabilities from breach characteristics. Achieve a Mean Absolute Percentage Error under 8% on historical breach settlements.

Machine LearningHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Build a loss exposure agent workflow using Mastra AI integrated with Novita AI model endpoints for high-throughput loss simulation.

Delivery guide

How work is evaluated

Evaluation

Evaluate loss estimate accuracy against real and synthetic breach settlement metrics.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

valid_variance_bounds

Calculated VaR is strictly greater than estimated loss mean.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

mape

Mean Absolute Percentage Error on loss estimation • target: 0.08 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Construct TypeScript agent workflows using Mastra AI

  • Connect Novita AI inference endpoints for rapid LLM calculations

  • Process complex corporate data breach features (PII counts, industry sector, exfiltration methods)

  • Evaluate statistical distribution fit for simulated loss outcomes

Resources and assets

Reference links and supporting material

Dataset notes

Historical data breach court settlement records combined with synthetic exposure profiles.

Challenge setup

Evaluate loss estimate accuracy against real and synthetic breach settlement metrics.

Challenge input

JSON with breach parameters (records exposed, regulatory fine history)

Mastra AI

TypeScript agent workflow engine with integrated step tracking.

Novita AI

High-throughput execution of open LLM inference calls.

Evaluated output

JSON with estimated loss mean, VaR 95%, and confidence bounds

Evaluation checks
  • Calculated VaR is strictly greater than estimated loss mean.
  • Mean Absolute Percentage Error on loss estimation • target: 0.08 • range: 0-1
Configured evaluation evidence
  • Benchmark: CyberLossBench-2026
  • Mape target: 0.08
  • 1 public reference case
Configured runtime
  • JavaScript execution harness
  • javascript sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Novita AI · Required
  • Mastra AI · Optional
  • LMDeploy · Optional
Orchestration
  • Mastra AI · Optional

Evaluation contract

  • valid_variance_bounds · Weight 1
  • mape · Weight 1

Metrics

  • mape · Target 0.08

Benchmarks

  • CyberLossBench-2026
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON with breach parameters (records exposed, regulatory fine history)

    Output: JSON with estimated loss mean, VaR 95%, and confidence bounds

    Execution harness: JavaScript

Evaluation tools

  • Mastra AI · Agent Framework
  • Novita AI · Inference Host

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Data Breach Loss Exposure Simulator using Mastra AI and Novita AI