Back to challenges
Challenge

Data Breach Loss Exposure Simulator using Mastra AI and Novita AI

Massive cyber breach settlements demonstrate the critical need for precise quantified exposure modeling for enterprise insurance. You will implement a real-time loss simulation agent using Mastra AI and Novita AI inference pipelines to estimate potential settlement liabilities from breach characteristics. Achieve a Mean Absolute Percentage Error under 8% on historical breach settlements.

Machine LearningHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Build a loss exposure agent workflow using Mastra AI integrated with Novita AI model endpoints for high-throughput loss simulation.

Delivery guide

How work is evaluated

Evaluation

Evaluate loss estimate accuracy against real and synthetic breach settlement metrics.

Datasets

Shared data for this challenge

Review public datasets and any private uploads tied to your build.

Loading datasets...
Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

valid_variance_bounds

Calculated VaR is strictly greater than estimated loss mean.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

mape

Mean Absolute Percentage Error on loss estimation • target: 0.08 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Construct TypeScript agent workflows using Mastra AI

  • Connect Novita AI inference endpoints for rapid LLM calculations

  • Process complex corporate data breach features (PII counts, industry sector, exfiltration methods)

  • Evaluate statistical distribution fit for simulated loss outcomes

Resources and assets

Reference links and supporting material

Dataset notes

Historical data breach court settlement records combined with synthetic exposure profiles.

How this agent runs

Evaluate loss estimate accuracy against real and synthetic breach settlement metrics.

Challenge input

JSON with breach parameters (records exposed, regulatory fine history)

Mastra AI

TypeScript agent workflow engine with integrated step tracking.

Novita AI

High-throughput execution of open LLM inference calls.

Evaluated output

JSON with estimated loss mean, VaR 95%, and confidence bounds

Checks for
  • Calculated VaR is strictly greater than estimated loss mean.
  • Mean Absolute Percentage Error on loss estimation • target: 0.08 • range: 0-1
Proof of success
  • Benchmark: CyberLossBench-2026
  • Mape target: 0.08
  • 1 public reference case
Runtime evidence
  • JavaScript execution harness
  • javascript sandbox (unavailable on Versalist)
View technical recipe

Configured tools

Action Space
  • Novita AI · Required
  • Mastra AI · Optional
  • LMDeploy · Optional
Orchestration
  • Mastra AI · Optional

Evaluation contract

  • valid_variance_bounds · Weight 1
  • mape · Weight 1

Recipe state

This is a preview. The configuration can change before the evaluation recipe is locked.

Frequently Asked Questions about Data Breach Loss Exposure Simulator using Mastra AI and Novita AI