Skip to main content
Back to challenges
Challenge

Develop a Cyber Data Breach Exposure Estimator Agent with Mastra AI Framework

Multi-million dollar cyber loss settlements like the recent $117.5M breach case require accurate loss modeling based on compromised record types. Build an autonomous breach estimation pipeline using Mastra AI in TypeScript with built-in state memory and RAG workflows. Achieve loss estimation within 15% of historical regulatory benchmark settlements.

Machine LearningHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Construct a TypeScript Mastra AI workflow that uses local vector memory and dynamic tools to calculate cyber breach loss exposure.

Delivery guide

How work is evaluated

Evaluation

Evaluates estimated cyber breach monetary reserve outputs against known benchmark settlement values.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

valid_confidence_interval

Confidence interval logically bounds the predicted estimate.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

mean_absolute_percentage_error

MAPE of total estimated monetary loss vs ground truth settlement • target: 0.15 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Implement autonomous agent workflows using Mastra AI framework.

  • Utilize Mastra's built-in memory and RAG capabilities.

  • Model cyber liability per record based on compromised PII/PHI categories.

  • Generate structured risk mitigation and loss reserve reports.

Resources and assets

Reference links and supporting material

Dataset notes

Dataset of 80 historical cyber data breach settlements with record counts and industry verticals.

Challenge setup

Evaluates estimated cyber breach monetary reserve outputs against known benchmark settlement values.

Challenge input

JSON containing record_count, record_types (PII, SSN, Health), and jurisdiction

Mastra AI

TypeScript framework with native memory and workflow orchestration.

Evaluated output

JSON with estimated_total_loss_usd, loss_per_record, and confidence_interval

Evaluation checks
  • Confidence interval logically bounds the predicted estimate.
  • MAPE of total estimated monetary loss vs ground truth settlement • target: 0.15 • range: 0-1
Configured evaluation evidence
  • Benchmark: CyberLossBench
  • Mean Absolute Percentage Error target: 0.15
  • 1 public reference case
Configured runtime
  • JavaScript execution harness
  • javascript sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Mastra AI · Required
  • Langbase · Optional
  • IBM watsonx.ai · Optional
Orchestration
  • Mastra AI · Required

Evaluation contract

  • valid_confidence_interval · Weight 1
  • mean_absolute_percentage_error · Weight 1

Metrics

  • mean_absolute_percentage_error · Target 0.15

Benchmarks

  • CyberLossBench
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON containing record_count, record_types (PII, SSN, Health), and jurisdiction

    Output: JSON with estimated_total_loss_usd, loss_per_record, and confidence_interval

    Execution harness: JavaScript

Evaluation tools

  • Mastra AI · Agent Framework

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Develop a Cyber Data Breach Exposure Estimator Agent with Mastra AI Framework