Skip to main content
Back to challenges
Challenge

Stateful Industry Loss Warranty Monitor with Google ADK and Letta

Industry Loss Warranties (ILWs) require verifiable index tracking across long-running event windows. You will develop a stateful ILW parametric monitor using Google Agent Development Kit (ADK) and Letta memory blocks to maintain property catastrophe loss tallying. The system must maintain 100% loss index tracking state accuracy over multi-week event simulation periods.

Machine LearningHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Build a stateful ILW loss aggregation agent using Google ADK and Letta persistent memory systems.

Delivery guide

How work is evaluated

Evaluation

Evaluates cumulative loss accuracy and trigger verification across persistent execution cycles.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

exact_tally_match

Validates exact match on cumulative loss calculation.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

state_retention_accuracy

Accuracy of state variable retention across sequential runs • target: 1 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Construct persistent agent memory structures using Letta blocks within Google ADK.

  • Process cumulative industry loss updates from PCS/PERILS index providers.

  • Verify trigger thresholds for parametric reinsurance contract settlement.

Resources and assets

Reference links and supporting material

Dataset notes

Time-series dataset of simulated regional storm loss estimates and ILW trigger threshold configurations.

Challenge setup

Evaluates cumulative loss accuracy and trigger verification across persistent execution cycles.

Challenge input

JSON object with updates (array of loss events), contract_trigger_threshold

Google ADK

Assigned agent framework for Gemini integration.

Letta

Assigned persistent state and memory block management system.

Evaluated output

JSON object with current_cumulative_loss, trigger_activated (boolean), memory_status

Evaluation checks
  • Validates exact match on cumulative loss calculation.
  • Accuracy of state variable retention across sequential runs • target: 1 • range: 0-1
Configured evaluation evidence
  • Benchmark: Stateful Agent Memory Retention Benchmark
  • State Retention Accuracy target: 1
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Google · Required
  • Letta · Optional
  • BentoML · Optional

Evaluation contract

  • exact_tally_match · Weight 1
  • state_retention_accuracy · Weight 1

Metrics

  • state_retention_accuracy · Target 1

Benchmarks

  • Stateful Agent Memory Retention Benchmark
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON object with updates (array of loss events), contract_trigger_threshold

    Output: JSON object with current_cumulative_loss, trigger_activated (boolean), memory_status

    Execution harness: Python

Evaluation tools

  • Google ADK · Agent Framework
  • Letta · Agent Memory System

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Stateful Industry Loss Warranty Monitor with Google ADK and Letta