Skip to main content
Back to challenges
Challenge

AutoGen Environmental Liability Hazard Scoring & Remediation Agent System

Environmental risk underwriting requires evaluating complex chemical hazard disclosures and soil contamination reports. Build an AutoGen multi-agent system to analyze environmental site assessment documents (Phase I/II ESA) and compute site remediation liability risk scores.

Machine LearningHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Build an AutoGen multi-agent system to automate environmental site assessment audits and hazard risk scoring.

Delivery guide

How work is evaluated

Evaluation

Evaluates REC extraction precision and estimated cleanup cost computation accuracy.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

recs_non_empty

Ensures RECs are identified when contaminants exceed threshold

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

rec_extraction_f1

F1 score of identified environmental concerns • target: 0.9 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Configure AutoGen agents (ESA Analyst, EPA Compliance Officer, Environmental Underwriter)

  • Parse Phase I and Phase II Environmental Site Assessment PDFs for Recognized Environmental Conditions (RECs)

  • Calculate estimated soil/groundwater cleanup liability costs

  • Generate environmental policy rate recommendation reports

Resources and assets

Reference links and supporting material

Dataset notes

Environmental Site Assessment summary documents and historical EPA chemical threshold data.

Challenge setup

Evaluates REC extraction precision and estimated cleanup cost computation accuracy.

Challenge input

JSON containing site_address, historical_use, and soil_sample_ppm details.

AutoGen

Agent collaboration and conversational document auditing

Evaluated output

JSON with recs_found list, estimated_remediation_cost_usd, and risk_tier.

Evaluation checks
  • Ensures RECs are identified when contaminants exceed threshold
  • F1 score of identified environmental concerns • target: 0.9 • range: 0-1
Configured evaluation evidence
  • Benchmark: EnvironmentalRiskBench
  • Rec Extraction F1 target: 0.9
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • AutoGen · Required
  • Cohere Platform · Optional
Policy Serving
  • Grok 4 Heavy · Optional

Evaluation contract

  • recs_non_empty · Weight 1
  • rec_extraction_f1 · Weight 1

Metrics

  • rec_extraction_f1 · Target 0.9

Benchmarks

  • EnvironmentalRiskBench
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON containing site_address, historical_use, and soil_sample_ppm details.

    Output: JSON with recs_found list, estimated_remediation_cost_usd, and risk_tier.

    Execution harness: Python

Evaluation tools

  • AutoGen · Multi-Agent Framework

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about AutoGen Environmental Liability Hazard Scoring & Remediation Agent System