Skip to main content
Loading navigation
Back to challenges

AutoGen Environmental Liability Hazard Scoring & Remediation Agent System

Environmental risk underwriting requires evaluating complex chemical hazard disclosures and soil contamination reports. Build an AutoGen multi-agent system to analyze environmental site assessment documents (Phase I/II ESA) and compute site remediation liability risk scores.

Machine LearningHosted by Vera
Sign in free to start

What you are building

Build an AutoGen multi-agent system to automate environmental site assessment audits and hazard risk scoring.

How work is evaluated

Evaluation

Evaluates REC extraction precision and estimated cleanup cost computation accuracy.

Datasets

Public datasets, plus the private datasets you have access to.

How submissions are scored

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

recs_non_empty

Ensures RECs are identified when contaminants exceed threshold

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

rec_extraction_f1

F1 score of identified environmental concerns • target: 0.9 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

  • Configure AutoGen agents (ESA Analyst, EPA Compliance Officer, Environmental Underwriter)

  • Parse Phase I and Phase II Environmental Site Assessment PDFs for Recognized Environmental Conditions (RECs)

  • Calculate estimated soil/groundwater cleanup liability costs

  • Generate environmental policy rate recommendation reports

Resources

Dataset notes

Environmental Site Assessment summary documents and historical EPA chemical threshold data.

Challenge setup

Evaluates REC extraction precision and estimated cleanup cost computation accuracy.

Challenge input

JSON containing site_address, historical_use, and soil_sample_ppm details.

AutoGen

Agent collaboration and conversational document auditing

Evaluated output

JSON with recs_found list, estimated_remediation_cost_usd, and risk_tier.

Evaluation checks
  • Ensures RECs are identified when contaminants exceed threshold
  • F1 score of identified environmental concerns • target: 0.9 • range: 0-1
Configured evaluation evidence
  • Benchmark: EnvironmentalRiskBench
  • Rec Extraction F1 target: 0.9
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Shows the first task, up to three tools, and up to three evidence items.

View technical recipe

Configured tools

Action Space
  • AutoGen · Required
  • Cohere Platform · Optional
Policy Serving
  • Grok 4 Heavy · Optional

Evaluation contract

  • recs_non_empty · Weight 1
  • rec_extraction_f1 · Weight 1

Metrics

  • rec_extraction_f1 · Target 0.9

Benchmarks

  • EnvironmentalRiskBench
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON containing site_address, historical_use, and soil_sample_ppm details.

    Output: JSON with recs_found list, estimated_remediation_cost_usd, and risk_tier.

    Execution harness: Python

Evaluation tools

  • AutoGen · Multi-Agent Framework

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about AutoGen Environmental Liability Hazard Scoring & Remediation Agent System