Wildfire Property Risk Underwriting Multi-Agent System with AutoGen & Helicone
With property insurance exposure hitting $1.4T across top wildfire-prone states, insurance carriers and CRE asset managers need automated, multi-modal risk assessment systems that combine geospatial satellite telemetry, local vegetation density, fuel models, and policy compliance rules. Manual underwriting processes fail to scale against escalating climate events and rapidly shifting regulatory standards. In this challenge, you will build an automated wildfire property risk underwriting workflow using Microsoft AutoGen and Helicone for observability. You will construct a multi-agent network comprising a GIS Geospatial Analyst Agent, a Structural Hardening Compliance Agent, and an Actuarial Risk Scoring Agent. The system ingests building coordinates, satellite canopy rasters, structural attributes, and local zoning codes to dynamically output an underwriteability score, hazard mitigation recommendations, and adjusted premium pricing tier. Your deployment must track all agent interactions, tool calls, LLM latency, and cost telemetry via Helicone's observability SDK, enforcing structured JSON output for downstream claims systems.
What you are building
The core problem, expected build, and operating context for this challenge.
Build an automated geospatial wildfire risk assessment and insurance underwriting pipeline using Microsoft AutoGen multi-agent orchestration and Helicone observability.
How work is evaluated
Evaluates the multi-agent system's ability to ingest property coordinates, accurately process defensible space risk metrics, route queries via AutoGen, and record traces via Helicone.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
valid_schema_check
Ensures output conforms strictly to the property risk decision schema.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
high_risk_flag_check
Verifies that properties with wood shake roofs and >15% slope receive a hazard score >= 75.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
risk_score_accuracy
Mean absolute percentage error compared to ground truth actuarial hazard matrix. • target: 0.92 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
helicone_trace_coverage
Percentage of agent interactions with logged Helicone session headers. • target: 1 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Orchestrate a multi-agent property risk assessment team using Microsoft AutoGen
Integrate geospatial GIS overlays (vegetation index, slope, defensible space) into dynamic agent context
Implement real-time LLM caching, prompt tracking, and latency monitoring using Helicone SDK
Generate structured JSON risk mitigation reports and insurance premium multipliers
Reference links and supporting material
Synthetic and open-source geospatial risk dataset containing 200 California and Colorado commercial property profiles with roof types, surrounding canopy rasters, ground slope percentages, and historical wildfire exposure tags.
Challenge setup
Evaluates the multi-agent system's ability to ingest property coordinates, accurately process defensible space risk metrics, route queries via AutoGen, and record traces via Helicone.
Challenge input
JSON containing property_id, coordinates, slope_pct, vegetation_density_30ft, roof_type, and wall_construction
AutoGen
Required multi-agent framework for orchestrating conversational reasoning between specialized risk agents.
Helicone
Provides proxy-level LLM caching, trace monitoring, latency tracking, and request tagging.
Evaluated output
JSON containing underwriting_decision, hazard_score, risk_multiplier, recommended_mitigations, and helicone_trace_id
- Ensures output conforms strictly to the property risk decision schema.
- Verifies that properties with wood shake roofs and >15% slope receive a hazard score >= 75.
- Mean absolute percentage error compared to ground truth actuarial hazard matrix. • target: 0.92 • range: 0-1
- Benchmark: ClimateRisk-Underwriting-Bench
- Risk Score Accuracy target: 0.92
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- AutoGen · Optional
- Deepgram · Optional
- Helicone · Required
Evaluation contract
- valid_schema_check · Weight 1
- high_risk_flag_check · Weight 1
- risk_score_accuracy · Weight 1
- helicone_trace_coverage · Weight 1
Metrics
- risk_score_accuracy · Target 0.92
- helicone_trace_coverage · Target 1
Benchmarks
- ClimateRisk-Underwriting-Bench
- energy
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON containing property_id, coordinates, slope_pct, vegetation_density_30ft, roof_type, and wall_construction
Output: JSON containing underwriting_decision, hazard_score, risk_multiplier, recommended_mitigations, and helicone_trace_id
Execution harness: Python
Evaluation tools
- AutoGen · Agent Framework
- Helicone · AI Workflow Automation / Observability
Recipe freeze marker
No recipe freeze time is recorded.