Type-Safe Climate Hazard Scoring Engine with Pydantic AI
Climate risk assessments require strict schema validation to prevent downstream rating errors. You will build a type-safe climate risk evaluation engine using Pydantic AI that ingests wildfire, flood, and extreme heat metrics for commercial assets. The system must enforce strong typing and output validated risk records with 100% schema compliance.
What you are building
The core problem, expected build, and operating context for this challenge.
Build a type-safe, validated geospatial climate hazard scoring engine using Pydantic AI.
How work is evaluated
Evaluates schema validation success rate and numeric accuracy of calculated combined climate hazard scores.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
pydantic_validation_pass
Validates output adheres 100% to Pydantic schema without validation errors.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
schema_compliance_rate
Percentage of test outputs passing Pydantic validation • target: 1 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Build strongly typed agents using Pydantic AI schema definitions.
Validate complex climate indicator inputs against custom Pydantic validators.
Produce risk outputs guaranteed to conform to structural schema types.
Reference links and supporting material
Dataset of 500 spatial coordinates across North America with associated flood, storm surge, and wildfire risk attributes.
Challenge setup
Evaluates schema validation success rate and numeric accuracy of calculated combined climate hazard scores.
Challenge input
JSON object with latitude, longitude, flood_zone, wildfire_score, structural_value
Pydantic AI
Assigned framework for type-safe Python agent creation.
Evaluated output
JSON object conforming strictly to AssetRiskScore schema (composite_score, primary_hazard, error_bounds)
- Validates output adheres 100% to Pydantic schema without validation errors.
- Percentage of test outputs passing Pydantic validation • target: 1 • range: 0-1
- Benchmark: Type-Safe LLM Output Benchmark
- Schema Compliance Rate target: 100%
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- Pydantic AI · Required
- Llama 3.1 405B · Optional
- Together AI · Optional
- Together AI · Optional
- Pydantic AI · Required
Evaluation contract
- pydantic_validation_pass · Weight 1
- schema_compliance_rate · Weight 1
Metrics
- schema_compliance_rate · Target 1
Benchmarks
- Type-Safe LLM Output Benchmark
- general
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON object with latitude, longitude, flood_zone, wildfire_score, structural_value
Output: JSON object conforming strictly to AssetRiskScore schema (composite_score, primary_hazard, error_bounds)
Execution harness: Python
Evaluation tools
- Pydantic AI · Type-Safe Agent Framework
Recipe freeze marker
No recipe freeze time is recorded.