Public evaluation

Resilience Gap Triage

Evaluation focuses on the accuracy of the resilience score and the agent's ability to cross-reference text with visual data.

Evaluation type

task based

Challenge

Multimodal Flood Resilience Agent with Google ADK and Llama 3.3 70B

Difficulty

Advanced

Rigor

Unspecified

Open linked challenge

Evaluation overview

How the linked challenge is judged: tasks, benchmarks, and criteria count.

Tasks

Benchmarks

Criteria

Task templates

Inputs and expected outputs.

Task 1

Resilience Gap Triage

Given a property image and a PDF prevention plan, determine if the plan's requirements are met.

Input format

JSON containing image_url and pdf_text

Output format

JSON with 'score' (0-1), 'violations' (list), and 'evidence' (string)

Linked challenge

Multimodal Flood Resilience Agent with Google ADK and Llama 3.3 70B

Inspired by the recent devastating floods in West Virginia and the failure of unaddressed prevention plans, this challenge tasks developers with building a sophisticated Flood Risk Assessment Agent. Using the Google Agent Development Kit (ADK), you will create an agent that can process multimodal inputs: geospatial satellite imagery of property footprints and historical flood prevention documents. The agent must evaluate the 'resilience gap' between proposed prevention plans and current physical infrastructure. You will utilize Llama 3.3 70B (running on Vertex AI) to parse and summarize complex legislative and engineering reports, while the ADK manages the agentic loop and tool integration for fetching real-time weather alerts and elevation data. The core of the challenge is the implementation of a reasoning chain that can identify if a property's current mitigation measures (visualized via computer vision) align with the state-level prevention strategies. This involves high-stakes decision-making for insurers to adjust underwriting premiums based on 'active resilience' rather than just static FEMA zones.

Open challenge