Back to challenges
Challenge

Build an Accessible Public Service Application Agent with Claude Agents SDK

Citizens with accessibility needs encounter digital barriers when applying for municipal assistance programs. Implement a Claude Agents SDK workflow using computer use and extended thinking capabilities to automatically navigate public forms, identify accessibility gaps, and complete applications with 95% task completion rate.

Workflow AutomationHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Build an accessible public service agent with Claude Agents SDK utilizing extended thinking to assist disabled citizens with public grant forms.

Delivery guide

How work is evaluated

Evaluation

Evaluates the agent's ability to complete public application forms while detecting screen reader incompatibilities.

Datasets

Shared data for this challenge

Review public datasets and any private uploads tied to your build.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

form_completed_test

Verifies target form was successfully submitted without critical error.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

completion_rate

Percentage of dynamic form scenarios successfully completed. • target: 0.95 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Master Claude Agents SDK extended thinking and tool integration patterns

  • Implement computer use tools to inspect and fill accessible web form elements

  • Identify WCAG compliance gaps in digital public service applications

  • Build resilient form-filling agents that handle multi-step citizen applications

Resources and assets

Reference links and supporting material

Dataset notes

Synthetic DOM accessibility dataset comprising 25 digital government grant application forms.

How this agent runs

Evaluates the agent's ability to complete public application forms while detecting screen reader incompatibilities.

Challenge input

HTML form snippet or DOM accessibility tree representation

Claude Agents SDK

Provides state-of-the-art computer use and extended thinking agent capabilities.

Evaluated output

JSON containing completed form state and flagged accessibility violations

Checks for
  • Verifies target form was successfully submitted without critical error.
  • Percentage of dynamic form scenarios successfully completed. • target: 0.95 • range: 0-1
Proof of success
  • Benchmark: WebArenaAccessibility
  • Completion Rate target: 95%
  • 1 public reference case
Runtime evidence
  • Python execution harness
  • Python sandbox (unavailable on Versalist)
View technical recipe

Configured tools

Action Space
  • Confident AI · Required
  • Chattforge (Chatforge) · Optional
  • Haystack · Optional

Evaluation contract

  • form_completed_test · Weight 1
  • completion_rate · Weight 1

Recipe state

This is a preview. The configuration can change before the evaluation recipe is locked.

Frequently Asked Questions about Build an Accessible Public Service Application Agent with Claude Agents SDK