Claude Agents SDK Voice Assistant for NYC FARE Act Compliance
Create an automated tenant agent using Claude Agents SDK and VAPI to conduct audio-driven intake and screen NYC rental disclosures for compliance under the FARE Act. The workflow validates fee allocations and outputs parsed compliance audit reports within 500ms.
What you are building
The core problem, expected build, and operating context for this challenge.
This project integrates Anthropic's Claude Agents SDK with VAPI audio tools to parse real-time tenant interaction streams and flag illegal broker fee transfers under NYC's FARE Act.
How work is evaluated
Evaluates accuracy of lease fee structure classifications and audio transcript extraction.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
Violation Flagging Accuracy
Ensures true positive violations are correctly flagged without false positives
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Classification F1 Score
F1 score across multi-class fee violation categories • target: 0.95 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Implement an agent using the Claude Agents SDK with dynamic tool calls
Configure VAPI voice integration for real-time audio transcript processing
Analyze lease disclosure documents against FARE Act regulatory constraints
Generate structured compliance audit reports with violation classifications
Reference links and supporting material
100 NYC rental transcripts and paired contract clauses representing legal and illegal fee allocations.
Challenge setup
Evaluates accuracy of lease fee structure classifications and audio transcript extraction.
Challenge input
JSON containing transcript text and lease agreement agreement_text
Claude Agents SDK
Provides structured tool interaction and agent reasoning.
VAPI
Handles real-time audio intake and speech-to-text pipeline.
Evaluated output
JSON with contains_violation (boolean), fee_amount, violation_type, and reason
- Ensures true positive violations are correctly flagged without false positives
- F1 score across multi-class fee violation categories • target: 0.95 • range: 0-1
- Benchmark: LegalComplianceBench-RealEstate
- Classification F1 Score target: 0.95
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- VAPI · Required
- Distyl · Optional
- Kira · Optional
Evaluation contract
- Violation Flagging Accuracy · Weight 1
- Classification F1 Score · Weight 1
Metrics
- Classification F1 Score · Target 0.95
Benchmarks
- LegalComplianceBench-RealEstate
- general
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON containing transcript text and lease agreement agreement_text
Output: JSON with contains_violation (boolean), fee_amount, violation_type, and reason
Execution harness: Python
Evaluation tools
- Claude Agents SDK · Core Framework
- VAPI · Voice Gateway
Recipe freeze marker
No recipe freeze time is recorded.