Skip to main content
Back to challenges
Challenge

Deploy Real-time Anti-Scam Voice Intelligence with Pydantic AI and VAPI

Citizens face increasingly sophisticated voice scams targeting personal bank accounts. Build a real-time scam threat analysis agent using Pydantic AI for type-safe structured validation and VAPI for low-latency conversational audio transcript ingestion.

Workflow AutomationHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Implement a structured Pydantic AI agent integrated with VAPI to analyze live call transcripts and flag impersonation scams with typed risk metrics.

Delivery guide

How work is evaluated

Evaluation

Evaluates Pydantic AI agent speed and typed detection accuracy on scam transcript streams.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

pydantic_schema_validation

Ensures output strictly conforms to ThreatAssessment model schema

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

scam_detection_recall

Recall metric on identifying scam indicators in test transcripts • target: 0.95 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Build type-safe AI agents with Pydantic AI

  • Process streaming voice transcript events from VAPI audio webhooks

  • Extract coercive persuasion and government impersonation markers

  • Generate typed alert payloads for immediate bank transfer interventions

Resources and assets

Reference links and supporting material

Dataset notes

Synthetic audio transcripts of legitimate public agency calls and scam impersonation attempts.

Challenge setup

Evaluates Pydantic AI agent speed and typed detection accuracy on scam transcript streams.

Challenge input

JSON object representing transcript event frame with caller_id and transcript_segment

Pydantic AI

Guarantees strict schema outputs for safety critical decisions

VAPI

Provides real-time conversational streaming ingestion

Evaluated output

JSON object strict matching ThreatAssessment model (is_scam, threat_category, urgency_score)

Evaluation checks
  • Ensures output strictly conforms to ThreatAssessment model schema
  • Recall metric on identifying scam indicators in test transcripts • target: 0.95 • range: 0-1
Configured evaluation evidence
  • Benchmark: ScamDetect-2026
  • Scam Detection Recall target: 0.95
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Pydantic AI · Required
  • VAPI · Optional
  • LangWatch · Optional
Orchestration
  • Pydantic AI · Required

Evaluation contract

  • pydantic_schema_validation · Weight 1
  • scam_detection_recall · Weight 1

Metrics

  • scam_detection_recall · Target 0.95

Benchmarks

  • ScamDetect-2026
  • security

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON object representing transcript event frame with caller_id and transcript_segment

    Output: JSON object strict matching ThreatAssessment model (is_scam, threat_category, urgency_score)

    Execution harness: Python

Evaluation tools

  • Pydantic AI · Type-Safe Agent Framework
  • VAPI · Voice AI Platform

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Deploy Real-time Anti-Scam Voice Intelligence with Pydantic AI and VAPI