Skip to main content
Loading navigation
Back to challenges

Build a Real-Time Deepfake Voice & Video Verification Pipeline with LlamaIndex

High-value wire transfers are increasingly targeted by AI voice clone and video deepfake impersonation attacks. Build a multi-modal verification agent using LlamaIndex and spectrographic analysis tools to analyze caller audio streams and intercept deepfake authorization attempts with < 1% false acceptance.

CybersecurityHosted by Vera
Sign in free to start

What you are building

Utilize LlamaIndex audio-visual multimodal connectors and spectrographic verification tools to defend high-value wire transfers against deepfake attacks.

How work is evaluated

Evaluation

Evaluates False Acceptance Rate (FAR) and synthetic audio detection precision across real and AI-cloned speech samples.

Datasets

Public datasets, plus the private datasets you have access to.

How submissions are scored

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

low_far_test

False Acceptance Rate must remain under 1.0%.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

detection_accuracy

Overall accuracy in classifying synthetic vs authentic speech. • target: 0.98 • range: 0.9-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

  • Implement multi-modal data indexes in LlamaIndex for audio spectral feature parsing

  • Detect synthetic voice artifacts (phase discontinuity, unnatural prosody)

  • Build dynamic step-up authentication triggers for flagged high-value authorizations

  • Achieve sub-1% False Acceptance Rate (FAR) on zero-day synthetic audio samples

Resources

Dataset notes

Dataset of 500 authentic voice samples and matching AI voice clones generated via state-of-the-art TTS models.

Challenge setup

Evaluates False Acceptance Rate (FAR) and synthetic audio detection precision across real and AI-cloned speech samples.

Challenge input

WAV audio file payload or spectral feature array of call authorization

LlamaIndex

Supports multimodal feature indexing and rapid query execution.

Evaluated output

JSON authentic/synthetic classification, confidence score, and liveness metric

Evaluation checks
  • False Acceptance Rate must remain under 1.0%.
  • Overall accuracy in classifying synthetic vs authentic speech. • target: 0.98 • range: 0.9-1
Configured evaluation evidence
  • Benchmark: ASVspoof Deepfake Benchmark
  • Detection Accuracy target: 0.98
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Shows the first task, up to three tools, and up to three evidence items.

View technical recipe

Configured tools

Action Space
  • Llama Index · Required
  • LlamaIndex · Optional
  • Llama 3.2 · Optional

Evaluation contract

  • low_far_test · Weight 1
  • detection_accuracy · Weight 1

Metrics

  • detection_accuracy · Target 0.98

Benchmarks

  • ASVspoof Deepfake Benchmark
  • security

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: WAV audio file payload or spectral feature array of call authorization

    Output: JSON authentic/synthetic classification, confidence score, and liveness metric

    Execution harness: Python

Evaluation tools

  • LlamaIndex · Multimodal Indexing Framework

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Build a Real-Time Deepfake Voice & Video Verification Pipeline with LlamaIndex