Skip to main content
Back to challenges
Challenge

Automated Financial Crime Investigation with Pydantic AI and Llama 3.3 70B

Inspired by recent advancements in AI-driven financial investigations, this challenge tasks you with building a structured agentic system to analyze transaction patterns. Using Pydantic AI, you will define type-safe schemas for investigative findings and implement dependency injection to handle real-time data ingestion. The system will leverage Llama 3.3 70B for decision-making logic, ensuring that every output adheres to strictly validated business rules. You will integrate the Braintrust platform to monitor and evaluate the decision-making process of your agents, ensuring high accuracy in compliance workflows. Additionally, you will implement a voice-activated reporting interface using Hamming for analysts to query findings verbally, while using Lakera for automated safety and policy guardrails to prevent data leakage during sensitive investigations.

Workflow AutomationHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Build a type-safe agentic investigator using Pydantic AI and Llama 3.3 70B for financial crime analysis and reporting.

Delivery guide

How work is evaluated

Evaluation

Evaluates the structural integrity and logic of the investigation reports.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

Schema Match

Output must match defined model

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

Confidence Score

Model confidence in reporting • target: 0.95 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Master Pydantic AI for creating type-safe LLM agents with strict validation patterns

  • Implement dependency injection in Pydantic AI to inject live transaction data streams

  • Design complex reasoning agents using Llama 3.3 70B for financial anomaly detection

  • Integrate Braintrust to log agent traces and evaluate performance against ground truth

  • Build a voice interface with Hamming for hands-free status updates and investigation summaries

  • Implement Lakera guardrails to sanitize input and prevent prompt injection in financial apps

Resources and assets

Reference links and supporting material

Dataset notes

Sample data for 1 tasks

Challenge setup

Evaluates the structural integrity and logic of the investigation reports.

Challenge input

JSON transaction log

Pydantic AI

Typed Python agent framework.

Llama 3.3 70B

LLM

Braintrust

Evaluation and tracing platform for AI apps.

Evaluated output

Validated Pydantic object

Evaluation checks
  • Output must match defined model
  • Model confidence in reporting • target: 0.95 • range: 0-1
Configured evaluation evidence
  • Confidence Score target: 0.95
  • 1 public reference case
Configured runtime
  • Python execution harness

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Pydantic AI · Required
  • Llama 3.3 70B · Optional
Observation
  • Braintrust · Optional
Reward / Eval
  • Braintrust · Optional
Orchestration
  • Pydantic AI · Required

Evaluation contract

  • Schema Match · Weight 1
  • Confidence Score · Weight 1

Metrics

  • Confidence Score · Target 0.95

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON transaction log

    Output: Validated Pydantic object

    Execution harness: Python

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Automated Financial Crime Investigation with Pydantic AI and Llama 3.3 70B