Back to challenges
Challenge

Automated Financial Crime Investigation with Pydantic AI and Llama 3.3 70B

Inspired by recent advancements in AI-driven financial investigations, this challenge tasks you with building a structured agentic system to analyze transaction patterns. Using Pydantic AI, you will define type-safe schemas for investigative findings and implement dependency injection to handle real-time data ingestion. The system will leverage Llama 3.3 70B for decision-making logic, ensuring that every output adheres to strictly validated business rules. You will integrate the Braintrust platform to monitor and evaluate the decision-making process of your agents, ensuring high accuracy in compliance workflows. Additionally, you will implement a voice-activated reporting interface using Hamming for analysts to query findings verbally, while using Lakera for automated safety and policy guardrails to prevent data leakage during sensitive investigations.

Workflow AutomationHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Inspired by recent advancements in AI-driven financial investigations, this challenge tasks you with building a structured agentic system to analyze transaction patterns. Using Pydantic AI, you will define type-safe schemas for investigative findings and implement dependency injection to handle real-time data ingestion. The system will leverage Llama 3.3 70B for decision-making logic, ensuring that every output adheres to strictly validated business rules. You will integrate the Braintrust platform to monitor and evaluate the decision-making process of your agents, ensuring high accuracy in compliance workflows. Additionally, you will implement a voice-activated reporting interface using Hamming for analysts to query findings verbally, while using Lakera for automated safety and policy guardrails to prevent data leakage during sensitive investigations.

Datasets

Shared data for this challenge

Review public datasets and any private uploads tied to your build.

Loading datasets...
Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

Schema Match

Output must match defined model

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

Confidence Score

Model confidence in reporting • target: 0.95 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Master Pydantic AI for creating type-safe LLM agents with strict validation patterns

  • Implement dependency injection in Pydantic AI to inject live transaction data streams

  • Design complex reasoning agents using Llama 3.3 70B for financial anomaly detection

  • Integrate Braintrust to log agent traces and evaluate performance against ground truth

  • Build a voice interface with Hamming for hands-free status updates and investigation summaries

  • Implement Lakera guardrails to sanitize input and prevent prompt injection in financial apps

How this agent runs

Evaluates the structural integrity and logic of the investigation reports.

Preview configuration

Challenge input

JSON transaction log

Pydantic AI

Typed Python agent framework.

Llama 3.3 70B

LLM

Braintrust

Evaluation and tracing platform for AI apps.

Evaluated output

Validated Pydantic object

Checks for
  • Output must match defined model
  • Model confidence in reporting • target: 0.95 • range: 0-1
Proof of success
  • Confidence Score target: 0.95
  • 1 public reference case
Runtime evidence
  • Python execution harness
View technical recipe

Configured tools

Action Space
  • Pydantic AI · Required
  • Llama 3.3 70B · Optional
Observation
  • Braintrust · Optional
Reward / Eval
  • Braintrust · Optional
Orchestration
  • Pydantic AI · Required

Evaluation contract

  • Schema Match · Weight 1
  • Confidence Score · Weight 1

Recipe state

This is a preview. The configuration can change before the evaluation recipe is locked.

Run this agent on your dataset and AI stack

Bring your dataset, model providers, and success criteria. We will scope the right managed run for your team.

Scope a managed run
Start from your terminal
$npx -y @versalist/cli start automated-financial-crime-investigation-with-pydantic-ai-and-llama-3-3-70b

[ok] Wrote CHALLENGE.md

[ok] Wrote .versalist.json

[ok] Wrote eval/examples.json

Requires VERSALIST_API_KEY. Works with any MCP-aware editor.

Docs
Manage API keys
Explore

Find another challenge

Jump to a random challenge when you want a fresh benchmark or a different problem space.

Useful when you want to pressure-test your workflow on a new dataset, new constraints, or a new evaluation rubric.

Frequently Asked Questions about Automated Financial Crime Investigation with Pydantic AI and Llama 3.3 70B