Public evaluation

Report Validation

Evaluates the structural integrity and logic of the investigation reports.

Evaluation type

task based

Challenge

Automated Financial Crime Investigation with Pydantic AI and Llama 3.3 70B

Difficulty

Intermediate

Rigor

Unspecified

Open linked challenge

Evaluation overview

How the linked challenge is judged: tasks, benchmarks, and criteria count.

Tasks

Benchmarks

Criteria

Task templates

Inputs and expected outputs.

Task 1

Report Validation

Ensures the generated report matches the Pydantic schema

Input format

JSON transaction log

Output format

Validated Pydantic object

Linked challenge

Automated Financial Crime Investigation with Pydantic AI and Llama 3.3 70B

Inspired by recent advancements in AI-driven financial investigations, this challenge tasks you with building a structured agentic system to analyze transaction patterns. Using Pydantic AI, you will define type-safe schemas for investigative findings and implement dependency injection to handle real-time data ingestion. The system will leverage Llama 3.3 70B for decision-making logic, ensuring that every output adheres to strictly validated business rules. You will integrate the Braintrust platform to monitor and evaluate the decision-making process of your agents, ensuring high accuracy in compliance workflows. Additionally, you will implement a voice-activated reporting interface using Hamming for analysts to query findings verbally, while using Lakera for automated safety and policy guardrails to prevent data leakage during sensitive investigations.

Open challenge