Build a Multi-Agent Fraud Investigation System using AutoGen and Claude 3.5 Sonnet
Inspired by the recent Memorandum of Understanding between the Monetary Authority of Singapore (MAS) and the Bank of Thailand regarding digital fraud protection, this challenge tasks you with building a collaborative multi-agent system to automate financial fraud investigations. You will leverage Microsoft's AutoGen framework to orchestrate a conversation between specialized agents: a Transaction Analyst, a Network Relationship Agent (Graph Specialist), and a Compliance Officer. Your goal is to develop a system that can consume streaming transaction logs and suspicious activity reports (SARs), perform entity resolution across cross-border payment rails, and generate a structured investigation report. The Transaction Analyst will identify anomalies, the Network Relationship Agent will query a graph database to find hidden clusters (ring-fencing), and the Compliance Officer agent will cross-reference findings against the MAS and Bank of Thailand's shared fraud intelligence frameworks to determine if the activity requires a formal regulatory filing.
What you are building
The core problem, expected build, and operating context for this challenge.
Inspired by the recent Memorandum of Understanding between the Monetary Authority of Singapore (MAS) and the Bank of Thailand regarding digital fraud protection, this challenge tasks you with building a collaborative multi-agent system to automate financial fraud investigations. You will leverage Microsoft's AutoGen framework to orchestrate a conversation between specialized agents: a Transaction Analyst, a Network Relationship Agent (Graph Specialist), and a Compliance Officer. Your goal is to develop a system that can consume streaming transaction logs and suspicious activity reports (SARs), perform entity resolution across cross-border payment rails, and generate a structured investigation report. The Transaction Analyst will identify anomalies, the Network Relationship Agent will query a graph database to find hidden clusters (ring-fencing), and the Compliance Officer agent will cross-reference findings against the MAS and Bank of Thailand's shared fraud intelligence frameworks to determine if the activity requires a formal regulatory filing.
Shared data for this challenge
Review public datasets and any private uploads tied to your build.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
Graph Traversal Integrity
Verify the Network Agent actually traversed the transaction graph.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Fraud Recall
Percentage of actual fraud cases identified. • target: 0.95 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Master the initialization of AutoGen agents with specific system instructions for financial roles
Implement a 'GroupChatManager' in AutoGen to coordinate data sharing between analysts and compliance agents
Build a custom tool for Claude 3.5 Sonnet to query graph-structured data representing transaction flows
Design prompt templates that enforce strict adherence to MAS/Bank of Thailand regulatory reporting standards
Orchestrate a 'termination condition' where agents reach a consensus on whether a transaction is fraudulent
Deploy a logging layer to capture the multi-agent 'inner monologue' for auditability and compliance review
Optimize Claude 3.5 Sonnet's tool usage for identifying 'Smurfing' patterns (large transactions split into smaller ones)
How this agent runs
Evaluation focuses on the system's ability to correctly identify a 'mule account ring' and generate an accurate regulatory report.
Challenge input
JSON transaction log containing 100+ entries with 'source_id', 'target_id', 'amount', and 'timestamp'.
AutoGen
Required framework for multi-agent logic.
Claude 3.5 Sonnet
Superior reasoning for financial entity resolution.
Neo4j
Graph relationships are essential for detecting mule rings.
Evaluated output
List of flagged account IDs and the specific reason for flags.
- Verify the Network Agent actually traversed the transaction graph.
- Percentage of actual fraud cases identified. • target: 0.95 • range: 0-1
- Benchmark: FIN-Crime-Agent-Bench
- Fraud Recall target: 0.95
- 1 public reference case
- Python execution harness
- Python sandbox
View technical recipe
Configured tools
- AutoGen · Required
- Claude Code · Optional
- Claude Code · Optional
- Claude 4.1 Opus · Optional
Evaluation contract
- Graph Traversal Integrity · Weight 1
- Fraud Recall · Weight 1
Recipe state
This is a preview. The configuration can change before the evaluation recipe is locked.
Run this agent on your dataset and AI stack
Bring your dataset, model providers, and success criteria. We will scope the right managed run for your team.
Scope a managed run[ok] Wrote CHALLENGE.md
[ok] Wrote .versalist.json
[ok] Wrote eval/examples.json
Requires VERSALIST_API_KEY. Works with any MCP-aware editor.
DocsFind another challenge
Jump to a random challenge when you want a fresh benchmark or a different problem space.