Skip to main content
Back to challenges
Challenge

Build a Multi-Agent Fraud Investigation System using AutoGen and Claude 3.5 Sonnet

Inspired by the recent Memorandum of Understanding between the Monetary Authority of Singapore (MAS) and the Bank of Thailand regarding digital fraud protection, this challenge tasks you with building a collaborative multi-agent system to automate financial fraud investigations. You will leverage Microsoft's AutoGen framework to orchestrate a conversation between specialized agents: a Transaction Analyst, a Network Relationship Agent (Graph Specialist), and a Compliance Officer. Your goal is to develop a system that can consume streaming transaction logs and suspicious activity reports (SARs), perform entity resolution across cross-border payment rails, and generate a structured investigation report. The Transaction Analyst will identify anomalies, the Network Relationship Agent will query a graph database to find hidden clusters (ring-fencing), and the Compliance Officer agent will cross-reference findings against the MAS and Bank of Thailand's shared fraud intelligence frameworks to determine if the activity requires a formal regulatory filing.

Cybersecurity & PrivacyHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Orchestrate an AutoGen multi-agent system using Claude 3.5 Sonnet to perform automated cross-border fraud entity resolution and regulatory reporting.

Delivery guide

How work is evaluated

Evaluation

Evaluation focuses on the system's ability to correctly identify a 'mule account ring' and generate an accurate regulatory report.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

Graph Traversal Integrity

Verify the Network Agent actually traversed the transaction graph.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

Fraud Recall

Percentage of actual fraud cases identified. • target: 0.95 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Master multi-agent orchestration patterns using AutoGen's ConversableAgent and GroupChat features

  • Implement entity resolution logic across disparate financial datasets using LLM-based reasoning

  • Integrate Claude 3.5 Sonnet's high-context reasoning for complex financial crime investigation

  • Design safe human-in-the-loop (HITL) workflows for financial compliance approvals

Resources and assets

Reference links and supporting material

Dataset notes

Synthetic financial transaction dataset simulating cross-border flows between Singapore and Thailand, featuring hidden 'mule rings' and structuring patterns.

Challenge setup

Evaluation focuses on the system's ability to correctly identify a 'mule account ring' and generate an accurate regulatory report.

Challenge input

JSON transaction log containing 100+ entries with 'source_id', 'target_id', 'amount', and 'timestamp'.

AutoGen

Required framework for multi-agent logic.

Claude 3.5 Sonnet

Superior reasoning for financial entity resolution.

Neo4j

Graph relationships are essential for detecting mule rings.

Evaluated output

List of flagged account IDs and the specific reason for flags.

Evaluation checks
  • Verify the Network Agent actually traversed the transaction graph.
  • Percentage of actual fraud cases identified. • target: 0.95 • range: 0-1
Configured evaluation evidence
  • Benchmark: FIN-Crime-Agent-Bench
  • Fraud Recall target: 0.95
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • AutoGen · Required
  • Claude Code · Optional
Observation
  • Claude Code · Optional
Policy Serving
  • Claude 4.1 Opus · Optional

Evaluation contract

  • Graph Traversal Integrity · Weight 1
  • Fraud Recall · Weight 1

Metrics

  • Fraud Recall · Target 0.95

Benchmarks

  • FIN-Crime-Agent-Bench
  • security

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON transaction log containing 100+ entries with 'source_id', 'target_id', 'amount', and 'timestamp'.

    Output: List of flagged account IDs and the specific reason for flags.

    Execution harness: Python

Evaluation tools

  • AutoGen · Orchestration Framework
  • Claude 3.5 Sonnet · LLM Backend
  • Neo4j · Database

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Build a Multi-Agent Fraud Investigation System using AutoGen and Claude 3.5 Sonnet