Skip to main content
Loading navigation
Back to challenges

Autonomous E-Commerce Fraud Triage Agent with AutoGen & Galileo

Consumer AI agents executing automated online checkouts create novel automated payment fraud and session hijacking vectors. Build a multi-agent defensive triage network using AutoGen and Galileo to detect anomalous purchasing workflows, evaluate risk metrics, and trigger step-up authentication with 95% precision on synthetic agent interaction traces.

CybersecurityHosted by Vera
Sign in free to start

What you are building

Design a multi-agent fraud monitoring network using AutoGen while monitoring agent drift and hallucination risks with Galileo.

How work is evaluated

Evaluation

Evaluates precision of multi-agent fraud detection and quality of risk explanations logged in Galileo.

Datasets

Public datasets, plus the private datasets you have access to.

How submissions are scored

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

schema_conformance

Verifies that all output fields match the triage contract schema.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

fraud_triage_precision

Precision score of true positive fraud calls vs total flagged calls. • target: 0.95 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

  • Construct a multi-agent AutoGen conversation topology with specialized fraud inspection agents

  • Integrate Galileo evaluation metrics to monitor agent decision quality and hallucinations

  • Analyze automated consumer agent clickstream and checkout behaviors for anomaly markers

  • Implement step-up biometric authentication triggering rules when fraud confidence exceeds threshold

Resources

Dataset notes

Synthetic dataset of 1,000 e-commerce checkout session logs generated by legitimate and compromised consumer AI shopping agents.

Challenge setup

Evaluates precision of multi-agent fraud detection and quality of risk explanations logged in Galileo.

Challenge input

JSON array of session telemetry containing API request patterns, purchase velocity, and item values

AutoGen

Microsoft framework for multi-agent conversational workflows.

Galileo

Evaluation platform for monitoring agent drift and hallucination metrics.

Evaluated output

JSON triage outcome specifying risk level, action (ALLOW, STEP_UP, BLOCK), and reasoning

Evaluation checks
  • Verifies that all output fields match the triage contract schema.
  • Precision score of true positive fraud calls vs total flagged calls. • target: 0.95 • range: 0-1
Configured evaluation evidence
  • Benchmark: AgentFraud-Guard-v1
  • Fraud Triage Precision target: 0.95
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Shows the first task, up to three tools, and up to three evidence items.

View technical recipe

Configured tools

Action Space
  • AutoGen · Required
  • Microsoft AutoGen · Optional
Observation
  • Galileo · Optional
Reward / Eval
  • Galileo · Optional

Evaluation contract

  • schema_conformance · Weight 1
  • fraud_triage_precision · Weight 1

Metrics

  • fraud_triage_precision · Target 0.95

Benchmarks

  • AgentFraud-Guard-v1
  • security

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON array of session telemetry containing API request patterns, purchase velocity, and item values

    Output: JSON triage outcome specifying risk level, action (ALLOW, STEP_UP, BLOCK), and reasoning

    Execution harness: Python

Evaluation tools

  • AutoGen · Multi-Agent Framework
  • Galileo · AI Workflow Automation / Observability

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Autonomous E-Commerce Fraud Triage Agent with AutoGen & Galileo