Autonomous E-Commerce Fraud Triage Agent with AutoGen & Galileo
Consumer AI agents executing automated online checkouts create novel automated payment fraud and session hijacking vectors. Build a multi-agent defensive triage network using AutoGen and Galileo to detect anomalous purchasing workflows, evaluate risk metrics, and trigger step-up authentication with 95% precision on synthetic agent interaction traces.
What you are building
The core problem, expected build, and operating context for this challenge.
Design a multi-agent fraud monitoring network using AutoGen while monitoring agent drift and hallucination risks with Galileo.
How work is evaluated
Evaluates precision of multi-agent fraud detection and quality of risk explanations logged in Galileo.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
schema_conformance
Verifies that all output fields match the triage contract schema.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
fraud_triage_precision
Precision score of true positive fraud calls vs total flagged calls. • target: 0.95 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Construct a multi-agent AutoGen conversation topology with specialized fraud inspection agents
Integrate Galileo evaluation metrics to monitor agent decision quality and hallucinations
Analyze automated consumer agent clickstream and checkout behaviors for anomaly markers
Implement step-up biometric authentication triggering rules when fraud confidence exceeds threshold
Reference links and supporting material
Synthetic dataset of 1,000 e-commerce checkout session logs generated by legitimate and compromised consumer AI shopping agents.
Challenge setup
Evaluates precision of multi-agent fraud detection and quality of risk explanations logged in Galileo.
Challenge input
JSON array of session telemetry containing API request patterns, purchase velocity, and item values
AutoGen
Microsoft framework for multi-agent conversational workflows.
Galileo
Evaluation platform for monitoring agent drift and hallucination metrics.
Evaluated output
JSON triage outcome specifying risk level, action (ALLOW, STEP_UP, BLOCK), and reasoning
- Verifies that all output fields match the triage contract schema.
- Precision score of true positive fraud calls vs total flagged calls. • target: 0.95 • range: 0-1
- Benchmark: AgentFraud-Guard-v1
- Fraud Triage Precision target: 0.95
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- AutoGen · Required
- Microsoft AutoGen · Optional
- Galileo · Optional
- Galileo · Optional
Evaluation contract
- schema_conformance · Weight 1
- fraud_triage_precision · Weight 1
Metrics
- fraud_triage_precision · Target 0.95
Benchmarks
- AgentFraud-Guard-v1
- security
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON array of session telemetry containing API request patterns, purchase velocity, and item values
Output: JSON triage outcome specifying risk level, action (ALLOW, STEP_UP, BLOCK), and reasoning
Execution harness: Python
Evaluation tools
- AutoGen · Multi-Agent Framework
- Galileo · AI Workflow Automation / Observability
Recipe freeze marker
No recipe freeze time is recorded.