Build a UPI Fraud Triage Pipeline with CrewAI and Metaflow Workflows
In this challenge, you will design and implement a real-time Unified Payments Interface (UPI) fraud detection and triage pipeline using CrewAI and Metaflow. With millions of micro-transactions processed daily across Indian digital public infrastructure, fraudulent transactions must be detected, triaged, and flagged in milliseconds using merchant risk profiles, device signals, and behavioral pattern analysis. You will orchestrate role-playing CrewAI agents (Risk Scorer, Device Forensic Agent, and Triage Coordinator) embedded inside a production-grade Metaflow pipeline. The pipeline will process streaming payment logs, assign confidence scores, and automatically generate consent-aware compliance logs for regulatory auditing.
What you are building
The core problem, expected build, and operating context for this challenge.
Orchestrate multi-agent UPI fraud triage workflows using CrewAI agents embedded in production-grade Metaflow pipelines.
How work is evaluated
Evaluates the accuracy and throughput of the UPI fraud triage system under simulated transaction spikes.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
high_risk_flagging_test
Ensures anomalous high-value new device transactions are flagged as high risk.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
accuracy
Precision and recall balance across test fraud datasets. • target: 0.95 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Orchestrate multi-agent CrewAI flows for multi-layered fraud risk scoring.
Integrate Metaflow steps to automate reproducible payment telemetry pipelines.
Implement real-time device fingerprinting and velocity checking logic.
Generate audit-ready regulatory reports compliant with RBI guidelines.
Reference links and supporting material
Synthetic UPI payment log stream with multi-dimensional risk markers including device IDs, VPAs, and timestamps.
Challenge setup
Evaluates the accuracy and throughput of the UPI fraud triage system under simulated transaction spikes.
Challenge input
JSON containing transaction payload with VPA, device ID, location, and amount.
CrewAI
Required framework for role-playing multi-agent system.
Metaflow
Required workflow automation tool for execution reproducibility.
Evaluated output
JSON containing risk_score, risk_level, and audit_trail.
- Ensures anomalous high-value new device transactions are flagged as high risk.
- Precision and recall balance across test fraud datasets. • target: 0.95 • range: 0-1
- Benchmark: DPI FinTech Fraud Standard
- Accuracy target: 0.95
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- Metaflow · Required
- CrewAI · Optional
- Meta · Optional
Evaluation contract
- high_risk_flagging_test · Weight 1
- accuracy · Weight 1
Metrics
- accuracy · Target 0.95
Benchmarks
- DPI FinTech Fraud Standard
- security
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON containing transaction payload with VPA, device ID, location, and amount.
Output: JSON containing risk_score, risk_level, and audit_trail.
Execution harness: Python
Evaluation tools
- CrewAI · Agent Orchestration
- Metaflow · Workflow Pipeline
Recipe freeze marker
No recipe freeze time is recorded.