UPI Fraud Detection Agent with OpenAI Agents SDK and Langfuse
High-volume UPI merchant transactions require real-time risk triage to prevent fraud without blocking legitimate users. Build an agentic triage system using OpenAI Agents SDK and track telemetry with Langfuse to achieve sub-200ms latency and 95% precision on fraudulent transaction flags.
What you are building
The core problem, expected build, and operating context for this challenge.
Implement an automated UPI fraud triage pipeline utilizing the OpenAI Agents SDK for decision-making and Langfuse for agent observability.
How work is evaluated
Evaluates the accuracy and latency of the UPI fraud agent using simulated payment event streams.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
json_schema_compliance
Checks if output conforms strictly to risk_status and risk_score format.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
precision
Precision of fraud detection flags. • target: 0.95 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
latency_ms
Average response time per transaction. • target: 200 • range: 0-2000
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Master tool calling and agent orchestration with OpenAI Agents SDK
Integrate Langfuse for real-time trace telemetry and score evaluation
Design low-latency fraud classification routines for high-throughput payment streams
Implement automated risk scoring using device and transaction context
Reference links and supporting material
Simulated stream of 1,000 UPI transactions labeled with fraud ground truth.
Challenge setup
Evaluates the accuracy and latency of the UPI fraud agent using simulated payment event streams.
Challenge input
JSON transaction object containing amount, payer_id, payee_id, location, and device_id.
OpenAI Agents SDK
Official framework for autonomous agent routing.
Langfuse
Provides structured logging and trace evaluation.
Evaluated output
JSON object with risk_status ('APPROVE', 'FLAG', 'BLOCK'), risk_score (0-100), and rationale string.
- Checks if output conforms strictly to risk_status and risk_score format.
- Precision of fraud detection flags. • target: 0.95 • range: 0-1
- Average response time per transaction. • target: 200 • range: 0-2000
- Benchmark: Financial Agent Safety & Speed Benchmark
- Precision target: 0.95
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- OpenAI · Optional
- Cartesia · Optional
- Langfuse · Required
- Langfuse · Required
Evaluation contract
- json_schema_compliance · Weight 1
- precision · Weight 1
- latency_ms · Weight 1
Metrics
- precision · Target 0.95
- latency_ms · Target 200
Benchmarks
- Financial Agent Safety & Speed Benchmark
- security
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON transaction object containing amount, payer_id, payee_id, location, and device_id.
Output: JSON object with risk_status ('APPROVE', 'FLAG', 'BLOCK'), risk_score (0-100), and rationale string.
Execution harness: Python
Evaluation tools
- OpenAI Agents SDK · Agent Framework
- Langfuse · Observability
Recipe freeze marker
No recipe freeze time is recorded.