Public evaluation

Fraud Detection Accuracy Task

The challenge is evaluated on the agent's ability to correctly flag fraudulent transactions while maintaining a low false-positive rate and providing detailed traces in Arize Phoenix.

Evaluation type

task based

Challenge

Build a Cross-Border Payment Fraud Agent with Mastra AI and Arize Phoenix

Difficulty

Advanced

Rigor

Unspecified

Open linked challenge

Evaluation overview

How the linked challenge is judged: tasks, benchmarks, and criteria count.

Tasks

Benchmarks

Criteria

Task templates

Inputs and expected outputs.

Task 1

Fraud Detection Accuracy Task

Process a batch of 50 transactions and return a structured JSON mapping transaction IDs to 'FLAG' or 'PASS'.

Input format

JSON array of transaction objects containing amount, currency, sender_country, receiver_country, and merchant_id.

Output format

JSON object with boolean flags and evidence strings.

Linked challenge

Build a Cross-Border Payment Fraud Agent with Mastra AI and Arize Phoenix

Inspired by the 2026 MAS and Bank of Thailand MOU on cybersecurity and digital fraud protection, this challenge tasks you with building a high-trust payment monitoring agent. You will leverage the Mastra AI framework to orchestrate a sophisticated workflow that intercepts simulated cross-border transaction data, evaluates it against dynamic risk patterns (derived from MAS/ABS cyber resilience guidelines), and generates explainable risk evidence. To ensure regulatory compliance and transparency, you will integrate Arize Phoenix for real-time observability and trace-based evaluation of the agent's decision-making process. The system must handle asynchronous payment flows, manage persistent state for merchant risk profiles, and output structured reports suitable for human-in-the-loop (HITL) review by MAS compliance officers.

Open challenge