Build a Cross-Border Payment Fraud Agent with Mastra AI and Arize Phoenix
Inspired by the 2026 MAS and Bank of Thailand MOU on cybersecurity and digital fraud protection, this challenge tasks you with building a high-trust payment monitoring agent. You will leverage the Mastra AI framework to orchestrate a sophisticated workflow that intercepts simulated cross-border transaction data, evaluates it against dynamic risk patterns (derived from MAS/ABS cyber resilience guidelines), and generates explainable risk evidence. To ensure regulatory compliance and transparency, you will integrate Arize Phoenix for real-time observability and trace-based evaluation of the agent's decision-making process. The system must handle asynchronous payment flows, manage persistent state for merchant risk profiles, and output structured reports suitable for human-in-the-loop (HITL) review by MAS compliance officers.
What you are building
The core problem, expected build, and operating context for this challenge.
Inspired by the 2026 MAS and Bank of Thailand MOU on cybersecurity and digital fraud protection, this challenge tasks you with building a high-trust payment monitoring agent. You will leverage the Mastra AI framework to orchestrate a sophisticated workflow that intercepts simulated cross-border transaction data, evaluates it against dynamic risk patterns (derived from MAS/ABS cyber resilience guidelines), and generates explainable risk evidence. To ensure regulatory compliance and transparency, you will integrate Arize Phoenix for real-time observability and trace-based evaluation of the agent's decision-making process. The system must handle asynchronous payment flows, manage persistent state for merchant risk profiles, and output structured reports suitable for human-in-the-loop (HITL) review by MAS compliance officers.
Shared data for this challenge
Review public datasets and any private uploads tied to your build.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
Schema Validation
Ensures the output matches the required regulatory JSON schema.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Trace Persistence
Verifies that traces were successfully exported to Arize Phoenix.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
F1 Score
Balance of precision and recall for fraud identification. • target: 0.85 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Master Mastra AI initialization and workflow definition using TypeScript to create structured state machines for transaction processing
Implement Mastra AI 'Tools' that interface with external blocklists and merchant reputation databases
Orchestrate complex human-in-the-loop (HITL) triggers using Mastra's 'Step' architecture for high-value transaction flags
Integrate Arize Phoenix for OpenInference-based tracing to capture LLM input/output pairs and latency metrics
Deploy Arize Phoenix evaluators to measure the precision and recall of the fraud detection agent against a ground-truth dataset
Build a custom 'Audit Trail' service that persists Mastra execution traces into a regulator-friendly format
Optimize agent performance by using Mastra AI memory to contextually remember repeat transaction patterns
How this agent runs
The challenge is evaluated on the agent's ability to correctly flag fraudulent transactions while maintaining a low false-positive rate and providing detailed traces in Arize Phoenix.
Challenge input
JSON array of transaction objects containing amount, currency, sender_country, receiver_country, and merchant_id.
Mastra AI
Required for building the agentic workflow in TypeScript.
Arize Phoenix
Required for tracing and evaluating the agent logic.
Evaluated output
JSON object with boolean flags and evidence strings.
- Ensures the output matches the required regulatory JSON schema.
- Verifies that traces were successfully exported to Arize Phoenix.
- Balance of precision and recall for fraud identification. • target: 0.85 • range: 0-1
- Benchmark: FinReg-Eval-2026
- F1 Score target: 0.85
- 1 public reference case
- JavaScript execution harness
- Javascript sandbox
View technical recipe
Configured tools
- Mastra AI · Required
- SHAP (Explainability) · Optional
- Arize Phoenix · Optional
- Arize Phoenix · Optional
- Mastra AI · Required
Evaluation contract
- Schema Validation · Weight 1
- Trace Persistence · Weight 1
- F1 Score · Weight 1
Recipe state
This is a preview. The configuration can change before the evaluation recipe is locked.
Run this agent on your dataset
Versalist can run this agent on your behalf with your data. Tell us about your dataset and the result you need.
Discuss your dataset[ok] Wrote CHALLENGE.md
[ok] Wrote .versalist.json
[ok] Wrote eval/examples.json
Requires VERSALIST_API_KEY. Works with any MCP-aware editor.
DocsFind another challenge
Jump to a random challenge when you want a fresh benchmark or a different problem space.