Automated Merchant Risk Scoring Agent with Pydantic AI and OpenRouter
Rapid proliferation of e-commerce AI agents requires real-time risk assessment for autonomous consumer payment actions. Build a type-safe merchant risk analyzer using Pydantic AI and OpenRouter to validate transaction intent, verify merchant legitimacy, and enforce dynamic spending controls. Maintain zero false positives on verified low-risk merchant checkouts while flagging fraudulent intent.
What you are building
Develop a real-time, type-safe merchant risk evaluation engine using Pydantic AI and OpenRouter to screen autonomous consumer agent transactions.
How work is evaluated
Evaluates accuracy of merchant threat assessment and validity of structured response models.
Datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
high_risk_flag_test
Verifies suspicious URLs are blocked or flagged for step-up MFA
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
risk_precision
Precision of risk decision classifications • target: 0.95 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Learning goals
Define strict Pydantic AI output models for merchant verification and transaction limits
Configure OpenRouter dynamic LLM fallback chains for low-latency scoring
Implement domain validation and merchant history lookup tools
Enforce step-up authentication when merchant risk parameters trigger medium-risk alerts
Resources
Dataset of 400 transaction checkout payloads with synthetic domain characteristics and merchant trust signals.
Challenge setup
Evaluates accuracy of merchant threat assessment and validity of structured response models.
Challenge input
JSON containing checkout details, merchant URL, user ID, and intent payload
Pydantic AI
Type-safe modeling framework
OpenRouter
Multi-provider model routing
Evaluated output
Pydantic validated JSON with risk_score, decision, and rationale
- Verifies suspicious URLs are blocked or flagged for step-up MFA
- Precision of risk decision classifications • target: 0.95 • range: 0-1
- Benchmark: FraudGuard-Eval
- Risk Precision target: 0.95
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Shows the first task, up to three tools, and up to three evidence items.
View technical recipe
Configured tools
- Pydantic AI · Required
- OpenRouter · Optional
- Pydantic AI · Required
- OpenRouter · Optional
Evaluation contract
- high_risk_flag_test · Weight 1
- risk_precision · Weight 1
Metrics
- risk_precision · Target 0.95
Benchmarks
- FraudGuard-Eval
- security
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON containing checkout details, merchant URL, user ID, and intent payload
Output: Pydantic validated JSON with risk_score, decision, and rationale
Execution harness: Python
Evaluation tools
- Pydantic AI · Agent Framework
- OpenRouter · LLM Gateway
Recipe freeze marker
No recipe freeze time is recorded.