Build Consumer AI Agent Session Risk Scoring with AI SDK and Novita AI
Autonomous consumer AI agents conducting online payments create novel transaction fraud and session hijacking risks. Build an AI SDK web application using Novita AI inference models to detect agent-initiated fraud, analyze behavioral session telemetry, and trigger step-up authentication. Maintain real-time threat detection with latency under 150ms while capping false positives under 3%.
What you are building
The core problem, expected build, and operating context for this challenge.
Build a Next.js/TypeScript application using Vercel AI SDK and Novita AI fast inference API to score session telemetry of web browsing AI agents in real time.
How work is evaluated
Evaluates session telemetry risk classifier for prompt execution speed and fraud classification accuracy.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
threat_score_range
Verifies threat score is bounded between 0.0 and 1.0
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
detection_accuracy
Accuracy in detecting synthetic browser agent sessions • target: 0.95 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Implement real-time streaming telemetry analysis using Vercel AI SDK
Integrate Novita AI high-speed model runtime for low-latency scoring
Classify human user vs autonomous agent behavioral patterns during payment flows
Trigger dynamic step-up authentication mechanisms on high-risk dynamic triggers
Reference links and supporting material
Dataset of 600 session telemetry traces recorded from real browser interactions and automated AI agent web actions.
Challenge setup
Evaluates session telemetry risk classifier for prompt execution speed and fraud classification accuracy.
Challenge input
JSON containing keystroke_dynamics, page_pacing_ms, and user_agent_header
AI SDK
Offers standard TypeScript streaming interfaces for AI interaction.
Novita AI
Provides low-latency model API runtime suitable for real-time web protection.
Evaluated output
JSON with threat_score, action_required, is_autonomous_agent
- Verifies threat score is bounded between 0.0 and 1.0
- Accuracy in detecting synthetic browser agent sessions • target: 0.95 • range: 0-1
- Benchmark: Bot & Fraud Detection Telemetry Suite
- Detection Accuracy target: 0.95
- 1 public reference case
- JavaScript execution harness
- javascript sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- Novita AI · Required
- Vercel AI SDK · Optional
- Vercel AI SDK · Optional
Evaluation contract
- threat_score_range · Weight 1
- detection_accuracy · Weight 1
Metrics
- detection_accuracy · Target 0.95
Benchmarks
- Bot & Fraud Detection Telemetry Suite
- security
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON containing keystroke_dynamics, page_pacing_ms, and user_agent_header
Output: JSON with threat_score, action_required, is_autonomous_agent
Execution harness: JavaScript
Evaluation tools
- AI SDK · Client/Server Agent SDK
- Novita AI · Inference Engine
Recipe freeze marker
No recipe freeze time is recorded.