Skip to main content
Back to challenges
Challenge

Passkey Authentication & Anomaly Detection Agent with OpenAI Agents SDK

Develop a session security agent using OpenAI Agents SDK to analyze FIDO2 passkey login assertions and behavioral metadata. Detect zero-day credential relay attacks while streamlining passwordless authentication for Singapore public applications.

Workflow AutomationHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Utilize OpenAI Agents SDK to inspect WebAuthn/Passkey authentication challenges and flag behavioral anomalies in real time.

Delivery guide

How work is evaluated

Evaluation

Tests identification of passkey relay attacks vs legitimate authentication flows.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

relay_attack_blocked

Ensures mismatched origin triggers immediate block

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

detection_accuracy

Accuracy of passkey origin verification • target: 1 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Implement autonomous agent loops with OpenAI Agents SDK tool definitions

  • Verify WebAuthn passkey client data JSON signatures and origin assertions

  • Evaluate device fingerprint and IP geo-velocity anomalies to detect relay attacks

  • Generate immediate stepped-up authentication directives for flagged login sessions

Resources and assets

Reference links and supporting material

Dataset notes

Synthetic dataset of WebAuthn authentication payloads and network contexts.

Challenge setup

Tests identification of passkey relay attacks vs legitimate authentication flows.

Challenge input

JSON with passkey_assertion and session_context

OpenAI Agents SDK

Provides official SDK patterns for multi-turn autonomous tools.

Evaluated output

JSON with action (allow/challenge/block), anomaly_type, and risk_score

Evaluation checks
  • Ensures mismatched origin triggers immediate block
  • Accuracy of passkey origin verification • target: 1 • range: 0-1
Configured evaluation evidence
  • Benchmark: Authentication Anomaly Benchmark
  • Detection Accuracy target: 1
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • OpenAI · Required
  • StackAI · Optional
  • Smolagents · Optional

Evaluation contract

  • relay_attack_blocked · Weight 1
  • detection_accuracy · Weight 1

Metrics

  • detection_accuracy · Target 1

Benchmarks

  • Authentication Anomaly Benchmark
  • security

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON with passkey_assertion and session_context

    Output: JSON with action (allow/challenge/block), anomaly_type, and risk_score

    Execution harness: Python

Evaluation tools

  • OpenAI Agents SDK · Agent Framework

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Passkey Authentication & Anomaly Detection Agent with OpenAI Agents SDK