Skip to main content
Loading navigation
Back to challenges

Passkey Authentication & Anomaly Detection Agent with OpenAI Agents SDK

Develop a session security agent using OpenAI Agents SDK to analyze FIDO2 passkey login assertions and behavioral metadata. Detect zero-day credential relay attacks while streamlining passwordless authentication for Singapore public applications.

Workflow AutomationHosted by Vera
Sign in free to start

What you are building

Utilize OpenAI Agents SDK to inspect WebAuthn/Passkey authentication challenges and flag behavioral anomalies in real time.

How work is evaluated

Evaluation

Tests identification of passkey relay attacks vs legitimate authentication flows.

Datasets

Public datasets, plus the private datasets you have access to.

How submissions are scored

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

relay_attack_blocked

Ensures mismatched origin triggers immediate block

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

detection_accuracy

Accuracy of passkey origin verification • target: 1 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

  • Implement autonomous agent loops with OpenAI Agents SDK tool definitions

  • Verify WebAuthn passkey client data JSON signatures and origin assertions

  • Evaluate device fingerprint and IP geo-velocity anomalies to detect relay attacks

  • Generate immediate stepped-up authentication directives for flagged login sessions

Resources

Dataset notes

Synthetic dataset of WebAuthn authentication payloads and network contexts.

Challenge setup

Tests identification of passkey relay attacks vs legitimate authentication flows.

Challenge input

JSON with passkey_assertion and session_context

OpenAI Agents SDK

Provides official SDK patterns for multi-turn autonomous tools.

Evaluated output

JSON with action (allow/challenge/block), anomaly_type, and risk_score

Evaluation checks
  • Ensures mismatched origin triggers immediate block
  • Accuracy of passkey origin verification • target: 1 • range: 0-1
Configured evaluation evidence
  • Benchmark: Authentication Anomaly Benchmark
  • Detection Accuracy target: 1
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Shows the first task, up to three tools, and up to three evidence items.

View technical recipe

Configured tools

Action Space
  • OpenAI · Required
  • StackAI · Optional
  • Smolagents · Optional

Evaluation contract

  • relay_attack_blocked · Weight 1
  • detection_accuracy · Weight 1

Metrics

  • detection_accuracy · Target 1

Benchmarks

  • Authentication Anomaly Benchmark
  • security

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON with passkey_assertion and session_context

    Output: JSON with action (allow/challenge/block), anomaly_type, and risk_score

    Execution harness: Python

Evaluation tools

  • OpenAI Agents SDK · Agent Framework

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Passkey Authentication & Anomaly Detection Agent with OpenAI Agents SDK