Public evaluation

PQC_Refactor_Audit

The evaluation measures the agent's ability to refactor a vulnerable codebase to be PQC-ready and crypto-agile.

Evaluation type

task based

Challenge

Build a Crypto-Agile PQC Migration Agent with OpenAI Agents SDK

Difficulty

Advanced

Rigor

Unspecified

Open linked challenge

Evaluation overview

How the linked challenge is judged: tasks, benchmarks, and criteria count.

Tasks

Benchmarks

Criteria

Task templates

Inputs and expected outputs.

Task 1

PQC_Refactor_Audit

Evaluates the agent's detection of hardcoded RSA/ECC and the correctness of the refactored PQC code.

Input format

Legacy source code with embedded RSA-2048 signing/encryption.

Output format

Refactored code using a generic 'AgileCrypto' class and liboqs backends.

Linked challenge

Build a Crypto-Agile PQC Migration Agent with OpenAI Agents SDK

With the rise of quantum computing threats, organizations must transition from classical RSA/ECC to NIST-standardized Post-Quantum Cryptography (PQC). This challenge involves building an autonomous migration agent using the OpenAI Agents SDK and Gemini 2.5 Flash to identify vulnerable cryptographic implementations in legacy codebases and refactor them into a crypto-agile architecture. You will implement a 'Wrapper-First' approach, allowing applications to switch between ML-KEM (Kyber) and classical algorithms without redeploying the entire service. The agent must perform static analysis to detect hardcoded primitives, suggest library replacements using `liboqs`, and generate unit tests that verify both security and performance. The system will leverage OpenAI's tool-calling capabilities to interface with Git repositories and static analysis tools, ensuring that the migration preserves the application's functional integrity while mitigating 'Harvest Now, Decrypt Later' risks.

Open challenge