Autonomous Security Operator Agents SDK
For general security intelligence, this challenge requires building an autonomous agent capable of identifying and fixing vulnerabilities in real-time. You will use the OpenAI Agents SDK to build a multi-turn operator that utilizes function calling to interface with Proxis AI for engineering diagnostics and Vellum for managing agent prompts and prompt variants. The agent must perform multi-step reasoning using the GPT-5.3-Codex model to simulate a human security engineer navigating complex codebases and Slack-based incident reports.
What you are building
The core problem, expected build, and operating context for this challenge.
Build an autonomous security remediation agent using the OpenAI Agents SDK and GPT-5.3-Codex to find and patch vulnerabilities.
How work is evaluated
Evaluation focuses on the agent's ability to successfully patch a known vulnerability and verify the fix without human intervention.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
Security Validation
Patch must eliminate the specific vulnerability without introducing new ones.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Remediation Speed
Time taken from ingestion to patch generation • target: 60 • range: 0-300
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Orchestrate OpenAI Agents SDK role-based teams with specialized agents for vulnerability detection and automated patching
Leverage the GPT-5.3-Codex model for advanced multi-step reasoning in security-sensitive contexts
Design autonomous tool-calling loops that allow agents to browse file systems and execute Proxis AI diagnostic tools
Implement Libretto to evaluate agent performance and route requests based on latency and security scoring
Integrate Cartesia voice interfaces to provide real-time audio status updates for security operations centers
Master the hand-off pattern in OpenAI Agents SDK to transition from detection agents to remediation specialists
Reference links and supporting material
Sample data for 1 tasks
Challenge setup
Evaluation focuses on the agent's ability to successfully patch a known vulnerability and verify the fix without human intervention.
Challenge input
JSON containing vulnerability description and target codebase snippet
OpenAI
OpenAI AI model provider
Vellum
LLM development and testing platform
GPT-5
Policy Serving in the agent workflow.
Evaluated output
JSON containing patch file and reasoning steps
- Patch must eliminate the specific vulnerability without introducing new ones.
- Time taken from ingestion to patch generation • target: 60 • range: 0-300
- Remediation Speed target: 60
- 1 public reference case
- Python execution harness
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- OpenAI · Required
- Vellum · Optional
- GPT-5 · Optional
Evaluation contract
- Security Validation · Weight 1
- Remediation Speed · Weight 1
Metrics
- Remediation Speed · Target 60
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON containing vulnerability description and target codebase snippet
Output: JSON containing patch file and reasoning steps
Execution harness: Python
Recipe freeze marker
No recipe freeze time is recorded.