Optimized High-Performance Inference Pipeline with Pydantic AI
Build an AI inference engine optimized for high-volume tasks using Pydantic AI. This challenge focuses on model routing between GPT-5.4 Pro and Claude Sonnet 4.6.6 for complex reasoning versus creative generation. You will deploy the application on Novita AI and Baseten to handle model serving, focusing on the architectural integrity provided by Pydantic validation for structured outputs in multi-step agent reasoning pipelines.
What you are building
Design an AI inference service using Pydantic AI with structured model routing between GPT-5.4 Pro and Claude Sonnet 4.6.6.
How work is evaluated
Benchmarking inference throughput and structural output accuracy
Datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
SchemaValidation
All responses pass schema check
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Latency
P99 latency (ms) • target: 200 • range: 0-1000
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Learning goals
Master Pydantic AI schemas for robust AI message passing and state management
Implement model routing logic between GPT-5.4 Pro (reasoning) and Claude Sonnet 4.6.6 (creative)
Deploy optimized endpoints on Novita AI for high-throughput model serving
Leverage Baseten for scalable inference infrastructure management
Apply dependency injection for modular agent configuration within Pydantic AI
Design advanced error handling for multi-model inference sequences
Resources
Dataset will be provided
Challenge setup
Benchmarking inference throughput and structural output accuracy
Challenge input
List of complex prompts
Baseten
ML infrastructure for deploying models
Novita AI
Action Space in the agent workflow.
Zed
High-performance code editor
Evaluated output
Latency metrics
- All responses pass schema check
- P99 latency (ms) • target: 200 • range: 0-1000
- Latency target: 200
- Python execution harness
Shows the first task, up to three tools, and up to three evidence items.
View technical recipe
Configured tools
- Baseten · Required
- Novita AI · Optional
- Zed · Optional
Evaluation contract
- SchemaValidation · Weight 1
- Latency · Weight 1
Metrics
- Latency · Target 200
Workflow tasks
- Task 1
Input: List of complex prompts
Output: Latency metrics
Execution harness: Python
Recipe freeze marker
No recipe freeze time is recorded.