Skip to main content
Back to challenges
Challenge

Optimized High-Performance Inference Pipeline with Pydantic AI

Build an AI inference engine optimized for high-volume tasks using Pydantic AI. This challenge focuses on model routing between GPT-5.4 Pro and Claude Sonnet 4.6.6 for complex reasoning versus creative generation. You will deploy the application on Novita AI and Baseten to handle model serving, focusing on the architectural integrity provided by Pydantic validation for structured outputs in multi-step agent reasoning pipelines.

Machine LearningHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Design an AI inference service using Pydantic AI with structured model routing between GPT-5.4 Pro and Claude Sonnet 4.6.6.

Delivery guide

How work is evaluated

Evaluation

Benchmarking inference throughput and structural output accuracy

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

SchemaValidation

All responses pass schema check

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

Latency

P99 latency (ms) • target: 200 • range: 0-1000

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Master Pydantic AI schemas for robust AI message passing and state management

  • Implement model routing logic between GPT-5.4 Pro (reasoning) and Claude Sonnet 4.6.6 (creative)

  • Deploy optimized endpoints on Novita AI for high-throughput model serving

  • Leverage Baseten for scalable inference infrastructure management

  • Apply dependency injection for modular agent configuration within Pydantic AI

  • Design advanced error handling for multi-model inference sequences

Resources and assets

Reference links and supporting material

Dataset notes

Dataset will be provided

Challenge setup

Benchmarking inference throughput and structural output accuracy

Challenge input

List of complex prompts

Baseten

ML infrastructure for deploying models

Novita AI

Action Space in the agent workflow.

Zed

High-performance code editor

Evaluated output

Latency metrics

Evaluation checks
  • All responses pass schema check
  • P99 latency (ms) • target: 200 • range: 0-1000
Configured evaluation evidence
  • Latency target: 200
Configured runtime
  • Python execution harness

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Baseten · Required
  • Novita AI · Optional
  • Zed · Optional

Evaluation contract

  • SchemaValidation · Weight 1
  • Latency · Weight 1

Metrics

  • Latency · Target 200

Workflow tasks

  1. Task 1

    Input: List of complex prompts

    Output: Latency metrics

    Execution harness: Python

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Optimized High-Performance Inference Pipeline with Pydantic AI