Pydantic AI & Baseten Distributed Inference Interconnect Simulator
Network interconnect bottlenecks between host nodes cause high pipeline latency in multi-node tensor parallel clusters. Build a Pydantic AI agent system deployed with Baseten runtime tools to simulate NVLink, NVSwitch, and InfiniBand cluster topology, inject network fault scenarios, and calculate tail latency recovery times.
What you are building
The core problem, expected build, and operating context for this challenge.
Develop a distributed inference interconnect simulator using Pydantic AI and Baseten to analyze network bottlenecks and tail latency under node faults.
How work is evaluated
Evaluates distributed interconnect simulator accuracy during hardware link fault injection tests.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
latency_degradation_check
Checks that degraded latency reflects simulated bandwidth constraints.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
simulation_accuracy
Correlation with physical NCCL All-Reduce trace benchmarks • target: 0.95 • range: 0.88-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Define type-safe interconnect models using Pydantic AI models and dependency injection
Deploy inference runtime tools on Baseten to profile pipeline and tensor parallel communication
Simulate NVLink/NVSwitch 900GB/s and InfiniBand 400Gbps network links
Inject packet drop and cable failure events to verify multi-node fault recovery speed
Reference links and supporting material
NCCL collective benchmark performance logs across varied cluster topologies.
Challenge setup
Evaluates distributed interconnect simulator accuracy during hardware link fault injection tests.
Challenge input
JSON containing cluster_size, topology, failure_event, model_tp_size
Pydantic AI
Structured Python agent framework with strict validation
Baseten
Model serving and system execution runtime
Evaluated output
JSON containing baseline_latency_ms, degraded_latency_ms, recovery_time_ms, throughput_degradation_pct
- Checks that degraded latency reflects simulated bandwidth constraints.
- Correlation with physical NCCL All-Reduce trace benchmarks • target: 0.95 • range: 0.88-1
- Benchmark: NCCL Performance Tests
- Simulation Accuracy target: 0.95
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- Pydantic AI · Required
- Baseten · Optional
- Anthropic · Optional
- Pydantic AI · Required
Evaluation contract
- latency_degradation_check · Weight 1
- simulation_accuracy · Weight 1
Metrics
- simulation_accuracy · Target 0.95
Benchmarks
- NCCL Performance Tests
- general
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON containing cluster_size, topology, failure_event, model_tp_size
Output: JSON containing baseline_latency_ms, degraded_latency_ms, recovery_time_ms, throughput_degradation_pct
Execution harness: Python
Evaluation tools
- Pydantic AI · Type-Safe Agent Framework
- Baseten · Inference Runtime Host
Recipe freeze marker
No recipe freeze time is recorded.