Interconnect Distributed Simulator and Fault Harness with Mastra AI and Braintrust
Multi-node inference requires high bandwidth interconnects and resilient fault handling across NVLink and InfiniBand clusters. You will build a distributed simulator agent using Mastra AI in TypeScript, instrumented with Braintrust tracing, to model link congestion, node failure, and state migration. The solution must demonstrate zero lost requests during simulated 10Gbps link degradations.
What you are building
Develop an interconnect-aware distributed inference simulator using Mastra AI agent workflows and Braintrust evaluation logging.
How work is evaluated
Simulates multi-node network degradation and evaluates failover time and request success rate.
Datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
zero_dropped_requests
Validates all active requests were safely reassigned during node failure
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
failover_duration_ms
Time taken to detect fault and reroute tensor parallel execution paths • target: 150 • range: 0-500
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Learning goals
Construct Mastra AI workflows modeling Tensor Parallelism (TP) and Pipeline Parallelism (PP) network communications
Integrate Braintrust SDK for continuous performance tracing and network bottleneck evaluations
Simulate NVLink (900 GB/s) vs InfiniBand (400 Gbps) inter-node throughput limits
Implement automatic rerouting and failover mechanisms when a worker node degrades
Resources
Network topologies and node fault traces representing multi-node GPU clusters.
Challenge setup
Simulates multi-node network degradation and evaluates failover time and request success rate.
Challenge input
JSON specifying nodes_count, interconnect_type (nvlink|infiniband), failed_node_id, and active_requests
Mastra AI
Provides structured workflow pipelines and agent state management in TypeScript.
Braintrust
Enables evaluation logging and trace metrics for distributed node workflows.
Evaluated output
JSON returning failover_duration_ms, recovered_requests_count, and network_overhead_mb
- Validates all active requests were safely reassigned during node failure
- Time taken to detect fault and reroute tensor parallel execution paths • target: 150 • range: 0-500
- Benchmark: Distributed-LLM-Resilience
- Failover Duration Ms target: 150
- 1 public reference case
- JavaScript execution harness
- javascript sandbox (unavailable on Versalist)
Shows the first task, up to three tools, and up to three evidence items.
View technical recipe
Configured tools
- Mastra AI · Required
- Anthropic · Optional
- Braintrust · Optional
- Braintrust · Optional
- Mastra AI · Required
Evaluation contract
- zero_dropped_requests · Weight 1
- failover_duration_ms · Weight 1
Metrics
- failover_duration_ms · Target 150
Benchmarks
- Distributed-LLM-Resilience
- general
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON specifying nodes_count, interconnect_type (nvlink|infiniband), failed_node_id, and active_requests
Output: JSON returning failover_duration_ms, recovered_requests_count, and network_overhead_mb
Execution harness: JavaScript
Evaluation tools
- Mastra AI · Agent Framework
- Braintrust · AI Workflow Automation
Recipe freeze marker
No recipe freeze time is recorded.