Skip to main content
Back to challenges
Challenge

CrewAI Distributed Interconnect & Tensor Parallelism Simulator

Design an interconnect-aware distributed inference simulator using CrewAI agents to model multi-node tensor parallel and pipeline parallel workloads across NVLink and AWS EFA topologies. Implement automated bottleneck detection that optimizes cross-node message passing efficiency, achieving a 20% latency reduction in simulated multi-GPU clusters.

EngineeringHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Coordinate role-playing CrewAI agents to simulate NVLink and AWS EFA cross-node bandwidth constraints for distributed tensor parallelism.

Delivery guide

How work is evaluated

Evaluation

Evaluates distributed parallel configuration suggestions on simulated multi-node clusters.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

gpu_topology_conservation

Validates TP * PP equals total cluster GPU count.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

latency_reduction_pct

Simulated latency improvement percentage over baseline naive TP=16. • target: 20 • range: 0-50

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Build a multi-agent distributed simulation using CrewAI framework

  • Model NVLink 900 GB/s intra-node and AWS EFA 800 Gbps inter-node network topologies

  • Simulate AllReduce and AllGather collective communication overheads in Tensor Parallelism (TP=8, PP=2)

  • Optimize parallel splitting parameters to reduce inter-node communication latency by 20%

Resources and assets

Reference links and supporting material

Dataset notes

Network trace profiles of NCCL AllReduce and AllGather operations captured across multi-node H100 and AWS Trainium2 cluster topologies.

Challenge setup

Evaluates distributed parallel configuration suggestions on simulated multi-node clusters.

Challenge input

JSON cluster spec containing GPU count, node count, interconnect bandwidth, and model size (70B parameters).

CrewAI

Multi-agent framework for collaborative role-playing simulation.

Evaluated output

JSON optimization layout specifying TP size, PP size, expected AllReduce latency, and throughput gain %.

Evaluation checks
  • Validates TP * PP equals total cluster GPU count.
  • Simulated latency improvement percentage over baseline naive TP=16. • target: 20 • range: 0-50
Configured evaluation evidence
  • Benchmark: Megatron-LM Distributed Benchmark
  • Latency Reduction Pct target: 20
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • CrewAI · Required
  • crewAI · Optional
  • Anthropic · Optional

Evaluation contract

  • gpu_topology_conservation · Weight 1
  • latency_reduction_pct · Weight 1

Metrics

  • latency_reduction_pct · Target 20

Benchmarks

  • Megatron-LM Distributed Benchmark
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON cluster spec containing GPU count, node count, interconnect bandwidth, and model size (70B parameters).

    Output: JSON optimization layout specifying TP size, PP size, expected AllReduce latency, and throughput gain %.

    Execution harness: Python

Evaluation tools

  • CrewAI · Agent Framework

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about CrewAI Distributed Interconnect & Tensor Parallelism Simulator