Back to challenges
Challenge

Interconnect-Aware Accelerator Profiler using Google ADK and AgentFlow

Build an interconnect-aware profiler using Google ADK and AgentFlow to model distributed inference workloads across AWS Trainium and NVIDIA H100 clusters. The system must predict total latency within a 5% error margin while dynamically selecting optimal tensor-parallel configurations.

EngineeringHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Utilize Google ADK and AgentFlow to model inter-node NVLink and AWS NeuronLink transfer latency for large models.

Delivery guide

How work is evaluated

Evaluation

Measures accuracy of predicted execution latency against baseline trace benchmarks across different tensor parallel topologies.

Datasets

Shared data for this challenge

Review public datasets and any private uploads tied to your build.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

Latency Accuracy Threshold

Error between predicted step time and ground truth must be <= 5%.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

Prediction MAPE

Mean Absolute Percentage Error across cluster layouts. • target: 0.02 • range: 0-0.05

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Build a multi-agent interconnect simulation pipeline using Google ADK and AgentFlow.

  • Profile network communication bottlenecks (AllReduce, AllGather) across H100 and Trainium2 topologies.

  • Select optimal Tensor Parallelism (TP) and Pipeline Parallelism (PP) split schemes automatically.

  • Benchmark model execution against measured baseline latency curves.

Resources and assets

Reference links and supporting material

Dataset notes

Network interconnect bandwidth matrices and tensor shapes for 8-node H100 and Trainium2 clusters.

How this agent runs

Measures accuracy of predicted execution latency against baseline trace benchmarks across different tensor parallel topologies.

Challenge input

JSON containing model architecture and cluster topology

Google ADK

Agent development kit for building structured multi-agent workflows

AgentFlow

Coordinates agent task flow for complex system modeling

Evaluated output

JSON containing predicted step_time_ms and recommended_tp

Checks for
  • Error between predicted step time and ground truth must be <= 5%.
  • Mean Absolute Percentage Error across cluster layouts. • target: 0.02 • range: 0-0.05
Proof of success
  • Benchmark: Distributed LLM Network Benchmark
  • Prediction MAPE target: 0.02
  • 1 public reference case
Runtime evidence
  • Python execution harness
  • Python sandbox (unavailable on Versalist)
View technical recipe

Configured tools

Action Space
  • AgentFlow · Required
  • Google · Optional
  • Google ADK · Optional

Evaluation contract

  • Latency Accuracy Threshold · Weight 1
  • Prediction MAPE · Weight 1

Recipe state

This is a preview. The configuration can change before the evaluation recipe is locked.

Frequently Asked Questions about Interconnect-Aware Accelerator Profiler using Google ADK and AgentFlow