Skip to main content
Back to challenges
Challenge

CrewAI & Agno Long-Context KV-Cache Capacity Planner

Serving long-context models causes severe KV-cache memory pressure, leading to out-of-memory errors and poor request throughput. Orchestrate CrewAI agents with Agno runtime monitoring to calculate PagedAttention KV-cache requirements and dynamically adjust admission control rules for 1M+ context windows.

EngineeringHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Utilize CrewAI and Agno to design an automated KV-cache memory manager with PagedAttention budgeting and dynamic request admission control.

Delivery guide

How work is evaluated

Evaluation

Tests KV-cache budget calculations and admission control decision validity under high memory stress.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

memory_bound_check

Checks that max concurrency does not exceed total available VRAM.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

capacity_utilization_ratio

Optimal utilization of VRAM without triggering OOM • target: 0.92 • range: 0.8-0.98

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Model KV-cache memory overhead per layer and head for FP16 and FP8 precision

  • Implement CrewAI role-playing agents for capacity planning, cache eviction, and load prediction

  • Integrate Agno agent system for low-latency operational telemetry collection

  • Build dynamic admission control algorithms that prevent OOM events under 1M context spikes

Resources and assets

Reference links and supporting material

Dataset notes

Synthetic load patterns with varying sequence lengths (1k to 1M tokens) and concurrency requests.

Challenge setup

Tests KV-cache budget calculations and admission control decision validity under high memory stress.

Challenge input

JSON containing gpu_memory_gb, num_layers, num_heads, num_kv_heads, head_dim, active_requests

CrewAI

Role-based agent orchestration for system planning

Agno

Lightweight system tracking and operational state

Evaluated output

JSON containing max_concurrency, required_kv_cache_gb, admission_decision

Evaluation checks
  • Checks that max concurrency does not exceed total available VRAM.
  • Optimal utilization of VRAM without triggering OOM • target: 0.92 • range: 0.8-0.98
Configured evaluation evidence
  • Benchmark: vLLM Production Memory Benchmark
  • Capacity Utilization Ratio target: 0.92
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • CrewAI · Required
  • Agno · Optional
  • crewAI · Optional
Orchestration
  • Agno · Optional

Evaluation contract

  • memory_bound_check · Weight 1
  • capacity_utilization_ratio · Weight 1

Metrics

  • capacity_utilization_ratio · Target 0.92

Benchmarks

  • vLLM Production Memory Benchmark
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON containing gpu_memory_gb, num_layers, num_heads, num_kv_heads, head_dim, active_requests

    Output: JSON containing max_concurrency, required_kv_cache_gb, admission_decision

    Execution harness: Python

Evaluation tools

  • CrewAI · Agent Framework
  • Agno · Real-time Telemetry Agent

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about CrewAI & Agno Long-Context KV-Cache Capacity Planner