CrewAI & Agno Long-Context KV-Cache Capacity Planner
Serving long-context models causes severe KV-cache memory pressure, leading to out-of-memory errors and poor request throughput. Orchestrate CrewAI agents with Agno runtime monitoring to calculate PagedAttention KV-cache requirements and dynamically adjust admission control rules for 1M+ context windows.
What you are building
Utilize CrewAI and Agno to design an automated KV-cache memory manager with PagedAttention budgeting and dynamic request admission control.
How work is evaluated
Tests KV-cache budget calculations and admission control decision validity under high memory stress.
Datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
memory_bound_check
Checks that max concurrency does not exceed total available VRAM.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
capacity_utilization_ratio
Optimal utilization of VRAM without triggering OOM • target: 0.92 • range: 0.8-0.98
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Learning goals
Model KV-cache memory overhead per layer and head for FP16 and FP8 precision
Implement CrewAI role-playing agents for capacity planning, cache eviction, and load prediction
Integrate Agno agent system for low-latency operational telemetry collection
Build dynamic admission control algorithms that prevent OOM events under 1M context spikes
Resources
Synthetic load patterns with varying sequence lengths (1k to 1M tokens) and concurrency requests.
Challenge setup
Tests KV-cache budget calculations and admission control decision validity under high memory stress.
Challenge input
JSON containing gpu_memory_gb, num_layers, num_heads, num_kv_heads, head_dim, active_requests
CrewAI
Role-based agent orchestration for system planning
Agno
Lightweight system tracking and operational state
Evaluated output
JSON containing max_concurrency, required_kv_cache_gb, admission_decision
- Checks that max concurrency does not exceed total available VRAM.
- Optimal utilization of VRAM without triggering OOM • target: 0.92 • range: 0.8-0.98
- Benchmark: vLLM Production Memory Benchmark
- Capacity Utilization Ratio target: 0.92
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Shows the first task, up to three tools, and up to three evidence items.
View technical recipe
Configured tools
- CrewAI · Required
- Agno · Optional
- crewAI · Optional
- Agno · Optional
Evaluation contract
- memory_bound_check · Weight 1
- capacity_utilization_ratio · Weight 1
Metrics
- capacity_utilization_ratio · Target 0.92
Benchmarks
- vLLM Production Memory Benchmark
- general
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON containing gpu_memory_gb, num_layers, num_heads, num_kv_heads, head_dim, active_requests
Output: JSON containing max_concurrency, required_kv_cache_gb, admission_decision
Execution harness: Python
Evaluation tools
- CrewAI · Agent Framework
- Agno · Real-time Telemetry Agent
Recipe freeze marker
No recipe freeze time is recorded.