Skip to main content
Back to challenges
Challenge

KV-Cache Capacity Planner Agent Mesh with CrewAI and Cursor

Develop a KV-cache memory capacity planner using CrewAI and Cursor to optimize page allocations for concurrent 128k context requests. Achieve zero out-of-memory preemption events across a 1,000-request benchmark while maintaining p95 prompt processing latency below 250ms.

EngineeringHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Orchestrate role-playing CrewAI agents developed in Cursor to manage vLLM-style paged KV-cache allocation rules.

Delivery guide

How work is evaluated

Evaluation

Evaluates OOM preemption rates and latency profiles of the KV-cache planner under continuous load.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

Zero Preemptions

Ensure zero requests are preempted due to out-of-memory conditions.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

P95 TTFT Latency

95th percentile latency for time-to-first-token in milliseconds. • target: 200 • range: 0-250

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Implement multi-agent role delegation using CrewAI for capacity planning.

  • Design dynamic block-allocation strategies for continuous long-context KV caches.

  • Prevent GPU memory fragmentation and request preemption under burst loads.

  • Measure impact on context swapping latency and time-to-first-token (TTFT).

Resources and assets

Reference links and supporting material

Dataset notes

Trace dataset containing context lengths and arrival times for 1,000 synthetic agentic interactions.

Challenge setup

Evaluates OOM preemption rates and latency profiles of the KV-cache planner under continuous load.

Challenge input

JSON payload containing request arrival rates and context lengths

CrewAI

Orchestrates dedicated role-playing agents for capacity management

Cursor

Accelerates development and refactoring of complex system tools

Evaluated output

JSON containing preemption_count and p95_ttft_ms

Evaluation checks
  • Ensure zero requests are preempted due to out-of-memory conditions.
  • 95th percentile latency for time-to-first-token in milliseconds. • target: 200 • range: 0-250
Configured evaluation evidence
  • Benchmark: vLLM Memory Management Benchmark
  • P95 TTFT Latency target: 200
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • CrewAI · Required
  • Cursor · Optional
  • crewAI · Optional
Observation
  • Cursor · Optional

Evaluation contract

  • Zero Preemptions · Weight 1
  • P95 TTFT Latency · Weight 1

Metrics

  • P95 TTFT Latency · Target 200

Benchmarks

  • vLLM Memory Management Benchmark
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON payload containing request arrival rates and context lengths

    Output: JSON containing preemption_count and p95_ttft_ms

    Execution harness: Python

Evaluation tools

  • CrewAI · Agent Framework
  • Cursor · AI Engineering Tooling

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about KV-Cache Capacity Planner Agent Mesh with CrewAI and Cursor