LangChain & AutoGen KV-Cache Multi-Tenant Capacity Planner
Build a multi-tenant KV-cache memory capacity manager using LangChain and AutoGen multi-agent simulation to analyze memory fragmentation under continuous batching. Implement dynamic cache reallocation algorithms that prevent out-of-memory errors and maintain sequence throughput above 850 tokens per second.
What you are building
The core problem, expected build, and operating context for this challenge.
Orchestrate dynamic KV-cache allocation under continuous batching using a hybrid LangChain and AutoGen multi-agent simulation framework.
How work is evaluated
Evaluates KV-cache planner accuracy under simulated bursty tenant workloads.
Shared data for this challenge
Review public datasets and any private uploads tied to your build.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
zero_oom_policy
Ensures no OOM events occurred during peak multi-tenant contention.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
throughput_tok_per_sec
Measures continuous token generation throughput. • target: 850 • range: 0-2000
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Build a LangChain-based KV-cache capacity planner for continuous batching runtimes
Simulate multi-tenant memory contention using AutoGen collaborative agent negotiation
Implement dynamic block-paged memory re-allocation algorithms to eliminate internal fragmentation
Achieve zero Out-Of-Memory (OOM) preemptions while sustaining 850+ tok/s output
Reference links and supporting material
Synthetic telemetry trace dataset representing multi-tenant LLM requests with dynamic sequence expansion and variable context lengths.
How this agent runs
Evaluates KV-cache planner accuracy under simulated bursty tenant workloads.
Challenge input
JSON array of incoming requests with start times, context lengths, and priority tiers.
LangChain
Provides structured tool abstractions for VRAM profiling.
Microsoft AutoGen
Simulates multi-tenant bargaining for memory allocation.
Evaluated output
JSON execution trace showing block allocations, swap events, and total throughput achieved.
- Ensures no OOM events occurred during peak multi-tenant contention.
- Measures continuous token generation throughput. • target: 850 • range: 0-2000
- Benchmark: vLLM Multi-Tenant Benchmark
- Throughput Tok Per Sec target: 850
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
View technical recipe
Configured tools
- Langchain · Required
- LangChain · Optional
- AutoGen · Optional
- Langchain · Required
- LangChain · Optional
Evaluation contract
- zero_oom_policy · Weight 1
- throughput_tok_per_sec · Weight 1
Recipe state
This is a preview. The configuration can change before the evaluation recipe is locked.