Long-Running Harness Capacity Planner with AutoGen and Kira Integration
Long-running software agents suffer from context bloating and hardware resource exhaustion over extended multi-turn sessions. You will construct a multi-agent system using AutoGen and Kira to monitor agent token context footprints, triggering context compaction and dynamic cluster autoscaling when memory bounds are breached. Success is evaluated on preserving 100% state accuracy while maintaining execution budget targets.
What you are building
The core problem, expected build, and operating context for this challenge.
Build an AutoGen multi-agent system integrated with Kira to plan, compress, and auto-scale hardware resources for long-running AI developers.
How work is evaluated
Evaluates AutoGen state compression and autoscaling recommendations over a simulated 50-turn agent trajectory.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
fact_preservation
Ensures key system configuration facts are strictly preserved after context compaction
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
compression_ratio
Ratio of original context tokens to compacted context tokens • target: 4 • range: 2-10
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Implement AutoGen agents (Planner, Monitor, CompressionAgent) collaborating on execution state retention
Integrate Kira AI tooling for real-time telemetry extraction and system context compression
Define context retention heuristics that compress historical code traces without losing structural context
Validate node scaling triggers when worker GPU/TPU RAM utilization exceeds 85%
Reference links and supporting material
Execution histories of long-running agent developer harnesses containing terminal outputs and multi-file diffs.
Challenge setup
Evaluates AutoGen state compression and autoscaling recommendations over a simulated 50-turn agent trajectory.
Challenge input
JSON with session_turns array, current_vram_usage, and memory_cap_mb
AutoGen
Provides robust multi-agent interaction and role orchestration.
Kira
Enables detailed context inspection and trace analysis.
Evaluated output
JSON detailing final_token_count, compression_ratio, scaling_signal_emitted, and preserved_facts_count
- Ensures key system configuration facts are strictly preserved after context compaction
- Ratio of original context tokens to compacted context tokens • target: 4 • range: 2-10
- Benchmark: LongBench-Agent-State
- Compression Ratio target: 4
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- AutoGen · Required
- Kira · Optional
- Anthropic · Optional
Evaluation contract
- fact_preservation · Weight 1
- compression_ratio · Weight 1
Metrics
- compression_ratio · Target 4
Benchmarks
- LongBench-Agent-State
- general
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON with session_turns array, current_vram_usage, and memory_cap_mb
Output: JSON detailing final_token_count, compression_ratio, scaling_signal_emitted, and preserved_facts_count
Execution harness: Python
Evaluation tools
- AutoGen · Agent Framework
- Kira · Engineering Intelligence Tool
Recipe freeze marker
No recipe freeze time is recorded.