Back to challenges
Challenge

Long-Running Harness Capacity Planner with AutoGen and Kira Integration

Long-running software agents suffer from context bloating and hardware resource exhaustion over extended multi-turn sessions. You will construct a multi-agent system using AutoGen and Kira to monitor agent token context footprints, triggering context compaction and dynamic cluster autoscaling when memory bounds are breached. Success is evaluated on preserving 100% state accuracy while maintaining execution budget targets.

EngineeringHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Build an AutoGen multi-agent system integrated with Kira to plan, compress, and auto-scale hardware resources for long-running AI developers.

Delivery guide

How work is evaluated

Evaluation

Evaluates AutoGen state compression and autoscaling recommendations over a simulated 50-turn agent trajectory.

Datasets

Shared data for this challenge

Review public datasets and any private uploads tied to your build.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

fact_preservation

Ensures key system configuration facts are strictly preserved after context compaction

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

compression_ratio

Ratio of original context tokens to compacted context tokens • target: 4 • range: 2-10

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Implement AutoGen agents (Planner, Monitor, CompressionAgent) collaborating on execution state retention

  • Integrate Kira AI tooling for real-time telemetry extraction and system context compression

  • Define context retention heuristics that compress historical code traces without losing structural context

  • Validate node scaling triggers when worker GPU/TPU RAM utilization exceeds 85%

Resources and assets

Reference links and supporting material

Dataset notes

Execution histories of long-running agent developer harnesses containing terminal outputs and multi-file diffs.

How this agent runs

Evaluates AutoGen state compression and autoscaling recommendations over a simulated 50-turn agent trajectory.

Challenge input

JSON with session_turns array, current_vram_usage, and memory_cap_mb

AutoGen

Provides robust multi-agent interaction and role orchestration.

Kira

Enables detailed context inspection and trace analysis.

Evaluated output

JSON detailing final_token_count, compression_ratio, scaling_signal_emitted, and preserved_facts_count

Checks for
  • Ensures key system configuration facts are strictly preserved after context compaction
  • Ratio of original context tokens to compacted context tokens • target: 4 • range: 2-10
Proof of success
  • Benchmark: LongBench-Agent-State
  • Compression Ratio target: 4
  • 1 public reference case
Runtime evidence
  • Python execution harness
  • Python sandbox (unavailable on Versalist)
View technical recipe

Configured tools

Action Space
  • AutoGen · Required
  • Kira · Optional
  • Anthropic · Optional

Evaluation contract

  • fact_preservation · Weight 1
  • compression_ratio · Weight 1

Recipe state

This is a preview. The configuration can change before the evaluation recipe is locked.

Frequently Asked Questions about Long-Running Harness Capacity Planner with AutoGen and Kira Integration