Build an Enterprise AI Token Cost Governance System using AutoGen
As Indian enterprises struggle with runaway AI model costs, effective observability and cost control engines become vital. In this challenge, you will construct a multi-agent conversation and cost governance system using AutoGen. The system features conversational agents (Cost Auditing Agent, Model Router, and Rate Limit Policy Agent) that dynamically inspect incoming LLM requests, project token consumption costs, down-route queries to smaller models when appropriate, and enforce hard budget policies across enterprise departments.
What you are building
The core problem, expected build, and operating context for this challenge.
Develop an enterprise AI token cost control engine using multi-agent AutoGen conversations to auto-throttle and route queries.
How work is evaluated
Evaluates prompt routing decisions and token cost savings against benchmark queries.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
economical_routing_test
Ensures simple translation prompt is routed to a lightweight model tier.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
cost_reduction_ratio
Percentage savings compared to unrouted baseline. • target: 0.6 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Orchestrate conversational multi-agent workflows using Microsoft AutoGen.
Implement dynamic LLM request semantic complexity scoring to select cheapest viable model.
Track per-department token burn rate against preset daily micro-budgets.
Generate automated cost reduction reports and anomaly alerts.
Reference links and supporting material
Enterprise prompt history log dataset annotated with token counts and task complexity scores.
Challenge setup
Evaluates prompt routing decisions and token cost savings against benchmark queries.
Challenge input
JSON with user_prompt, department, remaining_daily_budget_usd.
AutoGen
Required framework for conversational multi-agent systems.
Evaluated output
JSON with assigned_model, estimated_cost_usd, budget_status.
- Ensures simple translation prompt is routed to a lightweight model tier.
- Percentage savings compared to unrouted baseline. • target: 0.6 • range: 0-1
- Benchmark: LLM Efficiency Benchmark
- Cost Reduction Ratio target: 0.6
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- AutoGen · Required
- Aim · Optional
- Hamming · Optional
Evaluation contract
- economical_routing_test · Weight 1
- cost_reduction_ratio · Weight 1
Metrics
- cost_reduction_ratio · Target 0.6
Benchmarks
- LLM Efficiency Benchmark
- general
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON with user_prompt, department, remaining_daily_budget_usd.
Output: JSON with assigned_model, estimated_cost_usd, budget_status.
Execution harness: Python
Evaluation tools
- AutoGen · Multi-Agent Framework
Recipe freeze marker
No recipe freeze time is recorded.