Back to prompt library
Build a Multi-Tenant Token Budget and Latency Guardrail Sentinel — implementation
Linked challenge: Build a Multi-Tenant Token Budget and Latency Guardrail Sentinel
Prompt source
implementation65 lines11 sectionsNo variables1 code block
## Build
Monitor real-time client API token consumption and request durations across internal enterprise teams calling hosted Claude. Reconcile departmental budgets and formulate token quota adjustments.
## User
Enterprise AI Governance Manager
## Workflow
1. Ingest structured client proxy logs recording tenant headers, token tallies, and response latencies.
2. Reconcile daily token burn against allocated departmental monthly spending caps.
3. Detect abnormal burst token usage or runaway agent loops from specific tenant API keys.
4. Generate a Claude-assisted governance digest with suggested dynamic rate-limit thresholds.
## Interface
Executive compliance web dashboard showing tenant quota consumption and automated alerts.
## Data
Construct a synthetic JSONL log of 10,000 proxy calls representing 5 distinct enterprise business units.
Starter examples below are synthetic. They are not downloaded records.
## Source
[Anthropic Python SDK](https://raw.githubusercontent.com/anthropics/anthropic-sdk-python/main/README.md)
Apply Anthropic client message interaction models and token tracking fields from the official Python SDK.
## Tools
pydantic-ai, anthropic, sqlite3.
pydantic-ai parses multi-tenant budget configurations and guarantees validated governance rules.
Hosted Claude API accessed through the official Python SDK for anomaly narrative generation.
## Deliverables
Submit runnable source, pinned dependencies, setup commands, a working interface, example outputs, and a recorded end-to-end demo. Include tests and an action log.
## Acceptance
- Aggregates token consumption across multiple tenant API keys within configurable time windows.
- Flags tenant keys exceeding 90 percent of allocated budget or displaying runaway retry volume.
- Produces structured JSON budget enforcement actions for API gateway quota middleware.
## Starter Examples
```json
[
{
"input": {
"tenant_id": "customer-analytics",
"monthly_budget_usd": 500,
"mtd_spend_usd": 485.5,
"consecutive_error_spikes": 34,
"avg_latency_ms": 1850
},
"expected_output": {
"budget_status": "NEAR_EXHAUSTION",
"utilization_pct": 97.1,
"action": "THROTTLE_TIER_2",
"gateway_directive": "enforce_rate_limit_10_rpm"
}
},
{
"input": {
"tenant_id": "hr-operations",
"monthly_budget_usd": 200,
"mtd_spend_usd": 45,
"consecutive_error_spikes": 0,
"avg_latency_ms": 950
},
"expected_output": {
"budget_status": "NORMAL",
"utilization_pct": 22.5,
"action": "ALLOW",
"gateway_directive": "standard_quota"
}
}
]
```
## Safety
Monitors client-side usage data only without modifying Anthropic organization credentials or production firewalls.
Use mock external writes. Require human review before any real deployment.