Build a Multi-Tenant Inference Interruption and Fair-Share Rescheduler

Simulate heterogeneous preemption events across multi-tenant inference nodes. Hosted Claude analyzes token consumption history to re-dispatch interrupted workloads according to fair-share policies.

Platform Capacity Engineer

Simulate node capacity reductions and compute eviction events across multi-team model serving pools.

Aggregate historical per-tenant quota usage, current token queue depths, and job priority tags.

Send normalized cluster queue state to hosted Claude to compute fair-share re-balancing allocations.

Generate an updated dispatch plan ensuring non-preempted tenants maintain agreed minimum throughput.

Interactive console displaying tenant quota consumption and rescheduled inference queues.

Construct synthetic JSON traces recording tenant IDs, token requests, preemption events, and available worker nodes. Starter examples below are synthetic. They are not downloaded records.

AWS Neuron documentation Applies AWS Neuron SDK multi-accelerator capacity planning concepts across pooled inference resources.

optuna, anthropic, pydantic, pandas. Optuna optimizes multi-objective objective functions balancing fair-share tenant allocations and latency minimization. Hosted Claude API for interpreting policy violations and generating fair-share scheduling rationales.

Submit runnable source, pinned dependencies, setup commands, a working interface, example outputs, and a recorded end-to-end demo. Include tests and an action log.

Ensures no tenant receives zero allocations when cluster capacity exceeds total minimum guarantees.

Re-dispatches evicted jobs to active nodes without exceeding target node memory limits.

Flags tenants that have consumed over 120% of their historical hourly quota for deferral.

```json [ { "input": { "available_capacity_tokens_per_sec": 1000, "evicted_jobs": [ { "tenant_id": "team-search", "tokens_requested": 400, "quota_used_pct": 85 }, { "tenant_id": "team-analytics", "tokens_requested": 400, "quota_used_pct": 130 } ] }, "expected_output": { "allocated_jobs": [ { "tenant_id": "team-search", "allocated_tokens_per_sec": 400, "status": "APPROVED" } ], "deferred_jobs": [ { "tenant_id": "team-analytics", "reason": "QUOTA_EXCEEDED" } ], "unallocated_capacity": 600 } }, { "input": { "available_capacity_tokens_per_sec": 200, "evicted_jobs": [ { "tenant_id": "team-finance", "tokens_requested": 500, "quota_used_pct": 50 } ] }, "expected_output": { "allocated_jobs": [], "deferred_jobs": [ { "tenant_id": "team-finance", "reason": "INSUFFICIENT_CLUSTER_CAPACITY" } ], "unallocated_capacity": 200 } } ] ```

Simulation only. Does not alter production tenant quotas or terminate live cloud instances. Use mock external writes. Require human review before any real deployment.