Build a Multi-Tenant Inference Interruption and Fair-Share Rescheduler — implementation

## Build
Simulate heterogeneous preemption events across multi-tenant inference nodes. Hosted Claude analyzes token consumption history to re-dispatch interrupted workloads according to fair-share policies.

## User
Platform Capacity Engineer

## Workflow
1. Simulate node capacity reductions and compute eviction events across multi-team model serving pools.
2. Aggregate historical per-tenant quota usage, current token queue depths, and job priority tags.
3. Send normalized cluster queue state to hosted Claude to compute fair-share re-balancing allocations.
4. Generate an updated dispatch plan ensuring non-preempted tenants maintain agreed minimum throughput.

## Interface
Interactive console displaying tenant quota consumption and rescheduled inference queues.

## Data
Construct synthetic JSON traces recording tenant IDs, token requests, preemption events, and available worker nodes.
Starter examples below are synthetic. They are not downloaded records.

## Source
[AWS Neuron documentation](https://raw.githubusercontent.com/aws-neuron/aws-neuron-sdk/master/README.md)
Applies AWS Neuron SDK multi-accelerator capacity planning concepts across pooled inference resources.

## Tools
optuna, anthropic, pydantic, pandas.
Optuna optimizes multi-objective objective functions balancing fair-share tenant allocations and latency minimization.
Hosted Claude API for interpreting policy violations and generating fair-share scheduling rationales.

## Deliverables
Submit runnable source, pinned dependencies, setup commands, a working interface, example outputs, and a recorded end-to-end demo. Include tests and an action log.

## Acceptance
- Ensures no tenant receives zero allocations when cluster capacity exceeds total minimum guarantees.
- Re-dispatches evicted jobs to active nodes without exceeding target node memory limits.
- Flags tenants that have consumed over 120% of their historical hourly quota for deferral.

## Starter Examples
```json
[
  {
    "input": {
      "available_capacity_tokens_per_sec": 1000,
      "evicted_jobs": [
        {
          "tenant_id": "team-search",
          "tokens_requested": 400,
          "quota_used_pct": 85
        },
        {
          "tenant_id": "team-analytics",
          "tokens_requested": 400,
          "quota_used_pct": 130
        }
      ]
    },
    "expected_output": {
      "allocated_jobs": [
        {
          "tenant_id": "team-search",
          "allocated_tokens_per_sec": 400,
          "status": "APPROVED"
        }
      ],
      "deferred_jobs": [
        {
          "tenant_id": "team-analytics",
          "reason": "QUOTA_EXCEEDED"
        }
      ],
      "unallocated_capacity": 600
    }
  },
  {
    "input": {
      "available_capacity_tokens_per_sec": 200,
      "evicted_jobs": [
        {
          "tenant_id": "team-finance",
          "tokens_requested": 500,
          "quota_used_pct": 50
        }
      ]
    },
    "expected_output": {
      "allocated_jobs": [],
      "deferred_jobs": [
        {
          "tenant_id": "team-finance",
          "reason": "INSUFFICIENT_CLUSTER_CAPACITY"
        }
      ],
      "unallocated_capacity": 200
    }
  }
]
```

## Safety
Simulation only. Does not alter production tenant quotas or terminate live cloud instances.
Use mock external writes. Require human review before any real deployment.