Build the application

## Build
Simulate spot instance preemption events across heterogeneous open-weight LLM inference workers. Reconstruct KV-cache eviction checkpoints and evaluate job replay schedules to minimize completion delays.

## User
Inference Platform Reliability Engineer

## Workflow
1. Ingest simulated spot preemption events and active open-weight inference request queues.
2. Compute KV-cache recovery penalties based on vLLM PagedAttention checkpoint intervals.
3. Prompt hosted Claude to evaluate priority rescheduling alternatives and migration sequences.
4. Render simulated recovery execution timelines and generate operator restart schedules.

## Interface
Web dashboard displaying preemption events, KV-cache reconstruction overhead, and rescheduled job queues with manual override buttons.

## Data
Generate synthetic JSON event logs representing spot instance preemptions, active inference job states, and open-weight model parameter sizes.
Starter examples below are synthetic. They are not downloaded records.

## Source
[vLLM open-model serving](https://raw.githubusercontent.com/vllm-project/vllm/main/README.md)
Applies vLLM PagedAttention and continuous batching concepts to calculate KV-cache memory restoration overhead during simulated inference node recovery.

## Tools
pydantic-ai, anthropic-claude, ray-serve.
Ensures strict type-safe validation of job state schemas, migration timelines, and KV-cache recovery parameters.
Hosted Claude API for reasoning over migration tradeoffs and rescheduling logic. Local open-weight profiling only.

## Deliverables
Submit runnable source, pinned dependencies, setup commands, a working interface, example outputs, and a recorded end-to-end demo. Include tests and an action log.

## Acceptance
- Validates preemption log payloads against Pydantic models with zero validation errors.
- Calculates simulated KV-cache restoration latency within 5% of open-weight benchmark formulas.
- Synthesizes an operator rescheduling plan with deterministic fallback for preempted nodes.

## Starter Examples
```json
[
  {
    "input": {
      "preemption_event": {
        "node_id": "worker-gpu-04",
        "timestamp": "2025-03-30T10:00:00Z",
        "drain_seconds": 30
      },
      "active_jobs": [
        {
          "job_id": "inf-101",
          "model": "Qwen-7B",
          "tokens_generated": 512,
          "kv_cache_blocks": 32,
          "sla_seconds": 12
        }
      ]
    },
    "expected_output": {
      "reschedule_decision": "migrate_immediate",
      "target_node": "worker-gpu-08",
      "kv_cache_recompute_penalty_ms": 420,
      "estimated_completion_delay_sec": 1.8,
      "status": "scheduled"
    }
  },
  {
    "input": {
      "preemption_event": {
        "node_id": "worker-gpu-02",
        "timestamp": "2025-03-30T10:05:00Z",
        "drain_seconds": 0
      },
      "active_jobs": []
    },
    "expected_output": {
      "reschedule_decision": "none_idle_node",
      "target_node": null,
      "kv_cache_recompute_penalty_ms": 0,
      "estimated_completion_delay_sec": 0,
      "status": "drained"
    }
  }
]
```

## Safety
Simulates rescheduling workflows only; does not send commands to live cloud instances or modify production cluster nodes.
Use mock external writes. Require human review before any real deployment.