Build an Inference Job Interruption Replay and Rescheduling Simulator

Simulate spot instance preemption events across heterogeneous open-weight LLM inference workers. Reconstruct KV-cache eviction checkpoints and evaluate job replay schedules to minimize completion delays.

Inference Platform Reliability Engineer

Ingest simulated spot preemption events and active open-weight inference request queues.

Compute KV-cache recovery penalties based on vLLM PagedAttention checkpoint intervals.

Prompt hosted Claude to evaluate priority rescheduling alternatives and migration sequences.

Render simulated recovery execution timelines and generate operator restart schedules.

Web dashboard displaying preemption events, KV-cache reconstruction overhead, and rescheduled job queues with manual override buttons.

Generate synthetic JSON event logs representing spot instance preemptions, active inference job states, and open-weight model parameter sizes. Starter examples below are synthetic. They are not downloaded records.

vLLM open-model serving Applies vLLM PagedAttention and continuous batching concepts to calculate KV-cache memory restoration overhead during simulated inference node recovery.

pydantic-ai, anthropic-claude, ray-serve. Ensures strict type-safe validation of job state schemas, migration timelines, and KV-cache recovery parameters. Hosted Claude API for reasoning over migration tradeoffs and rescheduling logic. Local open-weight profiling only.

Submit runnable source, pinned dependencies, setup commands, a working interface, example outputs, and a recorded end-to-end demo. Include tests and an action log.

Validates preemption log payloads against Pydantic models with zero validation errors.

Calculates simulated KV-cache restoration latency within 5% of open-weight benchmark formulas.

Synthesizes an operator rescheduling plan with deterministic fallback for preempted nodes.

```json [ { "input": { "preemption_event": { "node_id": "worker-gpu-04", "timestamp": "2025-03-30T10:00:00Z", "drain_seconds": 30 }, "active_jobs": [ { "job_id": "inf-101", "model": "Qwen-7B", "tokens_generated": 512, "kv_cache_blocks": 32, "sla_seconds": 12 } ] }, "expected_output": { "reschedule_decision": "migrate_immediate", "target_node": "worker-gpu-08", "kv_cache_recompute_penalty_ms": 420, "estimated_completion_delay_sec": 1.8, "status": "scheduled" } }, { "input": { "preemption_event": { "node_id": "worker-gpu-02", "timestamp": "2025-03-30T10:05:00Z", "drain_seconds": 0 }, "active_jobs": [] }, "expected_output": { "reschedule_decision": "none_idle_node", "target_node": null, "kv_cache_recompute_penalty_ms": 0, "estimated_completion_delay_sec": 0, "status": "drained" } } ] ```

Simulates rescheduling workflows only; does not send commands to live cloud instances or modify production cluster nodes. Use mock external writes. Require human review before any real deployment.