Build a Cloud Accelerator Preemption and Job Rescheduling Replayer

Simulate spot preemption events and reconstruct KV-cache checkpointing costs for open-weight serving nodes. Formulate dynamic job migration and re-computation schedules to preserve inference SLAs.

Distributed Systems Reliability Engineer

Ingest simulated spot preemption notification timestamps across heterogeneous cloud worker instances.

Calculate KV-cache serialization state and transfer overhead to designated backup workers.

Replay request backlog to estimate latency penalties under varying re-computation strategies.

Invoke hosted Claude via Anthropic API to draft a preemption resiliency runbook.

Interactive preemption chaos simulator UI with worker state diagrams and SLA recovery timelines.

Construct synthetic cluster state records modeling 10 spot GPU worker preemption events over a 24-hour cycle. Starter examples below are synthetic. They are not downloaded records.

vLLM open-model serving Reference vLLM distributed inference parallelism models and KV-cache state preservation concepts.

ray-serve, anthropic, pydantic. ray-serve models multi-worker replica failover topologies and request re-routing state. Hosted Claude API for runbook generation and risk synthesis; simulation targets open-model serving logic.

Submit runnable source, pinned dependencies, setup commands, a working interface, example outputs, and a recorded end-to-end demo. Include tests and an action log.

Models the exact latency tradeoff between cold-restart re-prefill versus snapshot KV-cache loading.

Identifies cluster failover capacity deficits that risk dropping client queries.

Exports structured migration schedules that maintain service availability above 99 percent.

```json [ { "input": { "preempted_node": "worker-spot-08", "active_sessions": 24, "kv_cache_size_mb": 4200, "network_transfer_mbps": 1000, "target_node": "worker-spot-02", "target_node_available_ram_mb": 8000 }, "expected_output": { "migration_strategy": "transfer_cache_snapshot", "transfer_time_sec": 4.2, "sla_breach": false, "target_admissibility": "APPROVED" } }, { "input": { "preempted_node": "worker-spot-03", "active_sessions": 48, "kv_cache_size_mb": 12000, "network_transfer_mbps": 500, "target_node": "worker-spot-04", "target_node_available_ram_mb": 4000 }, "expected_output": { "migration_strategy": "evict_and_cold_recompute", "transfer_time_sec": null, "sla_breach": true, "target_admissibility": "REJECTED_INSUFFICIENT_MEMORY" } } ] ```

Simulates cluster fault tolerance and failover policies purely in software without terminating active cloud nodes. Use mock external writes. Require human review before any real deployment.