Build a Spot Accelerator Preemption and Checkpoint Recovery Simulator

Simulate spot preemption events and KV-cache recovery costs across heterogeneous open-model workers. Plan migration schedules and evaluate resumption overhead using hosted Claude.

Distributed Training and Serving Architect

Model synthetic cloud accelerator spot termination notices with variable countdown windows.

Calculate uncommitted KV-cache flush times and checkpoint state transfer latencies across nodes.

Use hosted Claude API to synthesize a failover sequence that minimizes request dropped-token counts.

Produce an incident replay report detailing service degradation and cost recovery.

Chaos-engineering console with preemption trigger buttons and timeline reconstruction.

Simulated cloud termination event streams and distributed state snapshot metrics in test JSON. Starter examples below are synthetic. They are not downloaded records.

AWS Neuron documentation Models AWS EC2 spot lifecycle behavior and accelerated instance preemption intervals.

ray-serve, anthropic-claude, pydantic-ai. Provides deployment abstractions for distributed inference actors and replica state migration. Hosted Claude API for post-preemption recovery planning; strictly remote API.

Submit runnable source, pinned dependencies, setup commands, a working interface, example outputs, and a recorded end-to-end demo. Include tests and an action log.

Accurately simulates 120-second cloud spot preemption notice constraint.

Quantifies time required to serialize and migrate open-model serving context.

Formulates worker draining sequence that avoids hard connection drops for active users.

Outputs complete JSON recovery audit trail without invoking live cloud APIs.

```json [ { "input": { "preempted_node": "spot-worker-inf2-09", "notice_window_sec": 120, "active_requests": 8, "kv_cache_state_mb": 420, "interconnect_egress_mbps": 100, "backup_target_node": "on-demand-worker-inf2-01" }, "expected_output": { "transfer_duration_sec": 4.2, "migration_feasible": true, "action_sequence": [ "Mark spot-worker-inf2-09 as unschedulable in router.", "Flush 420MB active KV context to on-demand-worker-inf2-01 (4.2s estimated).", "Drain in-flight generation requests with remaining 115.8s buffer.", "Deregister spot worker instance prior to hardware termination." ], "dropped_request_count": 0 } }, { "input": { "preempted_node": "spot-worker-g5-02", "notice_window_sec": 15, "active_requests": 40, "kv_cache_state_mb": 2400, "interconnect_egress_mbps": 50, "backup_target_node": "on-demand-worker-g5-01" }, "expected_output": { "transfer_duration_sec": 48, "migration_feasible": false, "action_sequence": [ "Insufficient time for full context migration (48s required vs 15s window).", "Abort KV-cache serialization to preserve network egress.", "Reroute 40 active requests to fallback queue with prompt re-prefill.", "Issue emergency worker disconnect signal." ], "dropped_request_count": 40 } } ] ```

Simulates preemption logic without executing cloud VM termination commands. Use mock external writes. Require human review before any real deployment.