Build a Thermal Throttling Incident Playbook and Workload Evacuation Planner
Monitor continuous temperature telemetry from DCGM metrics to detect localized hot spots. Plan progressive workload evacuation from overheating chassis to prevent ungraceful job failures.
What you are building
Monitor continuous temperature telemetry from DCGM metrics to detect localized hot spots. Plan progressive workload evacuation from overheating chassis to prevent ungraceful job failures.
HPC Systems Administrator
Process simulated DCGM thermal and power telemetry streams across rack enclosures.
Detect persistent thermal spikes exceeding safe operating margins.
Prompt hosted Claude to synthesize evacuation playbooks and rank recipient target nodes.
Construct an executable migration plan with container checkpointing checkpoints.
Terminal dashboard showing rack-level thermal contours and interactive workload evacuation action cards.
Generate synthetic multi-node DCGM metrics including GPU temperature, memory temperature, and power draw under load. Starter examples below are synthetic. They are not downloaded records.
NVIDIA DCGM exporter Extracts DCGM thermal metric structures and device status flags to identify hardware cooling failure modes.
anthropic-claude, pydantic-ai, mlflow. Reasons over complex thermal incident progressions and formulates prioritized workload draining instructions. Hosted Claude API for synthesizing evacuation playbooks and assessing cooling risk. Local models restricted to open weights.
Submit runnable source, pinned dependencies, setup commands, a working interface, example outputs, and a recorded end-to-end demo. Include tests and an action log.
Detects thermal anomaly patterns 5 minutes prior to simulated critical thermal shutdown.
Generates evacuation plans ordering jobs by SLA criticality and checkpoint overhead.
Validates target node thermal headroom before assigning migrated workloads.
```json [ { "input": { "chassis_id": "chassis-08", "sensors": { "intake_temp_c": 34.5, "gpu0_temp_c": 88, "gpu1_temp_c": 89.5 }, "active_tasks": [ { "task_id": "train-job-4", "gpu_count": 2, "priority": "high", "checkpoint_ready": true } ] }, "expected_output": { "evacuation_required": true, "urgency": "immediate", "target_chassis": "chassis-02", "evacuation_sequence": [ "Pause task train-job-4", "Commit state checkpoint", "Drain chassis-08", "Resume on chassis-02" ], "operator_alert": "Suspected chassis intake airflow restriction" } }, { "input": { "chassis_id": "chassis-01", "sensors": { "intake_temp_c": 22, "gpu0_temp_c": 62, "gpu1_temp_c": 64 }, "active_tasks": [ { "task_id": "inference-prod", "gpu_count": 1, "priority": "critical", "checkpoint_ready": false } ] }, "expected_output": { "evacuation_required": false, "urgency": "nominal", "target_chassis": null, "evacuation_sequence": [], "operator_alert": "Normal thermal envelope" } } ] ```
Simulation and guidance only; does not send shutdown signals or adjust physical chiller controls. Use mock external writes. Require human review before any real deployment.
Rules
Simulation and guidance only; does not send shutdown signals or adjust physical chiller controls. Use mock external writes and human review.
How work is evaluated
Detects thermal anomaly patterns 5 minutes prior to simulated critical thermal shutdown. Generates evacuation plans ordering jobs by SLA criticality and checkpoint overhead. Validates target node thermal headroom before assigning migrated workloads.
Runnable source, pinned dependencies, setup commands, working interface, example outputs, recorded demo, tests, and action logs.
Datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
Complete workflow
Process simulated DCGM thermal and power telemetry streams across rack enclosures. Detect persistent thermal spikes exceeding safe operating margins. Prompt hosted Claude to synthesize evacuation playbooks and rank recipient target nodes. Construct an executable migration plan with container checkpointing checkpoints.
Missing or incorrect.
Partly complete, with documented gaps.
Complete, with reproducible evidence.
Correctness and evidence
Detects thermal anomaly patterns 5 minutes prior to simulated critical thermal shutdown. Generates evacuation plans ordering jobs by SLA criticality and checkpoint overhead. Validates target node thermal headroom before assigning migrated workloads.
Missing or incorrect.
Partly complete, with documented gaps.
Complete, with reproducible evidence.
Review interface
Terminal dashboard showing rack-level thermal contours and interactive workload evacuation action cards.
Missing or incorrect.
Partly complete, with documented gaps.
Complete, with reproducible evidence.
Safe execution
Simulation and guidance only; does not send shutdown signals or adjust physical chiller controls.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Learning goals
Process simulated DCGM thermal and power telemetry streams across rack enclosures.
Detect persistent thermal spikes exceeding safe operating margins.
Prompt hosted Claude to synthesize evacuation playbooks and rank recipient target nodes.
Construct an executable migration plan with container checkpointing checkpoints.
Resources
Generate synthetic multi-node DCGM metrics including GPU temperature, memory temperature, and power draw under load. Supplied starter examples are synthetic, not source extracts.
Challenge setup
Challenge input
Input details are not configured.
Anthropic Claude
LLM for code generation & more
Evaluated output
No output checks are configured.
- Process simulated DCGM thermal and power telemetry streams across rack enclosures. Detect persistent therma...
- Detects thermal anomaly patterns 5 minutes prior to simulated critical thermal shutdown. Generates evacuati...
- Terminal dashboard showing rack-level thermal contours and interactive workload evacuation action cards.
- 2 public reference cases
- 1 configured tool
Shows the first task, up to three tools, and up to three evidence items.
View technical recipe
Configured tools
- Anthropic Claude · Optional
Evaluation contract
- Complete workflow · Weight 40
- Correctness and evidence · Weight 30
- Review interface · Weight 20
- Safe execution · Weight 10
Reference cases
- 2 public reference cases
Workflow tasks
No workflow tasks are configured.
Recipe freeze marker
No recipe freeze time is recorded.