Build a Multi-Tenant GPU Power Capping and Quota Manager

Analyze DCGM power draw telemetry across clusters to enforce multi-tenant energy budgets. Detect workloads operating below thermal design power and generate dynamic power cap adjustments for heterogeneous workloads using hosted Claude.

Datacenter Operations Engineer

Ingest normalized time-series DCGM power usage (Watts) and active tenant mappings per GPU.

Detect workloads operating below their thermal design power (TDP) that could safely run under a lowered power limit.

Invoke hosted Claude to analyze power efficiency profiles and suggest dynamic software power caps.

Export an executable bash script containing nvidia-smi commands to enforce the updated power budgets.

Command-line utility and reporting dashboard for cluster administrators to review and apply power budget adjustments per tenant.

Generate simplified synthetic app records in JSON modeling DCGM GPU metrics (average power draw, utilization percentages, tenant mappings) with varying workload intensity profiles. Starter examples below are synthetic. They are not downloaded records.

NVIDIA DCGM exporter References DCGM power metrics (power draw and utilization) as exposed by dcgm-exporter for accurate cluster profiling.

anthropic-claude, pandas, pydantic. anthropic-claude processes complex multi-metric power timelines to generate precise, natural-language rationales and actionable scripts. Hosted Claude API for power cap reasoning; local metric parsing utilities.

Submit runnable source, pinned dependencies, setup commands, a working interface, example outputs, and a recorded end-to-end demo. Include tests and an action log.

Correctly isolates average power consumption per tenant over the specified timeframe.

Produces safe power cap reductions only for memory-bound workloads not utilizing full SM power.

Ensures all adjustments respect the hardware's minimum and maximum allowed power limits.

Validates that output commands are syntactically correct.

```json [ { "input": { "node_id": "gpu-node-04", "gpu_max_power_w": 400, "gpu_min_power_w": 100, "telemetry": [ { "tenant": "nlp-batch", "gpu_id": 0, "avg_power_draw_w": 185, "utilization_pct": 98 }, { "tenant": "vision-train", "gpu_id": 1, "avg_power_draw_w": 380, "utilization_pct": 99 } ] }, "expected_output": { "power_optimizations_found": true, "proposed_caps": [ { "gpu_id": 0, "new_limit_w": 250, "rationale": "Memory-bound workload maxing at 185W; safe to cap at 250W to save node power budget." } ], "executable_script": "nvidia-smi -i 0 -pl 250" } }, { "input": { "node_id": "gpu-node-05", "gpu_max_power_w": 400, "gpu_min_power_w": 100, "telemetry": [ { "tenant": "rl-train", "gpu_id": 0, "avg_power_draw_w": 395, "utilization_pct": 100 } ] }, "expected_output": { "power_optimizations_found": false, "proposed_caps": [], "executable_script": "" } } ] ```

Avoids executing live server power adjustments directly, relying on generated scripts for manual administrator review. Use mock external writes. Require human review before any real deployment.