Skip to main content
Challenge

Build a Multi-Tenant GPU Reservation and Chargeback Reconciler

Reconcile scheduled multi-team GPU reservation windows against active DCGM telemetry metrics to quantify idle waste. Synthesize chargeback adjustments and capacity release proposals for infrastructure finance teams.

EngineeringHosted by Vera
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Reconcile scheduled multi-team GPU reservation windows against active DCGM telemetry metrics to quantify idle waste. Synthesize chargeback adjustments and capacity release proposals for infrastructure finance teams.

Infrastructure Operations and FinOps Lead

Ingest scheduled team GPU reservation windows alongside normalized Prometheus DCGM exporter time-series telemetry records.

Calculate active execution ratios, compute memory residency, and unallocated idle hours across individual accelerator slots.

Invoke hosted Claude via Anthropic API to analyze underutilization patterns and generate policy-compliant refund chargeback adjustments.

Publish structured accounting reconciliation ledgers with itemized reclamation recommendations for engineering team review.

Web FinOps console featuring reservation timeline diffs, utilization heatmaps, simulated credit adjustment vouchers, and exportable reconciliation reports.

Generate 40 normalized synthetic DCGM exporter time-series records covering 8 GPU nodes alongside reservation schedules in JSON format. Starter examples below are synthetic. They are not downloaded records.

NVIDIA DCGM exporter Extracts GPU utilization and memory metrics exposed by NVIDIA DCGM exporter in Prometheus text format to determine active vs idle state.

mlflow, python-dateutil. MLflow provides native tracking and experiment artifact logging to audit FinOps reconciliation runs across compute pools. Hosted Claude API for telemetry reconciliation reasoning and accounting summaries. Local hardware executing open workload accounting logic only.

Submit runnable source, pinned dependencies, setup commands, a working interface, example outputs, and a recorded end-to-end demo. Include tests and an action log.

Identifies active vs idle GPU allocation windows with zero missing metric intervals in test datasets.

Computes team chargeback penalty adjustments matching predefined contract accounting formulas.

Outputs valid structured JSON reconciliation ledgers including team IDs, idle credits, and recommended quota adjustments.

```json [ { "input": { "team_id": "nlp-research", "reservation_hours": 24, "allocated_gpus": 8, "hourly_rate": 3.5, "dcgm_active_utilization_pct": [ 12, 10, 8, 15, 9, 11 ] }, "expected_output": { "average_utilization_pct": 10.83, "underutilized_hours": 24, "idle_waste_cost": 599.76, "chargeback_adjustment_status": "CREDIT_RECOMMENDED", "suggested_quota_reduction_gpus": 6 } }, { "input": { "team_id": "speech-prod", "reservation_hours": 0, "allocated_gpus": 4, "hourly_rate": 3.5, "dcgm_active_utilization_pct": [] }, "expected_output": { "average_utilization_pct": 0, "underutilized_hours": 0, "idle_waste_cost": 0, "chargeback_adjustment_status": "SKIPPED_EMPTY_WINDOW", "suggested_quota_reduction_gpus": 0 } } ] ```

Simulates financial cost adjustments and capacity changes. Requires explicit FinOps administrator sign-off before modifying enterprise cloud billing systems. Use mock external writes. Require human review before any real deployment.

Rules

Participation guide

Simulates financial cost adjustments and capacity changes. Requires explicit FinOps administrator sign-off before modifying enterprise cloud billing systems. Use mock external writes and human review.

Delivery guide

How work is evaluated

Evaluation

Identifies active vs idle GPU allocation windows with zero missing metric intervals in test datasets. Computes team chargeback penalty adjustments matching predefined contract accounting formulas. Outputs valid structured JSON reconciliation ledgers including team IDs, idle credits, and recommended quota adjustments.

Submission format

Runnable source, pinned dependencies, setup commands, working interface, example outputs, recorded demo, tests, and action logs.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
4 scoring checks
Binary
1 pass or fail dimensions
Ordinal
3 scaled dimensions
Dimension 1

Complete workflow

Ingest scheduled team GPU reservation windows alongside normalized Prometheus DCGM exporter time-series telemetry records. Calculate active execution ratios, compute memory residency, and unallocated idle hours across individual accelerator slots. Invoke hosted Claude via Anthropic API to analyze underutilization patterns and generate policy-compliant refund chargeback adjustments. Publish structured accounting reconciliation ledgers with itemized reclamation recommendations for engineering team review.

Score scale
012
Score 0

Missing or incorrect.

Score 1

Partly complete, with documented gaps.

Score 2

Complete, with reproducible evidence.

Dimension 2

Correctness and evidence

Identifies active vs idle GPU allocation windows with zero missing metric intervals in test datasets. Computes team chargeback penalty adjustments matching predefined contract accounting formulas. Outputs valid structured JSON reconciliation ledgers including team IDs, idle credits, and recommended quota adjustments.

Score scale
012
Score 0

Missing or incorrect.

Score 1

Partly complete, with documented gaps.

Score 2

Complete, with reproducible evidence.

Dimension 3

Review interface

Web FinOps console featuring reservation timeline diffs, utilization heatmaps, simulated credit adjustment vouchers, and exportable reconciliation reports.

Score scale
012
Score 0

Missing or incorrect.

Score 1

Partly complete, with documented gaps.

Score 2

Complete, with reproducible evidence.

Dimension 4

Safe execution

Simulates financial cost adjustments and capacity changes. Requires explicit FinOps administrator sign-off before modifying enterprise cloud billing systems.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Ingest scheduled team GPU reservation windows alongside normalized Prometheus DCGM exporter time-series telemetry records.

  • Calculate active execution ratios, compute memory residency, and unallocated idle hours across individual accelerator slots.

  • Invoke hosted Claude via Anthropic API to analyze underutilization patterns and generate policy-compliant refund chargeback adjustments.

  • Publish structured accounting reconciliation ledgers with itemized reclamation recommendations for engineering team review.

Resources and assets

Reference links and supporting material

Dataset notes

Generate 40 normalized synthetic DCGM exporter time-series records covering 8 GPU nodes alongside reservation schedules in JSON format. Supplied starter examples are synthetic, not source extracts.

Challenge setup

Configuration summary for this challenge.

Challenge input

Input details are not configured.

MLflow

MLOps

Evaluated output

No output checks are configured.

Evaluation checks
  • Ingest scheduled team GPU reservation windows alongside normalized Prometheus DCGM exporter time-series tel...
  • Identifies active vs idle GPU allocation windows with zero missing metric intervals in test datasets. Compu...
  • Web FinOps console featuring reservation timeline diffs, utilization heatmaps, simulated credit adjustment...
Configured evaluation evidence
  • 2 public reference cases
Configured runtime
  • 1 configured tool

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • MLflow · Optional

Evaluation contract

  • Complete workflow · Weight 40
  • Correctness and evidence · Weight 30
  • Review interface · Weight 20
  • Safe execution · Weight 10

Reference cases

  • 2 public reference cases

Workflow tasks

No workflow tasks are configured.

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Build a Multi-Tenant GPU Reservation and Chargeback Reconciler