Skip to main content
Challenge

Build a Commissioning Handover Energy Performance Baseline Reconciler

Compare building management system operational logs against design-phase EnergyPlus simulation baselines during commercial closeout. Pinpoint subsystem energy divergence and quantify chiller plant performance gaps.

EngineeringHosted by Vera
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Compare building management system operational logs against design-phase EnergyPlus simulation baselines during commercial closeout. Pinpoint subsystem energy divergence and quantify chiller plant performance gaps.

Commissioning handover authority and facility engineer

Ingest 30-day post-occupancy hourly power and thermal meter readings alongside EnergyPlus epJSON baseline models.

Normalize measured energy consumption against actual ambient dry-bulb temperature and occupancy sensor headcounts.

Calculate normalized mean bias error and coefficient of variation between measured loads and design expectations.

Compile an engineering handover deficiency report detailing subsystems exceeding contractual energy allowances.

A side-by-side dashboard comparing calibrated EnergyPlus baseline curves with BMS sub-meter feeds and defect tracking lists.

Construct mock epJSON building models with paired synthetic 30-day hourly CSV meters covering chillers, pumps, fans, and tenant plug loads. Starter examples below are synthetic. They are not downloaded records.

EnergyPlus Extracts building envelope, HVAC sizing, and baseline operational consumption parameters from EnergyPlus epJSON schemas.

dspy, pandas, fastapi. Optimizes prompt signatures to consistently extract specific HVAC parameters and draft standardized engineering deficiency citations. Local or hosted LLM with strong analytical reasoning over tabular engineering telemetry and schema parameters.

Submit runnable source, pinned dependencies, setup commands, a working interface, example outputs, and a recorded end-to-end demo. Include tests and an action log.

Parses epJSON building models and extracts designated cooling setpoints, chilled water supply temps, and design COP.

Computes ASHRAE Guideline 14 compliant statistical metrics (NMBE and CV-RMSE) across 720 consecutive hourly timesteps.

Generates categorized snag tickets for any thermal sub-zone with sustained energy variance exceeding plus 15 percent.

```json [ { "input": { "subsystem": "Central Chiller 01", "measured_kwh_total": 84500, "baseline_kwh_total": 73200, "ambient_temp_avg_c": 39.5, "baseline_temp_avg_c": 38.8 }, "expected_output": { "variance_percent": 15.44, "nmbe_within_tolerance": false, "deficiency_flag": "excessive_lift_detected", "action_item": "Verify condenser water approach and refrigerant charge" } }, { "input": { "subsystem": "Air Handling Unit 04", "measured_kwh_total": 12400, "baseline_kwh_total": 12100, "ambient_temp_avg_c": 39.1, "baseline_temp_avg_c": 38.8 }, "expected_output": { "variance_percent": 2.48, "nmbe_within_tolerance": true, "deficiency_flag": "none", "action_item": "Accept subsystem handover" } } ] ```

Produces advisory closeout audit reports. Does not issue contractual defect certificates without certified commissioning agent approval. Use mock external writes. Require human review before any real deployment.

Rules

Participation guide

Produces advisory closeout audit reports. Does not issue contractual defect certificates without certified commissioning agent approval. Use mock external writes and human review.

Delivery guide

How work is evaluated

Evaluation

Parses epJSON building models and extracts designated cooling setpoints, chilled water supply temps, and design COP. Computes ASHRAE Guideline 14 compliant statistical metrics (NMBE and CV-RMSE) across 720 consecutive hourly timesteps. Generates categorized snag tickets for any thermal sub-zone with sustained energy variance exceeding plus 15 percent.

Submission format

Runnable source, pinned dependencies, setup commands, working interface, example outputs, recorded demo, tests, and action logs.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
4 scoring checks
Binary
1 pass or fail dimensions
Ordinal
3 scaled dimensions
Dimension 1

Complete workflow

Ingest 30-day post-occupancy hourly power and thermal meter readings alongside EnergyPlus epJSON baseline models. Normalize measured energy consumption against actual ambient dry-bulb temperature and occupancy sensor headcounts. Calculate normalized mean bias error and coefficient of variation between measured loads and design expectations. Compile an engineering handover deficiency report detailing subsystems exceeding contractual energy allowances.

Score scale
012
Score 0

Missing or incorrect.

Score 1

Partly complete, with documented gaps.

Score 2

Complete, with reproducible evidence.

Dimension 2

Correctness and evidence

Parses epJSON building models and extracts designated cooling setpoints, chilled water supply temps, and design COP. Computes ASHRAE Guideline 14 compliant statistical metrics (NMBE and CV-RMSE) across 720 consecutive hourly timesteps. Generates categorized snag tickets for any thermal sub-zone with sustained energy variance exceeding plus 15 percent.

Score scale
012
Score 0

Missing or incorrect.

Score 1

Partly complete, with documented gaps.

Score 2

Complete, with reproducible evidence.

Dimension 3

Review interface

A side-by-side dashboard comparing calibrated EnergyPlus baseline curves with BMS sub-meter feeds and defect tracking lists.

Score scale
012
Score 0

Missing or incorrect.

Score 1

Partly complete, with documented gaps.

Score 2

Complete, with reproducible evidence.

Dimension 4

Safe execution

Produces advisory closeout audit reports. Does not issue contractual defect certificates without certified commissioning agent approval.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Ingest 30-day post-occupancy hourly power and thermal meter readings alongside EnergyPlus epJSON baseline models.

  • Normalize measured energy consumption against actual ambient dry-bulb temperature and occupancy sensor headcounts.

  • Calculate normalized mean bias error and coefficient of variation between measured loads and design expectations.

  • Compile an engineering handover deficiency report detailing subsystems exceeding contractual energy allowances.

Resources and assets

Reference links and supporting material

Dataset notes

Construct mock epJSON building models with paired synthetic 30-day hourly CSV meters covering chillers, pumps, fans, and tenant plug loads. Supplied starter examples are synthetic, not source extracts.

Challenge setup

Configuration summary for this challenge.

Challenge input

Input details are not configured.

DSPy

Prompt and LM program optimization framework.

Evaluated output

No output checks are configured.

Evaluation checks
  • Ingest 30-day post-occupancy hourly power and thermal meter readings alongside EnergyPlus epJSON baseline m...
  • Parses epJSON building models and extracts designated cooling setpoints, chilled water supply temps, and de...
  • A side-by-side dashboard comparing calibrated EnergyPlus baseline curves with BMS sub-meter feeds and defec...
Configured evaluation evidence
  • 2 public reference cases
Configured runtime
  • 3 configured tools

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • DSPy · Optional
Training Infra
  • DSPy · Optional
Orchestration
  • DSPy · Optional

Evaluation contract

  • Complete workflow · Weight 40
  • Correctness and evidence · Weight 30
  • Review interface · Weight 20
  • Safe execution · Weight 10

Reference cases

  • 2 public reference cases

Workflow tasks

No workflow tasks are configured.

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Build a Commissioning Handover Energy Performance Baseline Reconciler