Skip to main content
Challenge

Build an ifcbimtester Gherkin Model Validation Runner

Utilize ifcbimtester to evaluate IFC building models against Gherkin behavior-driven development scenarios. Validate strict attribute presence and geometric constraints for automated architectural handover and compliance tracking.

EngineeringHosted by Vera
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Utilize ifcbimtester to evaluate IFC building models against Gherkin behavior-driven development scenarios. Validate strict attribute presence and geometric constraints for automated architectural handover and compliance tracking.

BIM manager

Ingest IFC models alongside structural requirement specifications written in Gherkin syntax.

Parse behavior-driven test scenarios targeting IFC entity classes using ifcbimtester.

Execute unit tests evaluating dimensional constraints and required property sets.

Compile an automated BIM quality assurance report identifying failing scenario steps.

Browser validation dashboard displaying Gherkin scenario results, failing IFC entity details, and a downloadable QA report.

Acquire open sample IFC structural models from the buildingSMART technical repository or synthesize mock IFC structural element dictionaries. Starter examples below are synthetic. They are not downloaded records.

IfcOpenShell Leverages ifcbimtester from the IfcOpenShell ecosystem to run Gherkin-based unit tests for IFC.

dspy, ifcbimtester, typer. DSPy provides programmatic prompt optimization to reliably translate heterogeneous IFC property requirements into rigid Gherkin test syntax. Instruction-tuned LLM capable of generating structured Gherkin syntax and interpreting BIM attribute requirements.

Submit runnable source, pinned dependencies, setup commands, a working interface, example outputs, and a recorded end-to-end demo. Include tests and an action log.

Parses Gherkin feature files targeting specific IFC entity classes.

Detects structural members missing defined properties like FireRating or LoadBearing.

Executes end-to-end validation without crashing on empty property sets.

```json [ { "input": { "model_id": "IFC-01", "gherkin_scenario": "Given an IfcDoor, Then it must have a FireRating property", "entities": [ { "type": "IfcDoor", "properties": { "FireRating": "30M" } } ], "_note": "These are simplified synthetic app records, not complete official schema files." }, "expected_output": { "model_id": "IFC-01", "scenarios_run": 1, "passed": 1, "failed": 0, "status": "COMPLIANT" } }, { "input": { "model_id": "IFC-02", "gherkin_scenario": "Given an IfcWall, Then it must have a LoadBearing property", "entities": [ { "type": "IfcWall", "properties": {} } ], "_note": "These are simplified synthetic app records, not complete official schema files." }, "expected_output": { "model_id": "IFC-02", "scenarios_run": 1, "passed": 0, "failed": 1, "status": "NON_COMPLIANT" } } ] ```

Generates validation reports based on model metadata only; does not perform physical load testing, structural certification, or site dispatch. Use mock external writes. Require human review before any real deployment.

Rules

Participation guide

Generates validation reports based on model metadata only; does not perform physical load testing, structural certification, or site dispatch. Use mock external writes and human review.

Delivery guide

How work is evaluated

Evaluation

Parses Gherkin feature files targeting specific IFC entity classes. Detects structural members missing defined properties like FireRating or LoadBearing. Executes end-to-end validation without crashing on empty property sets.

Submission format

Runnable source, pinned dependencies, setup commands, working interface, example outputs, recorded demo, tests, and action logs.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
4 scoring checks
Binary
1 pass or fail dimensions
Ordinal
3 scaled dimensions
Dimension 1

Complete workflow

Ingest IFC models alongside structural requirement specifications written in Gherkin syntax. Parse behavior-driven test scenarios targeting IFC entity classes using ifcbimtester. Execute unit tests evaluating dimensional constraints and required property sets. Compile an automated BIM quality assurance report identifying failing scenario steps.

Score scale
012
Score 0

Missing or incorrect.

Score 1

Partly complete, with documented gaps.

Score 2

Complete, with reproducible evidence.

Dimension 2

Correctness and evidence

Parses Gherkin feature files targeting specific IFC entity classes. Detects structural members missing defined properties like FireRating or LoadBearing. Executes end-to-end validation without crashing on empty property sets.

Score scale
012
Score 0

Missing or incorrect.

Score 1

Partly complete, with documented gaps.

Score 2

Complete, with reproducible evidence.

Dimension 3

Review interface

Browser validation dashboard displaying Gherkin scenario results, failing IFC entity details, and a downloadable QA report.

Score scale
012
Score 0

Missing or incorrect.

Score 1

Partly complete, with documented gaps.

Score 2

Complete, with reproducible evidence.

Dimension 4

Safe execution

Generates validation reports based on model metadata only; does not perform physical load testing, structural certification, or site dispatch.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Ingest IFC models alongside structural requirement specifications written in Gherkin syntax.

  • Parse behavior-driven test scenarios targeting IFC entity classes using ifcbimtester.

  • Execute unit tests evaluating dimensional constraints and required property sets.

  • Compile an automated BIM quality assurance report identifying failing scenario steps.

Resources and assets

Reference links and supporting material

Dataset notes

Acquire open sample IFC structural models from the buildingSMART technical repository or synthesize mock IFC structural element dictionaries. Supplied starter examples are synthetic, not source extracts.

Challenge setup

Configuration summary for this challenge.

Challenge input

Input details are not configured.

DSPy

Prompt and LM program optimization framework.

Evaluated output

No output checks are configured.

Evaluation checks
  • Ingest IFC models alongside structural requirement specifications written in Gherkin syntax. Parse behavior...
  • Parses Gherkin feature files targeting specific IFC entity classes. Detects structural members missing defi...
  • Browser validation dashboard displaying Gherkin scenario results, failing IFC entity details, and a downloa...
Configured evaluation evidence
  • 2 public reference cases
Configured runtime
  • 3 configured tools

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • DSPy · Optional
Training Infra
  • DSPy · Optional
Orchestration
  • DSPy · Optional

Evaluation contract

  • Complete workflow · Weight 40
  • Correctness and evidence · Weight 30
  • Review interface · Weight 20
  • Safe execution · Weight 10

Reference cases

  • 2 public reference cases

Workflow tasks

No workflow tasks are configured.

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Build an ifcbimtester Gherkin Model Validation Runner