Skip to main content
Challenge

Build a Bonsai IFC Quality Assurance Gherkin Rule Verifier

Formulate automated Gherkin acceptance test scenarios for architectural IFC models using ifcbimtester principles. Validate deliverable models before client handover against contractual BIM execution plans and classification standards.

EngineeringHosted by Vera
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Formulate automated Gherkin acceptance test scenarios for architectural IFC models using ifcbimtester principles. Validate deliverable models before client handover against contractual BIM execution plans and classification standards.

BIM Quality Assurance Manager

Ingest client BIM execution plan clauses and convert requirements into Gherkin feature files.

Execute ifcbimtester rules against incoming architectural IFC model files.

Evaluate entity classifications, mandatory attributes, and geometric representations.

Produce an interactive HTML compliance audit dashboard with actionable remediation steps.

QA scorecard interface displaying pass/fail step indicators, failing IFC entity IDs, and error traces.

Generate mock IFC4 architectural models paired with synthetic Gherkin feature specification scripts. Starter examples below are synthetic. They are not downloaded records.

IfcOpenShell Integrates the ifcbimtester tool from the IfcOpenShell tool directory for Gherkin-based BIM testing.

mastra-ai, ifcopenshell, behave. Mastra AI orchestrates test execution workflows, parameter bindings, and structured evaluation logging. API-hosted LLM capable of translating contractual BIM requirements into formal Gherkin scenarios.

Submit runnable source, pinned dependencies, setup commands, a working interface, example outputs, and a recorded end-to-end demo. Include tests and an action log.

Parses Gherkin feature files defining IFC model acceptance criteria.

Executes entity-level rule evaluation for IfcWall, IfcDoor, and IfcSpace entities.

Outputs structured compliance report itemizing failing entity GlobalIds.

```json [ { "input": { "gherkin_rule": "Feature: Wall Classification\nScenario: All external walls must have FireRating\nGiven the model has IfcWall entities\nThen every external IfcWall must have property FireRating", "ifc_entities": [ { "global_id": "WALL-001", "type": "IfcWall", "is_external": true, "properties": { "FireRating": "2HR" } }, { "global_id": "WALL-002", "type": "IfcWall", "is_external": false, "properties": {} } ] }, "expected_output": { "total_tested": 2, "passed": true, "violations": [] } }, { "input": { "gherkin_rule": "Feature: Wall Classification\nScenario: All external walls must have FireRating\nGiven the model has IfcWall entities\nThen every external IfcWall must have property FireRating", "ifc_entities": [ { "global_id": "WALL-003", "type": "IfcWall", "is_external": true, "properties": {} } ] }, "expected_output": { "total_tested": 1, "passed": false, "violations": [ { "global_id": "WALL-003", "error": "External IfcWall lacks mandatory property 'FireRating'" } ] } } ] ```

Ensures acceptance tests run in sandboxed read-only memory; does not modify submitted master models. Use mock external writes. Require human review before any real deployment.

Rules

Participation guide

Ensures acceptance tests run in sandboxed read-only memory; does not modify submitted master models. Use mock external writes and human review.

Delivery guide

How work is evaluated

Evaluation

Parses Gherkin feature files defining IFC model acceptance criteria. Executes entity-level rule evaluation for IfcWall, IfcDoor, and IfcSpace entities. Outputs structured compliance report itemizing failing entity GlobalIds.

Submission format

Runnable source, pinned dependencies, setup commands, working interface, example outputs, recorded demo, tests, and action logs.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
4 scoring checks
Binary
1 pass or fail dimensions
Ordinal
3 scaled dimensions
Dimension 1

Complete workflow

Ingest client BIM execution plan clauses and convert requirements into Gherkin feature files. Execute ifcbimtester rules against incoming architectural IFC model files. Evaluate entity classifications, mandatory attributes, and geometric representations. Produce an interactive HTML compliance audit dashboard with actionable remediation steps.

Score scale
012
Score 0

Missing or incorrect.

Score 1

Partly complete, with documented gaps.

Score 2

Complete, with reproducible evidence.

Dimension 2

Correctness and evidence

Parses Gherkin feature files defining IFC model acceptance criteria. Executes entity-level rule evaluation for IfcWall, IfcDoor, and IfcSpace entities. Outputs structured compliance report itemizing failing entity GlobalIds.

Score scale
012
Score 0

Missing or incorrect.

Score 1

Partly complete, with documented gaps.

Score 2

Complete, with reproducible evidence.

Dimension 3

Review interface

QA scorecard interface displaying pass/fail step indicators, failing IFC entity IDs, and error traces.

Score scale
012
Score 0

Missing or incorrect.

Score 1

Partly complete, with documented gaps.

Score 2

Complete, with reproducible evidence.

Dimension 4

Safe execution

Ensures acceptance tests run in sandboxed read-only memory; does not modify submitted master models.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Ingest client BIM execution plan clauses and convert requirements into Gherkin feature files.

  • Execute ifcbimtester rules against incoming architectural IFC model files.

  • Evaluate entity classifications, mandatory attributes, and geometric representations.

  • Produce an interactive HTML compliance audit dashboard with actionable remediation steps.

Resources and assets

Reference links and supporting material

Dataset notes

Generate mock IFC4 architectural models paired with synthetic Gherkin feature specification scripts. Supplied starter examples are synthetic, not source extracts.

Challenge setup

Configuration summary for this challenge.

Challenge input

Input details are not configured.

Mastra AI

TypeScript agent framework.

Evaluated output

No output checks are configured.

Evaluation checks
  • Ingest client BIM execution plan clauses and convert requirements into Gherkin feature files. Execute ifcbi...
  • Parses Gherkin feature files defining IFC model acceptance criteria. Executes entity-level rule evaluation...
  • QA scorecard interface displaying pass/fail step indicators, failing IFC entity IDs, and error traces.
Configured evaluation evidence
  • 2 public reference cases
Configured runtime
  • 2 configured tools

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Mastra AI · Optional
Orchestration
  • Mastra AI · Optional

Evaluation contract

  • Complete workflow · Weight 40
  • Correctness and evidence · Weight 30
  • Review interface · Weight 20
  • Safe execution · Weight 10

Reference cases

  • 2 public reference cases

Workflow tasks

No workflow tasks are configured.

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Build a Bonsai IFC Quality Assurance Gherkin Rule Verifier