Build an Instrumentation Calibration Certificate Verification Engine
Extract calibration dates, technician signatures, and accuracy limits from digital calibration certificates. Correlate against open62541 OPC UA instrument registers to flag expired or non-compliant field sensors.
What you are building
The core problem, expected build, and operating context for this challenge.
Extract calibration dates, technician signatures, and accuracy limits from digital calibration certificates. Correlate against open62541 OPC UA instrument registers to flag expired or non-compliant field sensors.
Metrology Compliance Specialist
Parse digital calibration certificate text and metadata using regularized key-value extractors.
Cross-reference certificate instrument IDs against open62541 OPC UA device inventory registers.
Identify sensors with missing calibration certificates or certificates exceeding validity horizons.
Output a metrology audit compliance report with flagged equipment for replacement.
Compliance checklist table highlighting certificate expiration dates and uncalibrated node IDs.
Construct simplified synthetic JSON certificates paired with simulated open62541 instrument node inventories. Starter examples below are synthetic. They are not downloaded records.
open62541 OPC UA Uses open62541 OPC UA server object references and standard server profile modeling from Source 0.
llamaindex, fastapi, pypdf, pandas. LlamaIndex indexes unstructured certificate PDFs and tabular instrument registries for precise metadata extraction. Anthropic Claude API for certificate metadata parsing and inconsistency extraction.
Submit runnable source, pinned dependencies, setup commands, a working interface, example outputs, and a recorded end-to-end demo. Include tests and an action log.
Identifies 100% of expired or mismatched certificates in test batches.
Emits validation alerts for sensors operating outside approved calibration validity windows.
Exports structured ISO/IEC 17025 compliance audit summaries.
```json [ { "input": { "certificate_id": "CAL-2023-991", "sensor_id": "OPC-SENS-PRESS-11", "calibration_date": "2023-01-10", "validity_months": 12, "current_audit_date": "2024-03-01" }, "expected_output": { "sensor_id": "OPC-SENS-PRESS-11", "compliance_status": "EXPIRED", "overdue_days": 51, "recommended_action": "SCHEDULE_RECALIBRATION" } }, { "input": { "certificate_id": "CAL-2024-102", "sensor_id": "OPC-SENS-TEMP-04", "calibration_date": "2024-02-01", "validity_months": 12, "current_audit_date": "2024-03-01" }, "expected_output": { "sensor_id": "OPC-SENS-TEMP-04", "compliance_status": "COMPLIANT", "overdue_days": 0, "recommended_action": "NONE" } } ] ```
Read-only compliance assessment; outputs verification tables without modifying instrumentation parameters. Use mock external writes. Require human review before any real deployment.
Participation guide
Read-only compliance assessment; outputs verification tables without modifying instrumentation parameters. Use mock external writes and human review.
How work is evaluated
Identifies 100% of expired or mismatched certificates in test batches. Emits validation alerts for sensors operating outside approved calibration validity windows. Exports structured ISO/IEC 17025 compliance audit summaries.
Runnable source, pinned dependencies, setup commands, working interface, example outputs, recorded demo, tests, and action logs.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
Complete workflow
Parse digital calibration certificate text and metadata using regularized key-value extractors. Cross-reference certificate instrument IDs against open62541 OPC UA device inventory registers. Identify sensors with missing calibration certificates or certificates exceeding validity horizons. Output a metrology audit compliance report with flagged equipment for replacement.
Missing or incorrect.
Partly complete, with documented gaps.
Complete, with reproducible evidence.
Correctness and evidence
Identifies 100% of expired or mismatched certificates in test batches. Emits validation alerts for sensors operating outside approved calibration validity windows. Exports structured ISO/IEC 17025 compliance audit summaries.
Missing or incorrect.
Partly complete, with documented gaps.
Complete, with reproducible evidence.
Review interface
Compliance checklist table highlighting certificate expiration dates and uncalibrated node IDs.
Missing or incorrect.
Partly complete, with documented gaps.
Complete, with reproducible evidence.
Safe execution
Read-only compliance assessment; outputs verification tables without modifying instrumentation parameters.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Parse digital calibration certificate text and metadata using regularized key-value extractors.
Cross-reference certificate instrument IDs against open62541 OPC UA device inventory registers.
Identify sensors with missing calibration certificates or certificates exceeding validity horizons.
Output a metrology audit compliance report with flagged equipment for replacement.
Reference links and supporting material
Construct simplified synthetic JSON certificates paired with simulated open62541 instrument node inventories. Supplied starter examples are synthetic, not source extracts.
Challenge setup
Configuration summary for this challenge.
Challenge input
Input details are not configured.
LlamaIndex
Data framework for LLM applications
Evaluated output
No output checks are configured.
- Parse digital calibration certificate text and metadata using regularized key-value extractors. Cross-refer...
- Identifies 100% of expired or mismatched certificates in test batches. Emits validation alerts for sensors...
- Compliance checklist table highlighting certificate expiration dates and uncalibrated node IDs.
- 2 public reference cases
- 1 configured tool
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- LlamaIndex · Optional
Evaluation contract
- Complete workflow · Weight 40
- Correctness and evidence · Weight 30
- Review interface · Weight 20
- Safe execution · Weight 10
Reference cases
- 2 public reference cases
Workflow tasks
No workflow tasks are configured.
Recipe freeze marker
No recipe freeze time is recorded.