Skip to main content
Back to challenges
Challenge

Plug-and-Produce Module Type Package (MTP) Orchestrator using AI SDK

Integrating modular process plants using Module Type Package (MTP) manifests requires dynamic service discovery and HMI generation. Build a TypeScript agent using Vercel AI SDK and Helicone to automatically parse MTP XML definitions, generate dynamic web control interfaces, and orchestrate unit operations.

EngineeringHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Develop an AI SDK based orchestrator in TypeScript monitored via Helicone to discover and connect MTP-compliant process modules dynamically.

Delivery guide

How work is evaluated

Evaluation

Tests the agent's ability to accurately ingest an MTP XML manifest and generate valid OPC UA tag bindings and UI schemas.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

service_extraction_match

Verifies extracted services match the source MTP specification.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

parsing_accuracy

Accuracy percentage across test suite of complex multi-module MTP manifests. • target: 0.98 • range: 0.85-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Parse standardized MTP manifest XML files using Vercel AI SDK tools

  • Generate React UI components for module state visualization dynamically

  • Configure Helicone observability wrappers to track prompt costs and execution latency

  • Orchestrate inter-module recipe execution scripts based on exposed MTP services

Resources and assets

Reference links and supporting material

Dataset notes

Standardized MTP XML manifests representing modular pumps, stirrers, reactors, and inline dosing modules.

Challenge setup

Tests the agent's ability to accurately ingest an MTP XML manifest and generate valid OPC UA tag bindings and UI schemas.

Challenge input

XML string containing MTP manifest data.

AI SDK

Provides seamless integration with modern web runtimes and AI tools.

Helicone

Tracks token consumption, latency, and cost across dynamic agent calls.

Evaluated output

JSON object with discovered_services array and opc_node_ids mapping.

Evaluation checks
  • Verifies extracted services match the source MTP specification.
  • Accuracy percentage across test suite of complex multi-module MTP manifests. • target: 0.98 • range: 0.85-1
Configured evaluation evidence
  • Benchmark: Modular Automation Service Discovery Benchmark
  • Parsing Accuracy target: 0.98
  • 1 public reference case
Configured runtime
  • JavaScript execution harness
  • javascript sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Vercel AI SDK · Optional
Observation
  • Helicone · Required
Orchestration
  • Vercel AI SDK · Optional

Evaluation contract

  • service_extraction_match · Weight 1
  • parsing_accuracy · Weight 1

Metrics

  • parsing_accuracy · Target 0.98

Benchmarks

  • Modular Automation Service Discovery Benchmark
  • robotics

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: XML string containing MTP manifest data.

    Output: JSON object with discovered_services array and opc_node_ids mapping.

    Execution harness: JavaScript

Evaluation tools

  • AI SDK · Frontend/TypeScript Agent SDK
  • Helicone · LLM Observability Engine

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Plug-and-Produce Module Type Package (MTP) Orchestrator using AI SDK