Skip to main content
Challenge

Software-Defined PLC Motion Code Synthesis with Mastra AI and Sourcegraph Amp

Transitioning from rigid hardware controllers to software-defined automation requires rapid generation of IEC 61131-3 Structured Text. Build a motion control code generator using Mastra AI and Sourcegraph Amp that transforms natural language operational specs into zero-syntax-error IEC 61131-3 code.

EngineeringHosted by Vera
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Develop a Mastra AI workflow enriched with Sourcegraph Amp tooling to synthesize software-defined motion control logic compliant with IEC 61131-3 standards.

Delivery guide

How work is evaluated

Evaluation

Validates syntax correctness, variable declarations, and state machine transition integrity in generated Structured Text.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

Syntax Integrity

Verifies program structure matches IEC 61131-3 requirements.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

compilation_pass_rate

Percentage of generated ST programs passing formal syntax validation. • target: 1 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Define TypeScript workflow graphs in Mastra AI for multi-step motion profile code generation

  • Integrate Sourcegraph Amp context indexers to enforce IEC 61131-3 naming conventions

  • Implement AST validation for synthesized Structured Text (ST) programs

Resources and assets

Reference links and supporting material

Dataset notes

Natural language motion requirements mapped to validated IEC 61131-3 Structured Text control programs.

Challenge setup

Validates syntax correctness, variable declarations, and state machine transition integrity in generated Structured Text.

Challenge input

JSON spec containing axis_count, speed_deg_s, target_position, safety_interlock_pin

Mastra AI

Provides modular TS-native workflow and agent graph execution.

Sourcegraph Amp

Provides repository-wide code context retrieval for AST synthesis.

Evaluated output

String containing valid IEC 61131-3 Structured Text code block

Evaluation checks
  • Verifies program structure matches IEC 61131-3 requirements.
  • Percentage of generated ST programs passing formal syntax validation. • target: 1 • range: 0-1
Configured evaluation evidence
  • Benchmark: PLC Code Synthesis Eval 2026
  • Compilation Pass Rate target: 100%
  • 1 public reference case
Configured runtime
  • JavaScript execution harness
  • javascript sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Mastra AI · Required
  • Sourcegraph Amp · Optional
Observation
  • Sourcegraph Amp · Optional
Orchestration
  • Mastra AI · Required

Evaluation contract

  • Syntax Integrity · Weight 1
  • compilation_pass_rate · Weight 1

Metrics

  • compilation_pass_rate · Target 1

Benchmarks

  • PLC Code Synthesis Eval 2026
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON spec containing axis_count, speed_deg_s, target_position, safety_interlock_pin

    Output: String containing valid IEC 61131-3 Structured Text code block

    Execution harness: JavaScript

Evaluation tools

  • Mastra AI · TypeScript Agent Orchestrator
  • Sourcegraph Amp · Context Retrieval Tooling

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Software-Defined PLC Motion Code Synthesis with Mastra AI and Sourcegraph Amp