Back to evaluations
Draft evaluation
Mastra AI MTP Orchestration Pipeline with Braintrust Evaluation — evaluation
Evaluates MTP manifest validation, state machine mapping accuracy, and workflow latency via Braintrust.
Evaluation type
task based
Challenge
Mastra AI MTP Orchestration Pipeline with Braintrust Evaluation
Difficulty
Intermediate
Rigor
Not declared
The author has not specified a rigor level.
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Tasks
1
Benchmarks
0
Criteria
0
Task templates
Inputs and expected outputs.
Task 1
mtp_validation_task
Validate MTP module interface definitions and output state compatibility status.
Input format
JSON containing MTP manifest data (services, eID, OPC UA endpoints)
Output format
JSON with is_valid (bool), state_machine_compliant (bool), missing_tags (list), processing_time_ms (int)