Skip to main content
Back to challenges
Challenge

Optimize High-Performance AI Inference Workflows with Mastra AI

Construct an AI inference orchestration system using Mastra AI to manage multiple model providers. This system dynamically routes high-complexity requests to GPT-5.4 Pro and lighter, sub-task requests to Claude Sonnet 4.6.6. The architecture utilizes Fireworks AI for low-latency inference and BentoML for scalable model serving, creating a production-grade infrastructure for enterprise data processing.

Machine LearningHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Architect an inference orchestration layer with Mastra AI that intelligently routes tasks to optimal models and serving engines.

Delivery guide

How work is evaluated

Evaluation

Benchmarking inference latency and routing accuracy.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

RoutingLogic

Verify complex tasks routed to GPT-5.4 Pro

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

Latency

P99 latency in ms • target: 150 • range: 0-500

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Implement Mastra AI workflows for managing complex model state and memory

  • Deploy optimized model endpoints using BentoML for high-throughput serving

  • Configure dynamic routing policies for GPT-5.4 Pro and Claude Sonnet 4.6.6

  • Leverage Fireworks AI runtime for accelerated inference execution

  • Design system-level failure recovery for multi-provider API calls

Resources and assets

Reference links and supporting material

Dataset notes

Sample data for 1 tasks

Challenge setup

Benchmarking inference latency and routing accuracy.

Challenge input

Concurrency level, Task Complexity

Mastra AI

TypeScript agent framework.

Fireworks AI

Fast inference and fine-tuning platform.

GPT-5

Policy Serving in the agent workflow.

Evaluated output

Latency Stats

Evaluation checks
  • Verify complex tasks routed to GPT-5.4 Pro
  • P99 latency in ms • target: 150 • range: 0-500
Configured evaluation evidence
  • Latency target: 150
  • 1 public reference case
Configured runtime
  • Python execution harness

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Mastra AI · Required
Policy Serving
  • Fireworks AI · Optional
  • GPT-5 · Optional
Training Infra
  • Fireworks AI · Optional
Orchestration
  • Mastra AI · Required

Evaluation contract

  • RoutingLogic · Weight 1
  • Latency · Weight 1

Metrics

  • Latency · Target 150

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: Concurrency level, Task Complexity

    Output: Latency Stats

    Execution harness: Python

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Optimize High-Performance AI Inference Workflows with Mastra AI