Continuous Batching Observability Dashboard with Mastra AI and Lovable
Design a real-time observability telemetry workflow for continuous batching LLM inference engines. Construct a Mastra AI workflow and frontend interface using Lovable to ingest, aggregate, and visualize token queuing delay, batch fill rates, and active GPU memory footprint.
What you are building
The core problem, expected build, and operating context for this challenge.
Build a Mastra AI workflow integrated with Lovable to track real-time LLM inference continuous batching engine metrics including token queue delays and memory consumption.
How work is evaluated
Evaluates Mastra AI telemetry ingestion accuracy and dashboard aggregation response times.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
Aggregation Precision Test
Verify aggregated math strictly matches target calculated averages.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Telemetry Processing Latency
Processing latency per metric frame in milliseconds. • target: 5 • range: 0-50
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Build a TypeScript workflow using Mastra AI to process continuous streaming telemetry metrics.
Design interactive performance monitoring UI components in Lovable.
Track queue time, batch saturation, and vLLM token generation rates.
Reference links and supporting material
Prometheus-style time-series trace dataset collected from continuous batching vLLM inference node logs.
Challenge setup
Evaluates Mastra AI telemetry ingestion accuracy and dashboard aggregation response times.
Challenge input
Array of engine metric events (queue_delay_ms, batch_size, memory_used_percent)
Mastra AI
TypeScript AI workflow engine.
Lovable
UI component generator and prototyping engine.
Evaluated output
JSON containing aggregated mean_queue_delay, max_batch_size, avg_memory_used
- Verify aggregated math strictly matches target calculated averages.
- Processing latency per metric frame in milliseconds. • target: 5 • range: 0-50
- Benchmark: Inference Engine Telemetry Benchmark
- Telemetry Processing Latency target: 5
- 1 public reference case
- JavaScript execution harness
- javascript sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- Mastra AI · Required
- Lovable · Optional
- Anthropic · Optional
- Mastra AI · Required
Evaluation contract
- Aggregation Precision Test · Weight 1
- Telemetry Processing Latency · Weight 1
Metrics
- Telemetry Processing Latency · Target 5
Benchmarks
- Inference Engine Telemetry Benchmark
- general
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: Array of engine metric events (queue_delay_ms, batch_size, memory_used_percent)
Output: JSON containing aggregated mean_queue_delay, max_batch_size, avg_memory_used
Execution harness: JavaScript
Evaluation tools
- Mastra AI · Agent Framework
- Lovable · AI Tooling
Recipe freeze marker
No recipe freeze time is recorded.