Skip to main content
Loading navigation
Back to challenges

Real-Time DPI API Monitoring Dashboard using Vercel AI SDK

Public digital infrastructure APIs experience regional spikes and latency degradations requiring rapid root-cause summaries. Build a streaming analytics agent using Vercel AI SDK that monitors DPI telemetry and streams actionable incident reports, delivering real-time diagnostic updates with under 100ms streaming latency.

Business OperationsHosted by Vera
Sign in free to start

What you are building

Develop a real-time streaming infrastructure monitoring agent using Vercel AI SDK and TypeScript for rapid DPI API anomaly reporting.

How work is evaluated

Evaluation

Evaluates streaming output response schema and time-to-first-token metric.

Datasets

Public datasets, plus the private datasets you have access to.

How submissions are scored

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

stream_chunk_validity

Ensures streamed chunks parse cleanly into JSON.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

streaming_latency_ms

Time to first token chunk. • target: 100 • range: 0-500

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

  • Implement real-time streaming text and object generations using Vercel AI SDK

  • Build TypeScript interfaces for consuming server-sent event streams

  • Process high-frequency API ping logs from public public infrastructure endpoints

  • Generate concise developer diagnostic briefs during active outages

Resources

Dataset notes

Telemetry stream logs from 10 simulated national public infrastructure microservices.

Challenge setup

Evaluates streaming output response schema and time-to-first-token metric.

Challenge input

JSON containing api_endpoint, current_error_rate, average_latency_ms, and region.

AI SDK

Vercel AI SDK provides low-latency streaming abstractions for TS.

Evaluated output

JSON containing incident_severity ('LOW', 'HIGH', 'CRITICAL'), suspected_cause, and stream_validity (bool).

Evaluation checks
  • Ensures streamed chunks parse cleanly into JSON.
  • Time to first token chunk. • target: 100 • range: 0-500
Configured evaluation evidence
  • Benchmark: Real-time Telemetry Processing Benchmark
  • Streaming Latency Ms target: 100
  • 1 public reference case
Configured runtime
  • JavaScript execution harness
  • javascript sandbox (unavailable on Versalist)

Shows the first task, up to three tools, and up to three evidence items.

View technical recipe

Configured tools

Action Space
  • Vercel AI SDK · Required
  • Agno · Optional
  • Ax (Adaptive Experimentation) · Optional
Orchestration
  • Vercel AI SDK · Required
  • Agno · Optional

Evaluation contract

  • stream_chunk_validity · Weight 1
  • streaming_latency_ms · Weight 1

Metrics

  • streaming_latency_ms · Target 100

Benchmarks

  • Real-time Telemetry Processing Benchmark
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON containing api_endpoint, current_error_rate, average_latency_ms, and region.

    Output: JSON containing incident_severity ('LOW', 'HIGH', 'CRITICAL'), suspected_cause, and stream_validity (bool).

    Execution harness: JavaScript

Evaluation tools

  • AI SDK · Streaming Engine

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Real-Time DPI API Monitoring Dashboard using Vercel AI SDK