Skip to main content
Loading navigation
Back to challenges

Real-Time Model Accelerator Selector API with Vercel AI SDK

Develop a real-time hardware accelerator selector API using Vercel AI SDK to route model requests based on memory footprint, interconnect bandwidth, and cost limits. System must achieve sub-10ms routing decisions and optimize cost efficiency by at least 20%.

EngineeringHosted by Vera
Sign in free to start

What you are building

Build a Next.js / TypeScript edge API route using Vercel AI SDK to dynamically select model hardware targets.

How work is evaluated

Evaluation

Evaluates routing decision latency and overall cost savings under dynamic traffic loads.

Datasets

Public datasets, plus the private datasets you have access to.

How submissions are scored

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

Routing Speed SLA

Routing decision latency must be strictly under 10ms.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

Cost Savings Percent

Percentage savings compared to unoptimized static routing. • target: 0.245 • range: 0.2-0.4

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

  • Build dynamic routing components using Vercel AI SDK.

  • Implement real-time accelerator cost/latency decision algorithms in TypeScript.

  • Evaluate hardware backends including AWS Trainium, H100, and A10G endpoints.

  • Deliver streaming responses with optimized target routing under 10ms overhead.

Resources

Dataset notes

Traffic workload trace containing 1,000 requests with token counts and latency SLA tags.

Challenge setup

Evaluates routing decision latency and overall cost savings under dynamic traffic loads.

Challenge input

JSON payload specifying request context length and max latency budget

AI SDK

TypeScript framework for streaming and agent tool routing at the edge

Evaluated output

JSON with selected_accelerator, routing_latency_ms, and estimated_cost_usd

Evaluation checks
  • Routing decision latency must be strictly under 10ms.
  • Percentage savings compared to unoptimized static routing. • target: 0.245 • range: 0.2-0.4
Configured evaluation evidence
  • Benchmark: Edge AI Dynamic Routing Benchmark
  • Cost Savings Percent target: 0.245
  • 1 public reference case
Configured runtime
  • JavaScript execution harness
  • javascript sandbox (unavailable on Versalist)

Shows the first task, up to three tools, and up to three evidence items.

View technical recipe

Configured tools

Action Space
  • Vercel AI SDK · Required
  • Anthropic · Optional
  • Anthropic Claude · Optional
Orchestration
  • Vercel AI SDK · Required

Evaluation contract

  • Routing Speed SLA · Weight 1
  • Cost Savings Percent · Weight 1

Metrics

  • Cost Savings Percent · Target 0.245

Benchmarks

  • Edge AI Dynamic Routing Benchmark
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON payload specifying request context length and max latency budget

    Output: JSON with selected_accelerator, routing_latency_ms, and estimated_cost_usd

    Execution harness: JavaScript

Evaluation tools

  • AI SDK · Agent Framework

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Real-Time Model Accelerator Selector API with Vercel AI SDK