Back to challenges
Challenge

Real-Time Model Accelerator Selector API with Vercel AI SDK

Develop a real-time hardware accelerator selector API using Vercel AI SDK to route model requests based on memory footprint, interconnect bandwidth, and cost limits. System must achieve sub-10ms routing decisions and optimize cost efficiency by at least 20%.

EngineeringHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Build a Next.js / TypeScript edge API route using Vercel AI SDK to dynamically select model hardware targets.

Delivery guide

How work is evaluated

Evaluation

Evaluates routing decision latency and overall cost savings under dynamic traffic loads.

Datasets

Shared data for this challenge

Review public datasets and any private uploads tied to your build.

Loading datasets...
Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

Routing Speed SLA

Routing decision latency must be strictly under 10ms.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

Cost Savings Percent

Percentage savings compared to unoptimized static routing. • target: 0.245 • range: 0.2-0.4

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Build dynamic routing components using Vercel AI SDK.

  • Implement real-time accelerator cost/latency decision algorithms in TypeScript.

  • Evaluate hardware backends including AWS Trainium, H100, and A10G endpoints.

  • Deliver streaming responses with optimized target routing under 10ms overhead.

Resources and assets

Reference links and supporting material

Dataset notes

Traffic workload trace containing 1,000 requests with token counts and latency SLA tags.

How this agent runs

Evaluates routing decision latency and overall cost savings under dynamic traffic loads.

Challenge input

JSON payload specifying request context length and max latency budget

AI SDK

TypeScript framework for streaming and agent tool routing at the edge

Evaluated output

JSON with selected_accelerator, routing_latency_ms, and estimated_cost_usd

Checks for
  • Routing decision latency must be strictly under 10ms.
  • Percentage savings compared to unoptimized static routing. • target: 0.245 • range: 0.2-0.4
Proof of success
  • Benchmark: Edge AI Dynamic Routing Benchmark
  • Cost Savings Percent target: 0.245
  • 1 public reference case
Runtime evidence
  • JavaScript execution harness
  • Javascript sandbox
View technical recipe

Configured tools

Action Space
  • Vercel AI SDK · Required
  • Anthropic · Optional
  • Anthropic Claude · Optional
Orchestration
  • Vercel AI SDK · Required

Evaluation contract

  • Routing Speed SLA · Weight 1
  • Cost Savings Percent · Weight 1

Recipe state

This is a preview. The configuration can change before the evaluation recipe is locked.

Frequently Asked Questions about Real-Time Model Accelerator Selector API with Vercel AI SDK