Skip to main content
Loading navigation
Back to challenges

Fault-Tolerant Distributed Inference Stream Router with AI SDK and Amazon Bedrock

Implement automated stream failover across cloud inference providers under hardware fault conditions. Build a resilient TypeScript application using the Vercel AI SDK connected to Amazon Bedrock endpoints that detects latency degradation or hardware errors and seamlessly shifts active generation streams.

EngineeringHosted by Vera
Sign in free to start

What you are building

Build a streaming router application with Vercel AI SDK and Amazon Bedrock that monitors hardware latency spikes and reroutes token output without dropping client streams.

How work is evaluated

Evaluation

Evaluates fault detection latency and stream continuity during endpoint failover.

Datasets

Public datasets, plus the private datasets you have access to.

How submissions are scored

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

Failover Trigger Test

Verify failover is executed when latency exceeds threshold.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

Stream Interruption Time

Downtime during region migration in milliseconds. • target: 100 • range: 0-1000

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

  • Implement resilient streaming routers using Vercel AI SDK.

  • Connect Amazon Bedrock model providers with automated error boundary catching.

  • Ensure seamless token stream migration upon hardware fault injection.

Resources

Dataset notes

Trace logs simulating multi-region AWS Bedrock latency spikes and 503 Service Unavailable responses.

Challenge setup

Evaluates fault detection latency and stream continuity during endpoint failover.

Challenge input

Primary endpoint status, timeout threshold ms

AI SDK

Vercel SDK for streaming AI applications.

Amazon Bedrock

AWS enterprise LLM host runtime.

Evaluated output

JSON failover_triggered, downtime_duration_ms, output_token_count

Evaluation checks
  • Verify failover is executed when latency exceeds threshold.
  • Downtime during region migration in milliseconds. • target: 100 • range: 0-1000
Configured evaluation evidence
  • Benchmark: High-Availability Streaming Benchmark
  • Stream Interruption Time target: 100
  • 1 public reference case
Configured runtime
  • JavaScript execution harness
  • javascript sandbox (unavailable on Versalist)

Shows the first task, up to three tools, and up to three evidence items.

View technical recipe

Configured tools

Action Space
  • Amazon Bedrock · Required
  • Vercel AI SDK · Optional
  • Anthropic · Optional
Orchestration
  • Vercel AI SDK · Optional

Evaluation contract

  • Failover Trigger Test · Weight 1
  • Stream Interruption Time · Weight 1

Metrics

  • Stream Interruption Time · Target 100

Benchmarks

  • High-Availability Streaming Benchmark
  • security

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: Primary endpoint status, timeout threshold ms

    Output: JSON failover_triggered, downtime_duration_ms, output_token_count

    Execution harness: JavaScript

Evaluation tools

  • AI SDK · Streaming Framework
  • Amazon Bedrock · Runtime Infrastructure

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Fault-Tolerant Distributed Inference Stream Router with AI SDK and Amazon Bedrock