Public evaluation

Model Router Test

Evaluation of orchestration efficiency and routing accuracy.

Evaluation type

task based

Challenge

Multi-Model Orchestration Platform with LangGraph and GPT-5.4 Pro

Difficulty

Advanced

Rigor

Unspecified

Open linked challenge

Evaluation overview

How the linked challenge is judged: tasks, benchmarks, and criteria count.

Tasks

Benchmarks

Criteria

Task templates

Inputs and expected outputs.

Task 1

Model Router Test

Verifies that the correct model is selected based on complexity.

Input format

Task description text

Output format

JSON indicating model used

Linked challenge

Multi-Model Orchestration Platform with LangGraph and GPT-5.4 Pro

Architect a stateful orchestration system using LangChain and LangGraph to manage complex enterprise workflows. This platform will switch between GPT-5.4 Pro and Claude Sonnet 4.6.6 as orchestrators depending on the complexity of the task, using Fireworks AI and Featherless AI as high-performance inference backends to serve the models. The architecture will feature state persistence to ensure workflows can resume after long pauses, ideal for media and entertainment asset management systems. This challenge emphasizes efficient model routing and the use of graph-based state management, moving beyond basic sequential chains. By using different inference backends, you will learn to manage model deployments and ensure low-latency performance in an enterprise-grade environment.

Open challenge