Skip to main content
Back to challenges
Challenge

Multi-Model Orchestration Platform with LangGraph and GPT-5.4 Pro

Architect a stateful orchestration system using LangChain and LangGraph to manage complex enterprise workflows. This platform will switch between GPT-5.4 Pro and Claude Sonnet 4.6.6 as orchestrators depending on the complexity of the task, using Fireworks AI and Featherless AI as high-performance inference backends to serve the models. The architecture will feature state persistence to ensure workflows can resume after long pauses, ideal for media and entertainment asset management systems. This challenge emphasizes efficient model routing and the use of graph-based state management, moving beyond basic sequential chains. By using different inference backends, you will learn to manage model deployments and ensure low-latency performance in an enterprise-grade environment.

Special Purpose AgentsHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Create a multi-model orchestration platform using LangGraph to route tasks between GPT-5.4 Pro and Claude Sonnet 4.6.6.

Delivery guide

How work is evaluated

Evaluation

Evaluation of orchestration efficiency and routing accuracy.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

Routing Logic

Verify logic routes correctly

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

Throughput

Tasks processed per second • target: 5 • range: 1-10

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Design LangGraph state machines for multi-step reasoning

  • Configure LangChain model routers to distribute load between GPT-5.4 Pro and Claude Sonnet 4.6.6

  • Deploy custom model endpoints on Fireworks AI and Featherless AI

  • Manage workflow checkpoints for long-running processes

  • Implement policy-based routing to ensure cost-efficiency

  • Integrate LangChain tool calling for dynamic orchestration

Resources and assets

Reference links and supporting material

Dataset notes

Sample data for 1 tasks

Challenge setup

Evaluation of orchestration efficiency and routing accuracy.

Challenge input

Task description text

Langchain

Building applications with LLMs

Fireworks AI

Fast inference and fine-tuning platform.

GPT-5

Policy Serving in the agent workflow.

Evaluated output

JSON indicating model used

Evaluation checks
  • Verify logic routes correctly
  • Tasks processed per second • target: 5 • range: 1-10
Configured evaluation evidence
  • Throughput target: 5
  • 1 public reference case
Configured runtime
  • Python execution harness

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Langchain · Optional
  • Fireworks AI · Optional
Policy Serving
  • GPT-5 · Required
Orchestration
  • Langchain · Optional

Evaluation contract

  • Routing Logic · Weight 1
  • Throughput · Weight 1

Metrics

  • Throughput · Target 5

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: Task description text

    Output: JSON indicating model used

    Execution harness: Python

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Multi-Model Orchestration Platform with LangGraph and GPT-5.4 Pro