Skip to main content
Back to challenges
Challenge

Multi-Model Creative Brief Generation Agent

Inspired by Google's Lyria 3 Pro music generation, this challenge focuses on building an advanced multi-model creative assistant using LangChain's LangGraph for complex workflow orchestration. Your system will generate comprehensive creative briefs for music video concepts, requiring a blend of strategic planning and artistic flair. GPT-5 Pro will act as the primary orchestrator, handling high-level creative direction and project management, while Claude 4 Sonnet will be leveraged for specialized tasks like lyrical analysis, mood board descriptions, and script generation. To ensure efficient and cost-effective inference, LocalAI will be used to serve smaller, optimized generative models (e.g., for specific image descriptions or style transfer snippets) on-premise. Hugging Face will manage the deployment, versioning, and inference endpoints for these specialized models, ensuring a robust and scalable architecture. This challenge emphasizes multi-model cooperation, dynamic model routing, and efficient inference management within a creative workflow.

Special Purpose AgentsHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Implement a LangChain (LangGraph) multi-agent system with GPT-5 Pro and Claude 4 Sonnet for creative content generation, leveraging LocalAI and Hugging Face for model serving and deployment.

Delivery guide

How work is evaluated

Evaluation

The evaluation module assesses the creative briefs generated by the multi-model system for coherence, creativity, completeness, and adherence to initial prompts. It also evaluates the efficiency of model routing and the effectiveness of local model inference.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
6 scoring checks
Binary
6 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

AllSectionsPresent

Ensures the generated creative brief contains all requested sections (e.g., concept, visuals, mood_board_desc).

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

CorrectModelRouting

Verifies that the appropriate model (GPT-5 Pro, Claude 4 Sonnet, LocalAI-served) is selected for at least 80% of test tasks.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 3

LocalAIIsUsed

Confirms that at least one task within a full creative brief generation successfully utilized a LocalAI-served model.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 4

CreativeCoherenceScore

An average score representing how well different sections of the brief align conceptually and stylistically. • target: 0.8 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 5

ContentDensityScore

Measures the richness and detail provided in key sections of the creative brief. • target: 0.75 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 6

AverageInferenceLatency

Average time taken for critical inference steps involving LocalAI-served models. • target: 100 • range: 0-500

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Master LangChain's LangGraph for building directed acyclic graph (DAG) workflows that manage state transitions and agent interactions across multi-step creative processes.

  • Integrate GPT-5 Pro as the 'Creative Director' agent, utilizing its advanced reasoning for conceptualizing overarching themes, target audience, and project goals.

  • Leverage Claude 4 Sonnet as specialist 'Lyric Analyst' and 'Visual Storyteller' agents, focusing on detailed content generation, emotional tone, and visual cues.

  • Implement LocalAI to serve task-specific generative models (e.g., image style generators, short text variations) within agent tools for rapid and controlled inference.

  • Utilize Hugging Face Hub and Endpoints for deploying, versioning, and managing access to various fine-tuned generative models used by LocalAI and other agents.

  • Design dynamic model routing logic within LangChain to intelligently select between GPT-5 Pro, Claude 4 Sonnet, or LocalAI-served models based on task complexity, cost, and desired output style.

  • Build custom LangChain tools that encapsulate calls to LocalAI-served models and Hugging Face inference endpoints, enabling agents to use these resources effectively.

Resources and assets

Reference links and supporting material

Dataset notes

Sample data for 2 tasks

Challenge setup

The evaluation module assesses the creative briefs generated by the multi-model system for coherence, creativity, completeness, and adherence to initial prompts. It also evaluates the efficiency of model routing and t...

Challenge input

{ 'song_lyrics': 'string', 'artist_style': 'string', 'target_mood': 'string', 'expected_sections': ['string', ...] }

GPT-5 Pro

Models · Large Language Models

LangChain

Framework for building LLM applications

GPT-5

Policy Serving in the agent workflow.

Evaluated output

{ 'creative_brief': {'title': 'string', 'concept': 'string', 'visuals': ['string', ...], 'mood_board_desc': 'string', 'suggested_scenes': ['string'...

Evaluation checks
  • Ensures the generated creative brief contains all requested sections (e.g., concept, visuals, mood_board_de...
  • Verifies that the appropriate model (GPT-5 Pro, Claude 4 Sonnet, LocalAI-served) is selected for at least 8...
  • Confirms that at least one task within a full creative brief generation successfully utilized a LocalAI-ser...
Configured evaluation evidence
  • CreativeCoherenceScore target: 0.8
  • 2 public reference cases
Configured runtime
  • Python execution harness

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • GPT-5 Pro · Required
  • LangChain · Optional
Policy Serving
  • GPT-5 · Optional
Orchestration
  • LangChain · Optional

Evaluation contract

  • AllSectionsPresent · Weight 1
  • CorrectModelRouting · Weight 1
  • LocalAIIsUsed · Weight 1
  • CreativeCoherenceScore · Weight 1
  • ContentDensityScore · Weight 1
  • AverageInferenceLatency · Weight 1

Metrics

  • CreativeCoherenceScore · Target 0.8
  • ContentDensityScore · Target 0.75
  • AverageInferenceLatency · Target 100

Reference cases

  • 2 public reference cases

Workflow tasks

  1. Task 1

    Input: { 'song_lyrics': 'string', 'artist_style': 'string', 'target_mood': 'string', 'expected_sections': ['string', ...] }

    Output: { 'creative_brief': {'title': 'string', 'concept': 'string', 'visuals': ['string', ...], 'mood_board_desc': 'string', 'suggested_scenes': ['string', ...]}, 'model_usage_log': [{'model': 'string', 'task': 'string', 'latency_ms': float}], 'score_creativity': float}

    Execution harness: Python

  2. Task 2

    Input: { 'task_description': 'string', 'expected_model_used': 'GPT-5 Pro|Claude 4 Sonnet|localai-model' }

    Output: { 'actual_model_used': 'string', 'routing_decision_reasoning': 'string', 'correct_routing': boolean, 'total_latency_ms': float }

    Execution harness: Python

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Multi-Model Creative Brief Generation Agent