Skip to main content
Back to challenges
Challenge

Evaluate AGI Paradigms with Hierarchical Agents

The debate around Artificial General Intelligence (AGI) continues, with prominent figures like Yann LeCun questioning the path of LLM-centric approaches and advocating for 'world models'. This challenge invites you to build a cutting-edge hierarchical multi-agent research system to go into this philosophical and architectural divide. Your system will use GPT-5 for its unparalleled reasoning and creative synthesis capabilities, allowing agents to generate complex arguments and hypothetical designs. DSPy will be crucial for programmatically optimizing prompts, ensuring agents produce high-quality, nuanced outputs when discussing abstract AI concepts. LlamaIndex will serve as the backbone for building a hierarchical agent structure and a comprehensive RAG knowledge base, enabling agents to conduct thorough research, debate, and synthesize findings. The ultimate goal is a sophisticated comparative analysis report on the viability and limitations of LLM-centric vs. 'world model' approaches to AGI.

Special Purpose AgentsHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Build a hierarchical multi-agent research system using GPT-5, DSPy, and LlamaIndex to critically evaluate and compare LLM-centric vs. 'world model' approaches to Artificial General Intelligence (AGI).

Delivery guide

How work is evaluated

Evaluation

The evaluation will assess the hierarchical agent system's ability to produce a nuanced, well-reasoned comparative analysis report on LLM-centric vs. 'world model' approaches to AGI, demonstrating effective use of GPT-5 for deep reasoning, DSPy for prompt optimization, and LlamaIndex for hierarchical organization and knowledge management.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Learning goals

What you should walk away with

  • Optimize LLM prompts using DSPy for complex reasoning tasks.

  • Implement hybrid reasoning with GPT-5 for conceptual exploration.

  • Conduct comparative analysis of AI paradigms using agents.

  • Design hierarchical agent systems with LlamaIndex.

  • Build a knowledge management system for AI research.

Resources and assets

Reference links and supporting material

Dataset notes

Sample data for 3 tasks

Challenge setup

The evaluation will assess the hierarchical agent system's ability to produce a nuanced, well-reasoned comparative analysis report on LLM-centric vs. 'world model' approaches to AGI, demonstrating effective use of GPT...

Challenge input

{ 'analysis_scope': 'AGI Paradigm Comparison', 'focus_areas': ['Capabilities', 'Limitations', 'Architectural Principles', 'Path to AGI'] }

Agent execution

No agent tool is listed for this challenge.

Evaluated output

{ 'title': '...', 'introduction': '...', 'sections': [ { 'paradigm': 'LLM-Centric', 'strengths': ['...'], 'weaknesses': ['...'] }, { 'paradigm': 'W...

Evaluation checks
  • Verify that the Comparative Analysis Report follows the specified JSON structure and includes all required...
  • Verify that agents' arguments demonstrate high quality and adherence to perspective, indicating successful...
  • Confirm that the LlamaIndex RAG system successfully retrieves relevant information for agent queries.
Configured evaluation evidence
  • Analysis Depth & Nuance Score target: 0.85
Configured runtime
  • Python execution harness

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

No tool records are attached.

Evaluation contract

  • Verify that the Comparative Analysis Report follows the specified JSON structure and includes all required sections.
  • Verify that agents' arguments demonstrate high quality and adherence to perspective, indicating successful DSPy optimization.
  • Confirm that the LlamaIndex RAG system successfully retrieves relevant information for agent queries.

Metrics

  • Analysis Depth & Nuance Score · Target 0.85
  • Argument Coherence Score · Target 0.9
  • Hybrid Reasoning Utilization · Target 0.75

Workflow tasks

  1. Task 1

    Input: { 'analysis_scope': 'AGI Paradigm Comparison', 'focus_areas': ['Capabilities', 'Limitations', 'Architectural Principles', 'Path to AGI'] }

    Output: { 'title': '...', 'introduction': '...', 'sections': [ { 'paradigm': 'LLM-Centric', 'strengths': ['...'], 'weaknesses': ['...'] }, { 'paradigm': 'World Models', 'strengths': ['...'], 'weaknesses': ['...'] } ], 'hybrid_approaches': ['...'], 'conclusion': '...' }

    Execution harness: Python

  2. Task 2

    Input: { 'topic': 'LLMs for AGI', 'perspective': 'LLM Advocate' }

    Output: { 'llm_argument': '...', 'world_model_counter': '...' }

    Execution harness: Python

  3. Task 3

    Input: { 'query': 'Main criticisms of LLMs for AGI (systems perspective)?' }

    Output: { 'query': '...', 'retrieved_documents': [ { 'doc_id': '...', 'title': '...', 'relevance_score': '...' } ], 'extracted_snippets': ['...'], 'agent_answer_preview': '...' }

    Execution harness: Python

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Evaluate AGI Paradigms with Hierarchical Agents