Evaluate AGI Paradigms with Hierarchical Agents
The debate around Artificial General Intelligence (AGI) continues, with prominent figures like Yann LeCun questioning the path of LLM-centric approaches and advocating for 'world models'. This challenge invites you to build a cutting-edge hierarchical multi-agent research system to go into this philosophical and architectural divide. Your system will use GPT-5 for its unparalleled reasoning and creative synthesis capabilities, allowing agents to generate complex arguments and hypothetical designs. DSPy will be crucial for programmatically optimizing prompts, ensuring agents produce high-quality, nuanced outputs when discussing abstract AI concepts. LlamaIndex will serve as the backbone for building a hierarchical agent structure and a comprehensive RAG knowledge base, enabling agents to conduct thorough research, debate, and synthesize findings. The ultimate goal is a sophisticated comparative analysis report on the viability and limitations of LLM-centric vs. 'world model' approaches to AGI.
What you are building
The core problem, expected build, and operating context for this challenge.
The debate around Artificial General Intelligence (AGI) continues, with prominent figures like Yann LeCun questioning the path of LLM-centric approaches and advocating for 'world models'. This challenge invites you to build a cutting-edge hierarchical multi-agent research system to go into this philosophical and architectural divide. Your system will use GPT-5 for its unparalleled reasoning and creative synthesis capabilities, allowing agents to generate complex arguments and hypothetical designs. DSPy will be crucial for programmatically optimizing prompts, ensuring agents produce high-quality, nuanced outputs when discussing abstract AI concepts. LlamaIndex will serve as the backbone for building a hierarchical agent structure and a comprehensive RAG knowledge base, enabling agents to conduct thorough research, debate, and synthesize findings. The ultimate goal is a sophisticated comparative analysis report on the viability and limitations of LLM-centric vs. 'world model' approaches to AGI.
Shared data for this challenge
Review public datasets and any private uploads tied to your build.
What you should walk away with
Master LlamaIndex for building hierarchical agent systems, where specialized 'researcher' agents (e.g., 'LLM Advocate', 'World Model Proponent', 'Critique Agent') report to a higher-level 'Synthesis Agent' to manage research objectives and consolidate findings.
Implement DSPy to programmatically optimize prompts for GPT-5, creating robust 'modules' for tasks like argument generation, counter-argument formulation, and conceptual synthesis, ensuring consistency and quality in agent outputs when discussing complex AI philosophies.
Leverage GPT-5's advanced reasoning capabilities for deep conceptual exploration, enabling agents to generate hypothetical architectural designs for 'world models' and critically analyze the emergent limitations and scaling laws of current LLMs for AGI.
Build a comprehensive RAG system using LlamaIndex, ingesting academic papers, expert opinions, and historical AI research (e.g., Yann LeCun's publications) to provide a rich and up-to-date context for the agents' discussions and analyses.
Orchestrate agent-to-agent communication within the LlamaIndex framework, facilitating a structured debate or collaborative research process where agents can share findings, challenge assumptions, and iteratively refine their understanding of AGI paradigms.
Design an evaluation strategy for the agent system's outputs, focusing on the coherence, depth, neutrality, and originality of its comparative analysis report, assessing its ability to articulate both strengths and weaknesses of each AGI approach.
How this agent runs
The evaluation will assess the hierarchical agent system's ability to produce a nuanced, well-reasoned comparative analysis report on LLM-centric vs. 'world model' approaches to AGI, demonstrating effective use of GPT...
Challenge input
{ 'analysis_scope': 'AGI Paradigm Comparison', 'focus_areas': ['Capabilities', 'Limitations', 'Architectural Principles', 'Path to AGI'] }
Agent execution
The configured agent processes the input under the challenge policy.
Evaluated output
{ 'title': '...', 'introduction': '...', 'sections': [ { 'paradigm': 'LLM-Centric', 'strengths': ['...'], 'weaknesses': ['...'] }, { 'paradigm': 'W...
- Verify that the Comparative Analysis Report follows the specified JSON structure and includes all required...
- Verify that agents' arguments demonstrate high quality and adherence to perspective, indicating successful...
- Confirm that the LlamaIndex RAG system successfully retrieves relevant information for agent queries.
- Analysis Depth & Nuance Score target: 0.85
- Python execution harness
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- The evaluation module defines the checks.
Recipe state
This is a preview. The configuration can change before the evaluation recipe is locked.
Run this agent on your dataset and AI stack
Bring your dataset, model providers, and success criteria. We will scope the right managed run for your team.
Scope a managed run[ok] Wrote CHALLENGE.md
[ok] Wrote .versalist.json
[ok] Wrote eval/examples.json
Requires VERSALIST_API_KEY. Works with any MCP-aware editor.
DocsFind another challenge
Jump to a random challenge when you want a fresh benchmark or a different problem space.