Agents for Scientific Discovery with Dynamic Knowledge Graphs
Inspired by OpenAI's 'for Science' initiative, this challenge involves building an AutoGen-based multi-agent system designed for abstract scientific discovery and hypothesis generation, specifically in a complex domain like astrophysics (e.g., black holes). The system will feature a collaborative team of agents, powered by OpenAI o3 (or latest available OpenAI model) and Claude Sonnet 4 (or latest available Anthropic model), capable of synthesizing information from vast scientific literature and dynamically building/updating a semantic knowledge graph. The agents will employ extended thinking patterns, iteratively exploring concepts, identifying gaps in current knowledge, and formulating novel hypotheses. A core component will be an adaptive reasoning budget mechanism, allowing agents to allocate computational resources efficiently based on the complexity of the scientific problem or the uncertainty of a hypothesis. The system will integrate with a specialized RAG pipeline to access scientific papers and datasets, and the dynamic knowledge graph will serve as the collective memory and reasoning canvas for the agent team, facilitating complex, multi-hop scientific inquiry.
What you are building
The core problem, expected build, and operating context for this challenge.
Develop an AutoGen multi-agent system with OpenAI o3 and Claude Sonnet 4 for scientific discovery, utilizing a dynamic knowledge graph, extended thinking, and adaptive reasoning budgets.
How work is evaluated
The evaluation will assess the AutoGen system's capacity for scientific synthesis, its ability to dynamically update and leverage a knowledge graph, the effectiveness of extended thinking, and the novelty and coherence of generated hypotheses. Emphasis will be on demonstrating genuine scientific inquiry and collaboration.
Challenge datasets
Public datasets, plus the private datasets you have access to.
What you should walk away with
Build and dynamically update a semantic knowledge graph to serve as a collective memory and reasoning engine for agents.
Implement extended thinking with adaptive reasoning budgets to explore abstract scientific concepts and generate hypotheses.
Formulate novel, testable scientific hypotheses based on multi-agent collaboration and knowledge graph insights.
Develop an advanced RAG pipeline for querying and synthesizing information from scientific literature and datasets.
Design and orchestrate a flexible, conversational multi-agent system using AutoGen for complex scientific inquiry.
Reference links and supporting material
Sample data for 2 tasks
Challenge setup
The evaluation will assess the AutoGen system's capacity for scientific synthesis, its ability to dynamically update and leverage a knowledge graph, the effectiveness of extended thinking, and the novelty and coherenc...
Challenge input
{'paper_snippets': [{'title': 'string', 'text': 'string'}]}
Agent execution
No agent tool is listed for this challenge.
Evaluated output
{'graph_updates': [{'subject': 'string', 'predicate': 'string', 'object': 'string'}]}
- Ensure a coherent, multi-turn conversation between agents leading to a scientific outcome.
- Verify that agents successfully update the knowledge graph with new, correct information during their reaso...
- Confirm implementation of adaptive reasoning budgets or iterative refinement loops for scientific exploration.
- HypothesisCoherence target: 4
- Python execution harness
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- Ensure a coherent, multi-turn conversation between agents leading to a scientific outcome.
- Verify that agents successfully update the knowledge graph with new, correct information during their reasoning.
- Confirm implementation of adaptive reasoning budgets or iterative refinement loops for scientific exploration.
Metrics
- HypothesisCoherence · Target 4
- KnowledgeGraphAccuracy · Target 85
- RAGContextualRelevance · Target 4
Workflow tasks
- Task 1
Input: {'paper_snippets': [{'title': 'string', 'text': 'string'}]}
Output: {'graph_updates': [{'subject': 'string', 'predicate': 'string', 'object': 'string'}]}
Execution harness: Python
- Task 2
Input: {'research_question': 'string', 'knowledge_graph_state': {'nodes': [], 'edges': []}, 'rag_results': [{'doc_id': 'string', 'snippet': 'string'}]}
Output: {'hypothesis_title': 'string', 'hypothesis_statement': 'string', 'supporting_evidence_ids': ['string'], 'gaps_identified': ['string']}
Execution harness: Python
Recipe freeze marker
No recipe freeze time is recorded.