Hybrid Reasoning AI Browser for Real-time Synthesis with Gemini 2.5 Pro & DSPy
Similar to leading AI browsers, this challenge tasks you with building an advanced information synthesis engine. You will create a system that can browse the web, extract critical information, and provide real-time, verified answers to complex queries, leveraging a hybrid reasoning approach. Your solution will combine the 'instant' reasoning of OpenAI GPT 5 for quick summarization with the 'deep' reasoning capabilities of Gemini 2.5 Pro, optimized using DSPy for robust ReAct agent patterns. The system will employ adaptive thinking budgets, ensuring efficient resource use while maintaining high accuracy and providing source verification through RAG.
What you are building
The core problem, expected build, and operating context for this challenge.
Develop a hybrid instant/deep reasoning AI browser using Gemini 2.5 Pro and DSPy with ReAct agents for real-time, verified information synthesis.
How work is evaluated
The evaluation will focus on the accuracy, completeness, and verifiability of the synthesized answers, as well as the efficiency of the hybrid reasoning process. It will assess how well the system uses its tools and adaptive budgets.
Challenge datasets
Public datasets, plus the private datasets you have access to.
What you should walk away with
Implement a ReAct agent architecture for dynamic web browsing and information extraction.
Optimize agent prompts and logic using DSPy for robust performance and reduced hallucinations.
Integrate adaptive thinking budgets to dynamically manage LLM usage based on query complexity.
Design a hybrid reasoning system utilizing both instant (OpenAI o3) and deep (Gemini 2.5 Pro) models.
Develop a robust RAG pipeline for source verification and answer grounding.
Reference links and supporting material
Sample data for 2 tasks
Challenge setup
The evaluation will focus on the accuracy, completeness, and verifiability of the synthesized answers, as well as the efficiency of the hybrid reasoning process. It will assess how well the system uses its tools and a...
Challenge input
String representing the user query.
Agent execution
No agent tool is listed for this challenge.
Evaluated output
JSON object with 'answer', 'sources' (list of URLs), 'reasoning_trace' (steps taken by agents).
- The generated answer is factually correct.
- At least 2 relevant and valid sources are provided.
- Evidence of 'Thought, Action, Observation' in reasoning_trace.
- Answer Completeness target: 90
- Python execution harness
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- The generated answer is factually correct.
- At least 2 relevant and valid sources are provided.
- Evidence of 'Thought, Action, Observation' in reasoning_trace.
- Both instant and deep models are leveraged appropriately for complex queries.
Metrics
- Answer Completeness · Target 90
- Hallucination Rate · Target 5
- Token Efficiency (Complex) · Target 1500
Workflow tasks
- Task 1
Input: String representing the user query.
Output: JSON object with 'answer', 'sources' (list of URLs), 'reasoning_trace' (steps taken by agents).
Execution harness: Python
- Task 2
Input: String representing the user query.
Output: JSON object with 'query', 'token_usage_total', 'response_time_ms', 'deep_reasoning_invoked'.
Execution harness: Python
Recipe freeze marker
No recipe freeze time is recorded.