ComplexQuerySynthesis
The evaluation will focus on the accuracy, completeness, and verifiability of the synthesized answers, as well as the efficiency of the hybrid reasoning process. It will assess how well the system uses its tools and adaptive budgets.
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Task templates
Inputs and expected outputs.
ComplexQuerySynthesis
Provide a complex, multi-faceted query requiring web browsing and synthesis from multiple sources.
String representing the user query.
JSON object with 'answer', 'sources' (list of URLs), 'reasoning_trace' (steps taken by agents).
AdaptiveBudgetEfficiency
Submit queries of varying complexity and evaluate the token usage and response time.
String representing the user query.
JSON object with 'query', 'token_usage_total', 'response_time_ms', 'deep_reasoning_invoked'.