Skip to main content
Back to challenges
Challenge

Robotics & Biotech Research Navigator Agent

Inspired by the advancements in robotics foundation models and the push for AI in traditional Chinese medicine, this challenge focuses on building a sophisticated multi-agent research system. Your task is to design and implement an autonomous research navigator that can ingest vast amounts of scientific literature (e.g., papers on robotics, biotechnology, or drug discovery), identify key trends, synthesize novel insights, and generate structured summaries or reports. Utilizing LangGraph, you will orchestrate a team of specialized agents, each with a distinct role—such as a 'Researcher' for information gathering, an 'Analyst' for data interpretation, and a 'Synthesizer' for report generation. The system should manage complex, stateful workflows, allowing agents to collaborate, iterate on findings, and dynamically adapt their research path based on intermediate results. This challenge emphasizes robust information retrieval, advanced reasoning, and structured output generation for scientific applications.

Special Purpose AgentsHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Orchestrate a multi-agent research navigator using LangChain and LangGraph to analyze scientific literature and synthesize insights.

Delivery guide

How work is evaluated

Evaluation

Evaluation will focus on the completeness, accuracy, and insightfulness of the generated research reports. It will also assess the efficiency of the multi-agent collaboration and tool usage.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
5 scoring checks
Binary
5 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

ReportCompleteness

All required sections in the output JSON are present and non-empty.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

KeyInformationPresence

Specific critical information (e.g., 'key_challenges' or 'ai_techniques_used') is identified and listed.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 3

FactualAccuracy

Percentage of extracted facts or statements that are verifiably true (0-1). • target: 0.9 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 4

InsightfulnessScore

Subjective score for the depth and novelty of the insights generated (1-5). • target: 4 • range: 1-5

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 5

ToolUsageEfficiency

Number of relevant tool calls per query compared to a baseline (ratio). • target: 1 • range: 0.5-2

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Master LangGraph for defining stateful, cyclic agentic workflows, including nodes, edges, and state management for complex research tasks

  • Implement distinct agent roles (e.g., 'Researcher', 'Analyst', 'Synthesizer') within LangGraph, assigning specialized tools and prompts to each for optimal performance with Claude Opus 4.1

  • Integrate `SerpAPI` for real-time, comprehensive web search and `ChromaDB` for indexing and retrieving domain-specific documents (e.g., scientific papers) within agent workflows

  • Design sophisticated prompt engineering strategies for each agent role, focusing on information extraction, critical analysis, and structured synthesis outputs from Claude Opus 4.1

  • Develop mechanisms for inter-agent communication and dynamic routing within the LangGraph framework, allowing agents to pass structured information and trigger subsequent actions based on intermediate findings

  • Deploy agent components and their dependencies using `Docker` for consistent environments and scalable execution of research workflows

  • Utilize `Arize AI` for observability and evaluation of agent performance, tracking metrics like information recall, summarization quality, and reasoning accuracy across research iterations

  • Implement a mechanism for the 'Synthesizer' agent to generate structured research reports, potentially using Pydantic models for output validation, based on the collective findings of the agent team

Resources and assets

Reference links and supporting material

Dataset notes

Sample data for 2 tasks

Challenge setup

Evaluation will focus on the completeness, accuracy, and insightfulness of the generated research reports. It will also assess the efficiency of the multi-agent collaboration and tool usage.

Challenge input

{'research_query': 'string', 'paper_urls': 'list_of_strings'}

Agent execution

No agent tool is listed for this challenge.

Evaluated output

{'summary': 'string', 'key_challenges': 'list_of_strings', 'future_directions': 'list_of_strings'}

Evaluation checks
  • All required sections in the output JSON are present and non-empty.
  • Specific critical information (e.g., 'key_challenges' or 'ai_techniques_used') is identified and listed.
  • Percentage of extracted facts or statements that are verifiably true (0-1). • target: 0.9 • range: 0-1
Configured evaluation evidence
  • FactualAccuracy target: 0.9
  • 2 public reference cases
Configured runtime
  • Python execution harness

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

No tool records are attached.

Evaluation contract

  • ReportCompleteness · Weight 1
  • KeyInformationPresence · Weight 1
  • FactualAccuracy · Weight 1
  • InsightfulnessScore · Weight 1
  • ToolUsageEfficiency · Weight 1

Metrics

  • FactualAccuracy · Target 0.9
  • InsightfulnessScore · Target 4
  • ToolUsageEfficiency · Target 1

Reference cases

  • 2 public reference cases

Workflow tasks

  1. Task 1

    Input: {'research_query': 'string', 'paper_urls': 'list_of_strings'}

    Output: {'summary': 'string', 'key_challenges': 'list_of_strings', 'future_directions': 'list_of_strings'}

    Execution harness: Python

  2. Task 2

    Input: {'research_query': 'string', 'search_keywords': 'list_of_strings'}

    Output: {'report_sections': {'introduction': 'string', 'ai_techniques_used': 'list_of_strings', 'potential_compounds': 'list_of_strings', 'challenges': 'list_of_strings'}}

    Execution harness: Python

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Robotics & Biotech Research Navigator Agent