Skip to main content
Back to challenges
Challenge

Build a Context-Aware Personal Knowledge Agent

This challenge involves building a sophisticated Personal Knowledge Agent. This agent will leverage a multi-modal RAG system to process uploaded documents, emails, and images, creating a deeply personalized understanding of user context. You will design a graph-based agent workflow using LangGraph, enabling extended thinking and adaptive reasoning budgets. The agent should be capable of synthesizing information from diverse sources and performing complex tasks by integrating with simulated external 'enterprise' systems via a robust MCP tool integration layer.

Special Purpose AgentsHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Build a graph-based multi-agent system using LangGraph with GPT-5.2 Pro for autonomous knowledge synthesis and task execution, inspired by Alexa+'s new capabilities.

Delivery guide

How work is evaluated

Evaluation

The evaluation will assess the agent's ability to accurately synthesize information from diverse inputs, intelligently plan tasks, and robustly execute actions via simulated MCP tools. Emphasis will be on context retention, reasoning depth, and adaptability to new information.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Learning goals

What you should walk away with

  • Utilize extended thinking with adaptive reasoning budgets for complex problem-solving.

  • Integrate MCP-enabled tools for simulated task execution based on user context.

  • Design and implement a graph-based multi-agent system for personalized knowledge management.

  • Master programmatic prompting and prompt optimization with DSPy for agent efficiency.

  • Develop a multimodal RAG pipeline to process and synthesize information from documents, emails, and images.

Resources and assets

Reference links and supporting material

Dataset notes

Sample data for 2 tasks

Challenge setup

The evaluation will assess the agent's ability to accurately synthesize information from diverse inputs, intelligently plan tasks, and robustly execute actions via simulated MCP tools. Emphasis will be on context rete...

Challenge input

{'documents': [{'content': '...', 'type': 'pdf/text'}], 'emails': [{'subject': '...', 'body': '...'}], 'images': [{'description': '...', 'tags': '....

Agent execution

No agent tool is listed for this challenge.

Evaluated output

{'summary': '...', 'related_projects': ['...', '...'], 'visual_references': ['...']}

Evaluation checks
  • Agent correctly identifies the appropriate MCP tool for the task.
  • All required parameters for the MCP tool are correctly extracted from context.
  • The generated summary in synthesis tasks is logically sound and accurate.
Configured evaluation evidence
  • F1 Score For Information Extraction target: 0.85
Configured runtime
  • Python execution harness

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

No tool records are attached.

Evaluation contract

  • Agent correctly identifies the appropriate MCP tool for the task.
  • All required parameters for the MCP tool are correctly extracted from context.
  • The generated summary in synthesis tasks is logically sound and accurate.
  • Information from all relevant modalities (text, email, image descriptions) is used when required.

Metrics

  • F1-Score for Information Extraction · Target 0.85
  • Reasoning Steps (Adaptive Budget Efficiency) · Target 4
  • Task Completion Rate (MCP) · Target 90

Workflow tasks

  1. Task 1

    Input: {'documents': [{'content': '...', 'type': 'pdf/text'}], 'emails': [{'subject': '...', 'body': '...'}], 'images': [{'description': '...', 'tags': '...'}], 'query': 'Summarize all projects related to AI initiatives mentioned across documents and emails, and identify any visual references in images.'}

    Output: {'summary': '...', 'related_projects': ['...', '...'], 'visual_references': ['...']}

    Execution harness: Python

  2. Task 2

    Input: {'uploaded_context_id': 'unique_id', 'request': 'Draft an email to Dr. Smith regarding Project Athena's budget update to $5.5M for LLM integration.'}

    Output: {'tool_name': 'email_sender_mcp_tool', 'tool_parameters': {'recipient': 'dr.smith@example.com', 'subject': 'Project Athena Budget Update', 'body': 'Dear Dr. Smith, Project Athena's budget has been revised to $5.5M due to new LLM integration efforts.'}, 'simulated_tool_output': 'Email to dr.smith@example.com sent successfully.'}

    Execution harness: Python

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Build a Context-Aware Personal Knowledge Agent