Graph-Based Legal Aid
This challenge tasks you with building a robust, hallucination-resistant legal aid agent system. You will design a graph-based workflow using LangGraph, leveraging Claude Opus 4.5 for nuanced legal reasoning and LlamaIndex for advanced RAG over legal documents. The system must incorporate self-correction mechanisms via A2A protocol for agent verification, and adaptive thinking budgets to ensure accuracy and timely responses in complex probate cases. The core focus is on mitigating factual errors and improving response relevance in sensitive legal contexts. You will integrate MCP-enabled tools for secure access to enterprise legal data and orchestrate the multi-agent system using Letta AI for comprehensive performance monitoring and evaluation, pushing the boundaries of reliable generative AI in public services.
What you are building
The core problem, expected build, and operating context for this challenge.
Build a graph-based, hallucination-resistant multi-agent legal aid system using LangGraph, Claude Opus 4.1, and Letta AI, with advanced RAG and A2A protocol for self-correction.
How work is evaluated
The evaluation will focus on the agent's ability to provide accurate, relevant, and hallucination-free legal advice for probate-related queries, specifically assessing its self-correction mechanisms, data retrieval efficacy, and the integration of A2A verification.
Shared data for this challenge
Review public datasets and any private uploads tied to your build.
What you should walk away with
Implement robust RAG for hallucination mitigation in legal contexts using LlamaIndex.
Integrate MCP-enabled tools for secure court system data access.
Utilize A2A protocol for agent self-correction and verification.
Orchestrate and evaluate agent performance with Letta AI.
Design a graph-based multi-agent architecture for legal aid with self-correction.
Reference links and supporting material
Sample data for 1 tasks
How this agent runs
The evaluation will focus on the agent's ability to provide accurate, relevant, and hallucination-free legal advice for probate-related queries, specifically assessing its self-correction mechanisms, data retrieval ef...
Challenge input
{"query": "string", "context_docs": ["string", ...], "expected_facts": [{"fact": "string", "critical": "boolean"}, ...]}
Agent execution
The configured agent processes the input under the challenge policy.
Evaluated output
{"response": "string", "cited_sources": [{"source_name": "string", "confidence": "float"}], "self_correction_logs": "string", "a2a_verification_sta...
- The response must not contain factual information unsupported by the provided context or general legal prin...
- All critical expected facts must be present and correctly explained in the response.
- The 'a2a_verification_status' must indicate successful verification or a clear attempt at agent-to-agent co...
- Factual Accuracy Score target: 0.95
- Python execution harness
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- The evaluation module defines the checks.
Recipe state
This is a preview. The configuration can change before the evaluation recipe is locked.