LegalQueryResponseAccuracy
The evaluation will focus on the agent's ability to provide accurate, relevant, and hallucination-free legal advice for probate-related queries, specifically assessing its self-correction mechanisms, data retrieval efficacy, and the integration of A2A verification.
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Task templates
Inputs and expected outputs.
LegalQueryResponseAccuracy
Evaluates the factual accuracy, completeness, and hallucination rate of the agent's response to a complex probate query, along with its ability to cite sources and demonstrate A2A verification.
{"query": "string", "context_docs": ["string", ...], "expected_facts": [{"fact": "string", "critical": "boolean"}, ...]}
{"response": "string", "cited_sources": [{"source_name": "string", "confidence": "float"}], "self_correction_logs": "string", "a2a_verification_status": "string"}