Back to evaluations
Public evaluation
agreement_compliance_check
Evaluates clause compliance detection and group chat termination accuracy.
Evaluation type
task based
Challenge
Energy Bilateral Trade Agreement Compliance Agent with AutoGen
Difficulty
Advanced
Rigor
Unspecified
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Tasks
1
Benchmarks
0
Criteria
0
Task templates
Inputs and expected outputs.
Task 1
agreement_compliance_check
Tests identification of non-compliant liability caps in $3.7bn trade contracts.
Input format
JSON with agreement_clause_text and target_jurisdiction
Output format
JSON with compliance_status (COMPLIANT|NON_COMPLIANT), flag_reason, and suggested_revision