Back to evaluations
Public evaluation

agreement_compliance_check

Evaluates clause compliance detection and group chat termination accuracy.

Evaluation type
task based
Challenge
Energy Bilateral Trade Agreement Compliance Agent with AutoGen
Difficulty
Advanced
Rigor
Unspecified

Evaluation overview

How the linked challenge is judged: tasks, benchmarks, and criteria count.

Tasks
1
Benchmarks
0
Criteria
0

Task templates

Inputs and expected outputs.

Task 1

agreement_compliance_check

Tests identification of non-compliant liability caps in $3.7bn trade contracts.

Input format

JSON with agreement_clause_text and target_jurisdiction

Output format

JSON with compliance_status (COMPLIANT|NON_COMPLIANT), flag_reason, and suggested_revision