Back to evaluations
Draft evaluation

Energy Bilateral Trade Agreement Compliance Agent with AutoGen — evaluation

Evaluates clause compliance detection and group chat termination accuracy.

Evaluation type
task based
Challenge
Energy Bilateral Trade Agreement Compliance Agent with AutoGen
Difficulty
Advanced
Rigor
Not declared

The author has not specified a rigor level.

Evaluation overview

How the linked challenge is judged: tasks, benchmarks, and criteria count.

Tasks
1
Benchmarks
0
Criteria
0

Task templates

Inputs and expected outputs.

Task 1

agreement_compliance_check

Tests identification of non-compliant liability caps in $3.7bn trade contracts.

Input format

JSON with agreement_clause_text and target_jurisdiction

Output format

JSON with compliance_status (COMPLIANT|NON_COMPLIANT), flag_reason, and suggested_revision