International Treaty Arbitral Claims & Legal Discovery Risk Evaluator
CK Hutchison filed a $1.5 billion international treaty arbitration case against Panama regarding container terminal concession rights. Analyzing multi-billion dollar international investment disputes requires cross-referencing bilateral investment treaties (BITs), concession contract clauses, and historical ICSID arbitral awards. In this challenge, you will implement an international arbitration discovery agent using the Claude Agents SDK. The agent processes complex treaty text, extracts expropriation and fair-and-equitable-treatment (FET) claims, and scores claim exposure probability using structured legal reasoning chains.
What you are building
The core problem, expected build, and operating context for this challenge.
Build an international treaty arbitration discovery agent with Claude Agents SDK to evaluate concession contract dispute risks.
How work is evaluated
Evaluates legal claim extraction accuracy and damage probability estimation.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
breach_category_match
Must correctly classify breach as Indirect Expropriation.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
merit_score_calibration
Absolute difference between calculated merit score and expert baseline • target: 0.05 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Construct Claude Agents SDK workflows for deep legal document analysis
Extract Fair and Equitable Treatment (FET) and Indirect Expropriation claim clauses
Map legal claim elements against historical ICSID precedent databases
Generate structured claim valuation risk assessments for legal counsel
Reference links and supporting material
International arbitration dataset containing Bilateral Investment Treaty texts, concession agreements, and ICSID award summaries.
Challenge setup
Evaluates legal claim extraction accuracy and damage probability estimation.
Challenge input
JSON containing concession agreement excerpt and treaty dispute claim summary
Claude Agents SDK
Required framework for deep legal reasoning and extended thinking analysis.
Evaluated output
JSON containing extracted breach category, estimated claim merit score, and precedent citations
- Must correctly classify breach as Indirect Expropriation.
- Absolute difference between calculated merit score and expert baseline • target: 0.05 • range: 0-1
- Benchmark: LegalBench-Arbitration
- Merit Score Calibration target: 0.05
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- Aim · Required
- Langroid · Optional
- Arize Phoenix · Optional
- Arize Phoenix · Optional
Evaluation contract
- breach_category_match · Weight 1
- merit_score_calibration · Weight 1
Metrics
- merit_score_calibration · Target 0.05
Benchmarks
- LegalBench-Arbitration
- general
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON containing concession agreement excerpt and treaty dispute claim summary
Output: JSON containing extracted breach category, estimated claim merit score, and precedent citations
Execution harness: Python
Evaluation tools
- Claude Agents SDK · Agent Framework
Recipe freeze marker
No recipe freeze time is recorded.