Skip to main content
Back to challenges
Challenge

Bilingual Iraqi Oil Contract Clause Extraction & Exception Handler with LangChain

Oil trade contracts written under Iraqi law feature specialized legal Arabic clauses regarding force majeure, port congestion waivers, and currency conversion terms. Build a LangChain extraction agent with custom fallback chains. The agent processes raw contract text, extracts key liability thresholds, flags non-standard exception clauses, and maps Arabic legal phrasing to standard English international oil trade terms.

Business OperationsHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Build a bilingual legal clause extraction agent using LangChain Expression Language (LCEL).

Delivery guide

How work is evaluated

Evaluation

Evaluates structured clause extraction precision from Arabic petroleum legal agreements.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

notice_period_check

Ensure notice period in hours is extracted accurately.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

clause_extraction_f1

F1 score across extracted contract clauses. • target: 0.95 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Construct LCEL chains using custom RunnableParallel and RunnableLambda functions.

  • Extract structured liability clauses from bilingual petroleum agreements.

  • Implement exception routing for ambiguous force majeure clauses.

Resources and assets

Reference links and supporting material

Dataset notes

25 Arabic-English oil contract clause pairs extracted from regional trade records.

Challenge setup

Evaluates structured clause extraction precision from Arabic petroleum legal agreements.

Challenge input

JSON with contract_text_ar

LangChain

Provides declarative syntax for multi-stage text processing.

Evaluated output

JSON with force_majeure_days, notice_period_hours, requires_somo_written_approval

Evaluation checks
  • Ensure notice period in hours is extracted accurately.
  • F1 score across extracted contract clauses. • target: 0.95 • range: 0-1
Configured evaluation evidence
  • Benchmark: LCEL-Extract-Bench
  • Clause Extraction F1 target: 0.95
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Langchain · Required
  • LangChain · Optional
  • Zed · Optional
Orchestration
  • Langchain · Required
  • LangChain · Optional

Evaluation contract

  • notice_period_check · Weight 1
  • clause_extraction_f1 · Weight 1

Metrics

  • clause_extraction_f1 · Target 0.95

Benchmarks

  • LCEL-Extract-Bench
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON with contract_text_ar

    Output: JSON with force_majeure_days, notice_period_hours, requires_somo_written_approval

    Execution harness: Python

Evaluation tools

  • LangChain · Chain Composition Engine

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Bilingual Iraqi Oil Contract Clause Extraction & Exception Handler with LangChain