Gas Field Expansion Contract Risk Analyzer using LlamaIndex and Claude 4.1 Opus
Complex gas processing contract expansions require analyzing thousands of engineering standards and change order clauses under tight schedules. Build an advanced LlamaIndex RAG pipeline powered by Claude 4.1 Opus to extract schedule risk factors from contractual documentation, identifying schedule delay drivers with a baseline recall score above 90%.
What you are building
The core problem, expected build, and operating context for this challenge.
Develop an advanced contract risk extraction pipeline utilizing LlamaIndex property graphs and Claude 4.1 Opus for mega gas field expansion engineering contracts.
How work is evaluated
Evaluates contract risk extraction recall and faithfulness on Aramco Jafurah EPC clauses.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
risk_penalty_detection
Ensure penalty percentage and max cap are correctly extracted
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
risk_extraction_recall
Recall metric of identified risk factors compared to standard ground truth • target: 0.92 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Build a hierarchical LlamaIndex VectorStoreIndex and PropertyGraphIndex over EPC contract specs
Configure Claude 4.1 Opus as the primary LLM engine for complex clause reasoning
Extract delay risk penalties, force majeure, and long-lead material clauses into dynamic risk trees
Optimize retrieval chunking strategies using sub-question query engines
Reference links and supporting material
EPC contract clauses, schedule penalty specs, and change order templates for Aramco Jafurah gas expansion project.
Challenge setup
Evaluates contract risk extraction recall and faithfulness on Aramco Jafurah EPC clauses.
Challenge input
Raw text excerpt from EPC contract phase IV agreement.
LlamaIndex
Required data framework for advanced RAG indexing
Claude 4.1 Opus
Required LLM model for high-reasoning contract parsing
Evaluated output
JSON containing detected risk clauses, max delay penalty %, and mitigation recommendations.
- Ensure penalty percentage and max cap are correctly extracted
- Recall metric of identified risk factors compared to standard ground truth • target: 0.92 • range: 0-1
- Benchmark: Legal RAG Analysis Benchmark
- Risk Extraction Recall target: 0.92
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- Llama Index · Required
- Exa · Optional
- Claude 4.1 Opus · Optional
Evaluation contract
- risk_penalty_detection · Weight 1
- risk_extraction_recall · Weight 1
Metrics
- risk_extraction_recall · Target 0.92
Benchmarks
- Legal RAG Analysis Benchmark
- general
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: Raw text excerpt from EPC contract phase IV agreement.
Output: JSON containing detected risk clauses, max delay penalty %, and mitigation recommendations.
Execution harness: Python
Evaluation tools
- LlamaIndex · Agent Framework
- Claude 4.1 Opus · Model Runtime
Recipe freeze marker
No recipe freeze time is recorded.