Skip to main content
Loading navigation
Back to challenges

Jafurah Gas Field Expansion Risk Evaluation with Pydantic AI

Evaluating contractor bids for Saudi Aramco's Jafurah fifth expansion phase requires type-safe extraction of reservoir pressure ratings and execution timelines. Build a Pydantic AI agent that enforces strong type hints and Pydantic schema validation for bid technical submittals. Achieve 100% type validation compliance with zero unhandled schema errors.

Business OperationsHosted by Vera
Sign in free to start

What you are building

Develop a type-safe Pydantic AI evaluation agent for Jafurah gas expansion contractor bids.

How work is evaluated

Evaluation

Evaluates Pydantic AI agent schema enforcement and bid evaluation accuracy.

Datasets

Public datasets, plus the private datasets you have access to.

How submissions are scored

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

Schema Validation Check

Verifies that output passes Pydantic data type validation without raising ValidationError.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

Schema Parsing Success Rate

Percentage of agent evaluations successfully parsed into Pydantic models. • target: 1 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

  • Implement Pydantic AI `Agent` with structured response validation

  • Validate technical drilling parameters against Saudi Aramco safety specs

  • Inject external dependency tools for live cost-index lookups

  • Generate clean JSON evaluation reports guaranteed to match target schema

Resources

Dataset notes

Contractor bid proposals narrative text for Saudi Aramco Jafurah unconventional gas field expansion.

Challenge setup

Evaluates Pydantic AI agent schema enforcement and bid evaluation accuracy.

Challenge input

Text prompt containing contractor proposal narrative

Pydantic AI

Assigned framework for structured validation.

Evaluated output

Validated JSON object adhering to JafurahBidEval Pydantic model

Evaluation checks
  • Verifies that output passes Pydantic data type validation without raising ValidationError.
  • Percentage of agent evaluations successfully parsed into Pydantic models. • target: 1 • range: 0-1
Configured evaluation evidence
  • Benchmark: PydanticBench
  • Schema Parsing Success Rate target: 100%
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Shows the first task, up to three tools, and up to three evidence items.

View technical recipe

Configured tools

Action Space
  • Pydantic AI · Required
  • AutoML (H2O) · Optional
  • All Hands AI · Optional
Orchestration
  • Pydantic AI · Required

Evaluation contract

  • Schema Validation Check · Weight 1
  • Schema Parsing Success Rate · Weight 1

Metrics

  • Schema Parsing Success Rate · Target 1

Benchmarks

  • PydanticBench
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: Text prompt containing contractor proposal narrative

    Output: Validated JSON object adhering to JafurahBidEval Pydantic model

    Execution harness: Python

Evaluation tools

  • Pydantic AI · Type-Safe Agent Framework

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Jafurah Gas Field Expansion Risk Evaluation with Pydantic AI