Skip to main content
Back to challenges
Challenge

Automated Crude Loading Manifest Validation Pipeline with Pydantic AI

Before an oil tanker can depart terminal waters, its bill of lading, customs clearance, and quality manifest must match export quota allowances down to the net standard volume (NSV). Build an automated validation agent with Pydantic AI. The agent receives manifest inputs, checks them against interim 3-month export license limits, and validates volume units across metric tons, long tons, and standard US barrels.

Business OperationsHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Develop an export loading manifest validation system using Pydantic AI schema enforcement.

Delivery guide

How work is evaluated

Evaluation

Evaluates loading manifest volumetric compliance against quota.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

quota_exceeded_false

Ensure manifest within quota gets valid clearance.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

volumetric_calculation_precision

Precision of calculated barrels from metric tons. • target: 0.99 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Use Pydantic AI to enforce multi-field numerical consistency.

  • Implement automatic volumetric conversions for crude oil quantities.

  • Generate export clearance tokens for valid manifests.

Resources and assets

Reference links and supporting material

Dataset notes

50 manifest records containing tonnage, API gravity, and quota limits.

Challenge setup

Evaluates loading manifest volumetric compliance against quota.

Challenge input

JSON with metric_tons, api_gravity, quota_max_bbls

Pydantic AI

Type-safe validation engine for structured agent outputs.

Evaluated output

JSON with calculated_bbls, quota_exceeded, clearance_valid

Evaluation checks
  • Ensure manifest within quota gets valid clearance.
  • Precision of calculated barrels from metric tons. • target: 0.99 • range: 0-1
Configured evaluation evidence
  • Benchmark: Pydantic-Validation-Bench
  • Volumetric Calculation Precision target: 0.99
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Pydantic AI · Required
  • VAPI · Optional
Observation
  • Langfuse · Optional
Reward / Eval
  • Langfuse · Optional
Orchestration
  • Pydantic AI · Required

Evaluation contract

  • quota_exceeded_false · Weight 1
  • volumetric_calculation_precision · Weight 1

Metrics

  • volumetric_calculation_precision · Target 0.99

Benchmarks

  • Pydantic-Validation-Bench
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON with metric_tons, api_gravity, quota_max_bbls

    Output: JSON with calculated_bbls, quota_exceeded, clearance_valid

    Execution harness: Python

Evaluation tools

  • Pydantic AI · Schema Engine

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Automated Crude Loading Manifest Validation Pipeline with Pydantic AI