Skip to main content
Back to challenges
Challenge

LangChain-Powered Megaproject Procurement Risk Agent for Qiddiya Infrastructure

Megaproject developments such as Saudi Arabia's Qiddiya City require strict oversight of specialised material supply chains, international stone sourcing, and subcontractor deliveries. Managing complex procurement workflows across bilingual (Arabic/English) contractual documents, material test certificates, and shipping manifests demands stateful agentic automation to preempt delivery bottlenecks and compliance failures. In this challenge, you will construct an automated procurement risk intelligence system using LangChain, LangGraph, and Langfuse. The agent orchestrates dynamic document parsing, bilingual risk assessment, and supplier verification against GCC regulatory standards. By leveraging stateful graph workflows, your agent will process material requisition orders, identify potential delivery delays, and generate actionable risk mitigation strategies. You will also integrate Langfuse for end-to-end tracing and evaluation of model decisions, prompt iterations, and execution latency. The resulting solution will process multi-modal procurement streams and issue structured risk scores and compliance recommendations for site engineering managers.

Business OperationsHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Build a stateful LangChain and LangGraph agent observed via Langfuse to analyze megaproject material procurement risks and compliance across GCC supply chain streams.

Delivery guide

How work is evaluated

Evaluation

Evaluates the agent's stateful graph execution, risk scoring accuracy, and Langfuse tracing integration.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
3 scoring checks
Binary
3 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

valid_pydantic_schema

Checks if output conforms to required JSON output schema.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

risk_score_bounds

Ensures risk score is a float within [0.0, 1.0].

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 3

risk_precision

Measures alignment of identified risk factors with expected ground truth risks. • target: 0.95 • range: 0.8-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Build stateful agent workflows using LangGraph and LangChain for multi-step procurement risk analysis.

  • Implement structured output generation for material compliance verification and delivery risk scoring.

  • Integrate Langfuse tracing to monitor LLM tool calls, prompt latency, and agent decision pathways.

  • Process bilingual (Arabic/English) procurement documentation and material specification sheets.

Resources and assets

Reference links and supporting material

Dataset notes

Dataset containing 150 synthetic and historical GCC megaproject purchase orders, material safety certificates, quarry logs, and shipping manifests in English and Arabic.

Challenge setup

Evaluates the agent's stateful graph execution, risk scoring accuracy, and Langfuse tracing integration.

Challenge input

JSON object containing PO details, supplier origin, material spec, and shipment logs

LangChain

Provides robust tool abstractions and integration patterns for building dynamic cognitive agents.

LangGraph

Enables resilient cyclic state graphs for complex procurement reasoning loops.

Langfuse

Captures full execution traces, latency, token metrics, and tool execution logs.

Evaluated output

JSON object containing risk_score, risk_factors, compliance_status, and mitigation_plan

Evaluation checks
  • Checks if output conforms to required JSON output schema.
  • Ensures risk score is a float within [0.0, 1.0].
  • Measures alignment of identified risk factors with expected ground truth risks. • target: 0.95 • range: 0.8-1
Configured evaluation evidence
  • Benchmark: GCC Infrastructure Procurement Benchmark v1
  • Risk Precision target: 0.95
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Langchain · Required
  • LangChain · Optional
Observation
  • Langfuse · Optional
Reward / Eval
  • Langfuse · Optional
Orchestration
  • Langchain · Required
  • LangChain · Optional

Evaluation contract

  • valid_pydantic_schema · Weight 1
  • risk_score_bounds · Weight 1
  • risk_precision · Weight 1

Metrics

  • risk_precision · Target 0.95

Benchmarks

  • GCC Infrastructure Procurement Benchmark v1
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON object containing PO details, supplier origin, material spec, and shipment logs

    Output: JSON object containing risk_score, risk_factors, compliance_status, and mitigation_plan

    Execution harness: Python

Evaluation tools

  • LangChain · LLM agent framework
  • LangGraph · Stateful workflow orchestration
  • Langfuse · Observability and tracing

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about LangChain-Powered Megaproject Procurement Risk Agent for Qiddiya Infrastructure