Skip to main content
Back to challenges
Challenge

Build a Real-Time Clearing Risk Engine using LlamaIndex RAG Framework

Real-time clearing facilities face high-throughput risk exposure and liquidity shocks during volatile market events. Build a LlamaIndex RAG-powered agent framework to query stream telemetry, compute capital adequacy metrics, and recommend real-time margin dynamic adjustments. Deliver an automated risk management agent executing evaluations in under 500ms with high operational accuracy.

CybersecurityHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Develop an agentic RAG solution with LlamaIndex to query clearing balance history and dynamically evaluate liquidity risk limits in post-trade settlement pipelines.

Delivery guide

How work is evaluated

Evaluation

Evaluates the RAG agent's margin call recommendations against standard clearinghouse capital adequacy formulas.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

margin_call_boolean_check

Verifies margin call requirement trigger threshold logic

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

calculation_accuracy

Accuracy of additional collateral calculation within 1% tolerance • target: 0.98 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Build a LlamaIndex vector store index over historical clearing firm ledger records

  • Construct a LlamaIndex AgentRunner for dynamic margin risk query processing

  • Calculate real-time capital margin calls based on volatility spikes

  • Implement low-latency streaming retrieval for real-time clearing alerts

Resources and assets

Reference links and supporting material

Dataset notes

Dataset of 300 clearing member exposure snapshots with associated historical volatility indices.

Challenge setup

Evaluates the RAG agent's margin call recommendations against standard clearinghouse capital adequacy formulas.

Challenge input

JSON containing member_id, current_exposure_usd, collateral_on_file_usd, volatility_index

LlamaIndex

Provides structured document indexing and tool-enabled agent runners.

Evaluated output

JSON with margin_call_required, required_additional_collateral_usd, risk_category

Evaluation checks
  • Verifies margin call requirement trigger threshold logic
  • Accuracy of additional collateral calculation within 1% tolerance • target: 0.98 • range: 0-1
Configured evaluation evidence
  • Benchmark: Clearing Risk Analytics Benchmark
  • Calculation Accuracy target: 0.98
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Llama Index · Required
  • LlamaIndex · Optional
  • Llama 3.2 · Optional

Evaluation contract

  • margin_call_boolean_check · Weight 1
  • calculation_accuracy · Weight 1

Metrics

  • calculation_accuracy · Target 0.98

Benchmarks

  • Clearing Risk Analytics Benchmark
  • security

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON containing member_id, current_exposure_usd, collateral_on_file_usd, volatility_index

    Output: JSON with margin_call_required, required_additional_collateral_usd, risk_category

    Execution harness: Python

Evaluation tools

  • LlamaIndex · RAG Framework

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Build a Real-Time Clearing Risk Engine using LlamaIndex RAG Framework