Skip to main content
Back to challenges
Challenge

Build CRE Default Risk Analyzer using LlamaIndex and Hyperbolic Automation

In response to rising distress in New York commercial office real estate—highlighted by American Strategic Investment Co.'s going-concern warning and debt default notices—asset managers require automated AI audit workflows. This challenge focuses on building a LlamaIndex agent capable of ingesting debt obligations, lease roll schedules, occupancy trends, and macroeconomic interest rate signals to project loan default risk across CRE portfolios. Using LlamaIndex's advanced RAG capabilities combined with Hyperbolic AI workflow execution engines, developers will create an end-to-end pipeline. The agent parses SEC 10-K filings, debt covenant terms, and tenant lease roll schedules into a structured Knowledge Graph, enabling real-time risk scoring, covenant breach warning alerts, and automated debt yield sensitivity modeling.

Business OperationsHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Construct an automated commercial real estate risk auditing agent using LlamaIndex and Hyperbolic to extract loan covenant data and score default probabilities.

Delivery guide

How work is evaluated

Evaluation

Evaluates the agent's ability to extract DSCR and debt terms, and accurately calculate default risk score under interest rate stress test scenarios.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

DSCR Calculation Test

Verifies that calculated DSCR matches expected financial math under interest rate shock

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

Extraction Precision

Accuracy of extracted covenant thresholds from unstructured SEC filing text • target: 0.95 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Extract structured loan covenants and debt maturity schedules from unstructured SEC filings using LlamaIndex

  • Build a debt yield sensitivity graph pipeline integrated with Hyperbolic AI workflows

  • Implement a real-time risk scoring engine that alerts portfolio managers of impending going-concern defaults

  • Deploy an end-to-end automated compliance and default prediction agent

Resources and assets

Reference links and supporting material

Dataset notes

Synthetic dataset of 50 CRE loan documents, lease schedules, and 10-K SEC filings representing distressed NYC office properties.

Challenge setup

Evaluates the agent's ability to extract DSCR and debt terms, and accurately calculate default risk score under interest rate stress test scenarios.

Challenge input

JSON containing asset_id, text_documents, and interest_rate_shock_bps

LlamaIndex

Required framework for document retrieval and multi-step reasoning.

Hyperbolic

Required tool for automated pipeline execution and workflow triggering.

Evaluated output

JSON with dscr, debt_yield, risk_level ('LOW', 'MEDIUM', 'HIGH', 'DEFAULT_CRITICAL'), and alert_reasons

Evaluation checks
  • Verifies that calculated DSCR matches expected financial math under interest rate shock
  • Accuracy of extracted covenant thresholds from unstructured SEC filing text • target: 0.95 • range: 0-1
Configured evaluation evidence
  • Benchmark: CRE Default Risk Bench 2025
  • Extraction Precision target: 0.95
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Hyperbolic · Required
  • Llama Index · Optional
  • Exa · Optional

Evaluation contract

  • DSCR Calculation Test · Weight 1
  • Extraction Precision · Weight 1

Metrics

  • Extraction Precision · Target 0.95

Benchmarks

  • CRE Default Risk Bench 2025
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON containing asset_id, text_documents, and interest_rate_shock_bps

    Output: JSON with dscr, debt_yield, risk_level ('LOW', 'MEDIUM', 'HIGH', 'DEFAULT_CRITICAL'), and alert_reasons

    Execution harness: Python

Evaluation tools

  • LlamaIndex · Agent Framework
  • Hyperbolic · AI Workflow Automation

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Build CRE Default Risk Analyzer using LlamaIndex and Hyperbolic Automation