Skip to main content
Loading navigation
Back to challenges

Build a Hines Development Feasibility RAG Agent with LlamaIndex and Mindset AI

Commercial real estate developers struggle to rapidly evaluate capital deployment signals and local zoning constraints across expanding development pipelines. Build a LlamaIndex RAG agent integrated with Mindset AI workflows to ingest real estate filings and market metrics, achieving an F1-score of at least 0.88 on feasibility query answers.

Business OperationsHosted by Vera
Sign in free to start

What you are building

Construct an enterprise RAG pipeline using LlamaIndex and Mindset AI to query complex commercial development filings and municipal zoning constraints.

How work is evaluated

Evaluation

Evaluates the precision and response accuracy of the LlamaIndex real estate development feasibility agent.

Datasets

Public datasets, plus the private datasets you have access to.

How submissions are scored

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

Schema Compliance Test

Verifies that the agent returns expected JSON keys.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

Extraction Precision

Percentage of correctly extracted underwriting features. • target: 0.9 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

  • Implement advanced vector indexing over commercial real estate SEC filings and development proposals using LlamaIndex.

  • Integrate Mindset AI workflow triggers for dynamic data retrieval and alert orchestration.

  • Evaluate RAG retrieval accuracy using precision and recall metrics on complex underwriting queries.

Resources

Dataset notes

Synthetic and publicly sourced real estate acquisition filings, zoning code summaries, and Hines capital allocation updates.

Challenge setup

Evaluates the precision and response accuracy of the LlamaIndex real estate development feasibility agent.

Challenge input

JSON containing document paths and underwriting questions

LlamaIndex

Required data framework for LLM indexing and RAG operations.

Mindset AI

Automates real estate deal pipeline workflows and triggers.

Evaluated output

JSON containing extracted parameters, zoning compliance flags, and confidence score

Evaluation checks
  • Verifies that the agent returns expected JSON keys.
  • Percentage of correctly extracted underwriting features. • target: 0.9 • range: 0-1
Configured evaluation evidence
  • Benchmark: RE_Underwriting_Eval
  • Extraction Precision target: 0.9
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Shows the first task, up to three tools, and up to three evidence items.

View technical recipe

Configured tools

Action Space
  • Mindset AI · Required
  • Llama Index · Optional
  • Exa · Optional

Evaluation contract

  • Schema Compliance Test · Weight 1
  • Extraction Precision · Weight 1

Metrics

  • Extraction Precision · Target 0.9

Benchmarks

  • RE_Underwriting_Eval
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON containing document paths and underwriting questions

    Output: JSON containing extracted parameters, zoning compliance flags, and confidence score

    Execution harness: Python

Evaluation tools

  • LlamaIndex · Data Framework
  • Mindset AI · Workflow Automation

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Build a Hines Development Feasibility RAG Agent with LlamaIndex and Mindset AI