Skip to main content
Back to challenges
Challenge

Build a Hines Development Feasibility RAG Agent with LlamaIndex and Mindset AI

Commercial real estate developers struggle to rapidly evaluate capital deployment signals and local zoning constraints across expanding development pipelines. Build a LlamaIndex RAG agent integrated with Mindset AI workflows to ingest real estate filings and market metrics, achieving an F1-score of at least 0.88 on feasibility query answers.

Business OperationsHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Construct an enterprise RAG pipeline using LlamaIndex and Mindset AI to query complex commercial development filings and municipal zoning constraints.

Delivery guide

How work is evaluated

Evaluation

Evaluates the precision and response accuracy of the LlamaIndex real estate development feasibility agent.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

Schema Compliance Test

Verifies that the agent returns expected JSON keys.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

Extraction Precision

Percentage of correctly extracted underwriting features. • target: 0.9 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Implement advanced vector indexing over commercial real estate SEC filings and development proposals using LlamaIndex.

  • Integrate Mindset AI workflow triggers for dynamic data retrieval and alert orchestration.

  • Evaluate RAG retrieval accuracy using precision and recall metrics on complex underwriting queries.

Resources and assets

Reference links and supporting material

Dataset notes

Synthetic and publicly sourced real estate acquisition filings, zoning code summaries, and Hines capital allocation updates.

Challenge setup

Evaluates the precision and response accuracy of the LlamaIndex real estate development feasibility agent.

Challenge input

JSON containing document paths and underwriting questions

LlamaIndex

Required data framework for LLM indexing and RAG operations.

Mindset AI

Automates real estate deal pipeline workflows and triggers.

Evaluated output

JSON containing extracted parameters, zoning compliance flags, and confidence score

Evaluation checks
  • Verifies that the agent returns expected JSON keys.
  • Percentage of correctly extracted underwriting features. • target: 0.9 • range: 0-1
Configured evaluation evidence
  • Benchmark: RE_Underwriting_Eval
  • Extraction Precision target: 0.9
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Mindset AI · Required
  • Llama Index · Optional
  • Exa · Optional

Evaluation contract

  • Schema Compliance Test · Weight 1
  • Extraction Precision · Weight 1

Metrics

  • Extraction Precision · Target 0.9

Benchmarks

  • RE_Underwriting_Eval
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON containing document paths and underwriting questions

    Output: JSON containing extracted parameters, zoning compliance flags, and confidence score

    Execution harness: Python

Evaluation tools

  • LlamaIndex · Data Framework
  • Mindset AI · Workflow Automation

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Build a Hines Development Feasibility RAG Agent with LlamaIndex and Mindset AI