Skip to main content
Loading navigation
Back to challenges

Advanced Compute Strategy Modeling with LlamaIndex

Create a decision support agent using LlamaIndex to model compute infrastructure strategies for AI developers. Inspired by the reported chip supply ambitions of major labs, this agent will evaluate compute procurement options. By utilizing Fireworks AI for model inference and Edge Impulse for performance simulation, the agent will analyze hypothetical supply chain risks and compute scalability, providing data-driven recommendations without relying on internal document databases.

Business OperationsHosted by Vera
Sign in free to start

What you are building

Build an agentic simulation platform using LlamaIndex and GPT-5.4 Pro to analyze AI infrastructure strategies.

How work is evaluated

Evaluation

Evaluate the agent's ability to propose optimal infrastructure configurations based on constraints.

Datasets

Public datasets, plus the private datasets you have access to.

How submissions are scored

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

Feasibility Check

Ensure proposed strategy fits within input budget

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

Resource Efficiency

Score of recommended strategy viability • target: 85 • range: 0-100

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

  • Master LlamaIndex agentic abstractions for complex reasoning tasks

  • Configure Fireworks AI as an inference backend for model performance

  • Integrate Edge Impulse APIs for simulated inference latency calculation

  • Design multi-step planning agents using GPT-5.4 Pro and Claude Sonnet 4.6.6

  • Orchestrate comparative strategy modeling between compute hardware providers

  • Build feedback loops between model inference performance and business requirements

Resources

Dataset notes

Sample data for 1 tasks

Challenge setup

Evaluate the agent's ability to propose optimal infrastructure configurations based on constraints.

Challenge input

Constraint map

Llama Index

Data framework for LLM

Yupp AI

LLM evaluation platform

Fireworks AI

Fast inference and fine-tuning platform.

Evaluated output

Infrastructure plan JSON

Evaluation checks
  • Ensure proposed strategy fits within input budget
  • Score of recommended strategy viability • target: 85 • range: 0-100
Configured evaluation evidence
  • Resource Efficiency target: 85
  • 1 public reference case
Configured runtime
  • Python execution harness

Shows the first task, up to three tools, and up to three evidence items.

View technical recipe

Configured tools

Action Space
  • Llama Index · Required
  • Yupp AI · Optional
Policy Serving
  • Fireworks AI · Optional
Training Infra
  • Fireworks AI · Optional

Evaluation contract

  • Feasibility Check · Weight 1
  • Resource Efficiency · Weight 1

Metrics

  • Resource Efficiency · Target 85

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: Constraint map

    Output: Infrastructure plan JSON

    Execution harness: Python

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Advanced Compute Strategy Modeling with LlamaIndex