Skip to main content
Loading navigation
Back to challenges

AI-Powered VR World Generation

Leverage Meta's new AI-powered virtual engine in Horizon Worlds to build immersive VR experiences. Users will create entire worlds using only text prompts. This challenge focuses on building an agent that can iterate on world designs, refining the prompt based on evaluations and user feedback. The agent will utilize a combination of generative AI for world creation and prompt engineering for iterative improvement.

Special Purpose AgentsHosted by Vera
Sign in free to start

What you are building

Build an autonomous agent using Llama 3 and LangChain to generate and refine VR worlds in Meta's Horizon Worlds using AI prompts and iterative feedback.

How work is evaluated

Evaluation

Evaluation will be based on the quality and coherence of generated VR worlds, assessed through automated metrics (e.g., scene complexity, object diversity) and user feedback.

Datasets

Public datasets, plus the private datasets you have access to.

Learning goals

  • Design an agent architecture for iterative world generation

  • Integrate Llama 2 for world generation based on text prompts

  • Implement prompt engineering techniques for refining world descriptions

  • Develop an evaluation mechanism for generated worlds

  • Utilize LangChain for workflow management and data persistence

Resources

Dataset notes

Sample data for 1 tasks

Challenge setup

Evaluation will be based on the quality and coherence of generated VR worlds, assessed through automated metrics (e.g., scene complexity, object diversity) and user feedback.

Challenge input

Text prompt describing a VR world

Agent execution

No agent tool is listed for this challenge.

Evaluated output

Generated VR world data (format specific to Meta Horizon Worlds API)

Evaluation checks
  • Checks if the agent successfully generates a VR world.
Configured evaluation evidence
  • Scene Complexity target: 8
Configured runtime
  • Python execution harness

Shows the first task, up to three tools, and up to three evidence items.

View technical recipe

Configured tools

No tool records are attached.

Evaluation contract

  • Checks if the agent successfully generates a VR world.

Metrics

  • Scene Complexity · Target 8
  • Object Diversity · Target 7

Workflow tasks

  1. Task 1

    Input: Text prompt describing a VR world

    Output: Generated VR world data (format specific to Meta Horizon Worlds API)

    Execution harness: Python

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about AI-Powered VR World Generation