Skip to main content
Back to challenges
Challenge

Build Autonomous Web Scraping Orchestrators with LlamaIndex and Qwen 3

Develop an autonomous agent network designed to manage web data acquisition pipelines following the recent funding news for infrastructure providers. You will use LlamaIndex to structure agentic workflows that interact with external data environments while leveraging Qwen 3 for high-performance reasoning. The system must navigate dynamic site structures, manage stateful execution in isolated environments, and route traffic efficiently to maximize data throughput.

Workflow AutomationHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Orchestrate an agentic pipeline using LlamaIndex and Qwen 3 to automate complex web data retrieval workflows.

Delivery guide

How work is evaluated

Evaluation

Validate agent stability and success rate of data extraction tasks.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

ExecutionSuccess

Ensure task completes without crash

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

Accuracy

Precision of extracted data • target: 95 • range: 0-100

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Master LlamaIndex agentic abstractions to manage task state and execution flows

  • Deploy Daytona environments as isolated sandboxes for autonomous agent operations

  • Design routing logic using OpenRouter to switch between model endpoints based on task complexity

  • Integrate Replit Agent to autonomously patch and update scraping logic in real-time

  • Orchestrate Synthflow interfaces to trigger voice-based notifications on pipeline failures

Resources and assets

Reference links and supporting material

Dataset notes

Sample data for 1 tasks

Challenge setup

Validate agent stability and success rate of data extraction tasks.

Challenge input

URL list

Llama Index

Data framework for LLM

Exa

Neural search API

Qwen 3

Policy Serving in the agent workflow.

Evaluated output

JSON summary

Evaluation checks
  • Ensure task completes without crash
  • Precision of extracted data • target: 95 • range: 0-100
Configured evaluation evidence
  • Accuracy target: 95
  • 1 public reference case
Configured runtime
  • Python execution harness

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Llama Index · Required
  • Exa · Optional
Policy Serving
  • Qwen 3 · Optional

Evaluation contract

  • ExecutionSuccess · Weight 1
  • Accuracy · Weight 1

Metrics

  • Accuracy · Target 95

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: URL list

    Output: JSON summary

    Execution harness: Python

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Build Autonomous Web Scraping Orchestrators with LlamaIndex and Qwen 3