Back to challenges
Challenge

LlamaIndex MCED Diagnostic Utility RAG with GPT-5 Pro

Multi-cancer early detection (MCED) tests produce complex biomarker reports requiring precise patient-facing context and clinical guidelines. Implement an agentic RAG system using LlamaIndex and GPT-5 Pro to map MCED blood signals to evidence-based diagnostic pathways. Achieve zero hallucinated clinical recommendations on test cases.

Data ScienceHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Build an advanced RAG and agent workflow using LlamaIndex with GPT-5 Pro to map multi-cancer early detection signals to clinical follow-up protocols.

Delivery guide

How work is evaluated

Evaluation

Evaluates the factual fidelity and clinical precision of RAG generated diagnostic follow-up recommendations.

Datasets

Shared data for this challenge

Review public datasets and any private uploads tied to your build.

Loading datasets...
Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

zero_hallucinated_citations

Verifies that all returned citations match indexed source guidelines.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

guideline_compliance

Percentage of correct clinical diagnostic steps suggested. • target: 0.95 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Construct a LlamaIndex VectorStoreIndex over MCED clinical practice guidelines

  • Integrate GPT-5 Pro model for zero-shot RAG synthesis with strict citation matching

  • Design custom LlamaIndex RouterQueryEngine to route patient inquiries to diagnostic sub-indices

  • Implement automated clinical factual consistency guardrails

Resources and assets

Reference links and supporting material

Dataset notes

Collection of public MCED clinical consensus guidelines, NCCN pathways, and simulated patient test report cases.

How this agent runs

Evaluates the factual fidelity and clinical precision of RAG generated diagnostic follow-up recommendations.

Challenge input

JSON with patient_age, mced_signal_tissue_of_origin, biomarker_level

LlamaIndex

Required framework for document indexing and retrieval routing.

GPT-5 Pro

Provides state-of-the-art context processing and clinical analysis.

Evaluated output

JSON containing recommended_imaging, specialist_referral, and citations

Checks for
  • Verifies that all returned citations match indexed source guidelines.
  • Percentage of correct clinical diagnostic steps suggested. • target: 0.95 • range: 0-1
Proof of success
  • Benchmark: ClinicalRAG-Bench
  • Guideline Compliance target: 0.95
  • 1 public reference case
Runtime evidence
  • Python execution harness
  • Python sandbox
View technical recipe

Configured tools

Action Space
  • GPT-5 Pro · Required
  • Llama Index · Optional
Policy Serving
  • GPT-5 · Optional

Evaluation contract

  • zero_hallucinated_citations · Weight 1
  • guideline_compliance · Weight 1

Recipe state

This is a preview. The configuration can change before the evaluation recipe is locked.

Frequently Asked Questions about LlamaIndex MCED Diagnostic Utility RAG with GPT-5 Pro