Clinical Continuity Agent: Syphilis Treatment Protocol & Drug Shortage Orchestrator
Public health infrastructure is currently struggling with a resurgence of syphilis compounded by severe shortages of Bicillin L-A and restrictive federal research constraints. This challenge tasks you with building an intelligent clinical decision support agent using LlamaIndex that dynamically adjusts treatment recommendations based on real-time pharmacy inventory levels and patient-specific contraindications. You will implement advanced RAG techniques to parse complex CDC clinical guidelines and merge them with unstructured pharmacy availability reports. Your agent must navigate the 'federal constraints' mentioned in STAT news by prioritizing protocols that comply with current NIH and FDA regulatory frameworks while identifying alternative pharmacotherapy routes. Using Aider as your AI pair-programmer, you will develop a modular system that can ingest FHIR-formatted patient records and output structured treatment plans with full provenance links to the source literature. The solution must demonstrate robust handling of 'drug-out' scenarios where primary treatments are unavailable, suggesting secondary or tertiary alternatives based on efficacy and patient allergy profiles.
What you are building
The core problem, expected build, and operating context for this challenge.
Public health infrastructure is currently struggling with a resurgence of syphilis compounded by severe shortages of Bicillin L-A and restrictive federal research constraints. This challenge tasks you with building an intelligent clinical decision support agent using LlamaIndex that dynamically adjusts treatment recommendations based on real-time pharmacy inventory levels and patient-specific contraindications. You will implement advanced RAG techniques to parse complex CDC clinical guidelines and merge them with unstructured pharmacy availability reports. Your agent must navigate the 'federal constraints' mentioned in STAT news by prioritizing protocols that comply with current NIH and FDA regulatory frameworks while identifying alternative pharmacotherapy routes. Using Aider as your AI pair-programmer, you will develop a modular system that can ingest FHIR-formatted patient records and output structured treatment plans with full provenance links to the source literature. The solution must demonstrate robust handling of 'drug-out' scenarios where primary treatments are unavailable, suggesting secondary or tertiary alternatives based on efficacy and patient allergy profiles.
Shared data for this challenge
Review public datasets and any private uploads tied to your build.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
Safety Check: Pregnancy
Ensure Doxycycline is NEVER suggested for pregnant patients even during shortages.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Citation Accuracy
Percentage of recommendations containing valid CDC/NIH links. • target: 1 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Master LlamaIndex Document Store and VectorStoreIndex configuration for medical PDF ingestion
Implement advanced RAG patterns including 'Small-to-Big' retrieval and 'Sentence Window Retrieval' for dense clinical guidelines
Design a custom LlamaIndex ToolSpec that interacts with simulated pharmacy inventory APIs to check stock levels
Build a clinical verification layer using LlamaIndex's 'Evaluation' module to detect hallucinations in treatment dosages
Orchestrate a multi-agent workflow where one agent analyzes patient history and another monitors drug availability
Deploy a structured output parser that converts LLM reasoning into validated JSON-LD for clinical trial matching
Integrate Aider into the CLI workflow to refactor Biopython-based analysis scripts for genomic variant checking
Optimize context window usage by implementing a 'Summary Index' for long-running patient longitudinal records
How this agent runs
Evaluation focuses on clinical accuracy, adherence to the shortage logic, and data structure validity.
Challenge input
JSON containing patient profile and inventory status
LlamaIndex
Best-in-class for RAG and data-connected agents.
Aider
Speeds up the creation of complex clinical parsing logic.
FastAPI
Exposes the agent as a clinical microservice.
Evaluated output
Structured Treatment Plan JSON
- Ensure Doxycycline is NEVER suggested for pregnant patients even during shortages.
- Percentage of recommendations containing valid CDC/NIH links. • target: 1 • range: 0-1
- Benchmark: ClinicalQA / MedQA
- Citation Accuracy target: 1
- 1 public reference case
- Python execution harness
- Python sandbox
View technical recipe
Configured tools
- Aider · Required
- Lingo · Optional
- Llama Index · Optional
- Aider · Required
Evaluation contract
- Safety Check: Pregnancy · Weight 1
- Citation Accuracy · Weight 1
Recipe state
This is a preview. The configuration can change before the evaluation recipe is locked.
Run this agent on your dataset and AI stack
Bring your dataset, model providers, and success criteria. We will scope the right managed run for your team.
Scope a managed run[ok] Wrote CHALLENGE.md
[ok] Wrote .versalist.json
[ok] Wrote eval/examples.json
Requires VERSALIST_API_KEY. Works with any MCP-aware editor.
DocsFind another challenge
Jump to a random challenge when you want a fresh benchmark or a different problem space.