Pediatric Speech NLP Predictor for Mood Disorders with LlamaIndex
NIH research indicates that a child's early spoken language patterns can predict the onset of depression and anxiety disorders years prior to clinical diagnosis. In this challenge, developers will build a specialized Retrieval-Augmented Generation (RAG) and index query pipeline using LlamaIndex. The system will index longitudinal speech transcripts and acoustic features, query prospective risk indicators, and output stratified pediatric psychological risk scores.
What you are building
The core problem, expected build, and operating context for this challenge.
Build a pediatric speech analysis RAG index using LlamaIndex to identify early predictive markers of internalizing disorders.
How work is evaluated
Evaluates index retrieval accuracy and risk level classification based on longitudinal transcripts.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
risk_level_valid
Ensure risk level matches allowed categories
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
risk_prediction_f1
F1 score of predicted risk levels compared to prospective longitudinal ground truth • target: 0.86 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Construct multi-modal document indices using LlamaIndex VectorStoreIndex
Extract semantic and structural sentiment markers from child transcript records
Implement metadata filtering based on age cohorts and longitudinal touchpoints
Query predictive clinical risk profiles while maintaining patient privacy controls
Reference links and supporting material
Synthetic pediatric interview transcripts annotated with developmental age, sentiment features, and long-term psychiatric follow-up outcomes.
Challenge setup
Evaluates index retrieval accuracy and risk level classification based on longitudinal transcripts.
Challenge input
JSON containing child_id, longitudinal_transcript_text, age_years
LlamaIndex
Mandatory LlamaIndex framework
Evaluated output
JSON containing predicted_risk_level (LOW/MODERATE/HIGH) and key_linguistic_markers (list)
- Ensure risk level matches allowed categories
- F1 score of predicted risk levels compared to prospective longitudinal ground truth • target: 0.86 • range:...
- Benchmark: PsychLing-Bench
- Risk Prediction F1 target: 0.86
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- Llama Index · Required
- Zed · Optional
- LastMile AI · Optional
Evaluation contract
- risk_level_valid · Weight 1
- risk_prediction_f1 · Weight 1
Metrics
- risk_prediction_f1 · Target 0.86
Benchmarks
- PsychLing-Bench
- biotech
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON containing child_id, longitudinal_transcript_text, age_years
Output: JSON containing predicted_risk_level (LOW/MODERATE/HIGH) and key_linguistic_markers (list)
Execution harness: Python
Evaluation tools
- LlamaIndex · Agent Framework
Recipe freeze marker
No recipe freeze time is recorded.