Duchenne Trial Risk Evaluator with Claude Agents SDK and Google Jules
Inspired by recent regulatory scrutiny surrounding Capricor Therapeutics' Duchenne muscular dystrophy treatment, this challenge focuses on building an automated trial outcome and FDA rejection risk evaluation agent. Clinical trials for rare pediatric neurodegenerative diseases face severe endpoints evaluation challenges, small cohort sizes, and baseline functional heterogeneity. Participants will construct an agentic system that ingests clinical trial protocol documents, biomarker efficacy metrics, and historical FDA advisory committee summaries to predict regulatory failure points.
What you are building
The core problem, expected build, and operating context for this challenge.
Build an agentic evaluation pipeline using the Claude Agents SDK and Google Jules to analyze clinical trial protocols and predict regulatory risk flags.
How work is evaluated
Evaluates the agent's ability to extract trial parameters and compute accurate regulatory risk scores.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
schema_validation
Output adheres strictly to expected JSON schema
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
f1_risk_flags
F1 score of identified regulatory risk flags against gold standard • target: 0.85 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Extract structured endpoint and cohort data from unstructured trial protocols using Claude Agents SDK
Deploy automated code analysis and risk modeling pipelines using Google Jules
Quantify FDA approval probability using historical regulatory advisory datasets
Implement audit logging and transparency checkpoints for clinical risk scoring
Reference links and supporting material
Synthetic dataset of Duchenne Muscular Dystrophy clinical trial protocols, primary/secondary endpoints, patient baseline metrics, and FDA regulatory decision histories.
Challenge setup
Evaluates the agent's ability to extract trial parameters and compute accurate regulatory risk scores.
Challenge input
JSON containing trial protocol text and efficacy biomarkers
Claude Agents SDK
Mandatory framework for agent orchestration
Google Jules
Assigned engineering workflow tool
Evaluated output
JSON with risk_score (float 0-1) and flagged_issues (list of strings)
- Output adheres strictly to expected JSON schema
- F1 score of identified regulatory risk flags against gold standard • target: 0.85 • range: 0-1
- Benchmark: ClinicalTrialQA
- F1 Risk Flags target: 0.85
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- Google Jules · Required
- Google · Optional
- LocalAI · Optional
- Google Jules · Required
Evaluation contract
- schema_validation · Weight 1
- f1_risk_flags · Weight 1
Metrics
- f1_risk_flags · Target 0.85
Benchmarks
- ClinicalTrialQA
- biotech
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON containing trial protocol text and efficacy biomarkers
Output: JSON with risk_score (float 0-1) and flagged_issues (list of strings)
Execution harness: Python
Evaluation tools
- Claude Agents SDK · Agent Framework
- Google Jules · AI Tooling
Recipe freeze marker
No recipe freeze time is recorded.