Trial Eligibility Engine for Anti-Inflammatory Drugs using Pydantic AI
Matching heart disease patients with anti-inflammatory clinical trials requires strict eligibility verification across EHR datasets. Developers will construct a Pydantic AI agent that parses OMOP CDM records, validates IL-6 pathway biomarker thresholds, and outputs validated trial eligibility decisions with full type enforcement.
What you are building
The core problem, expected build, and operating context for this challenge.
Build a Pydantic AI agent to parse OMOP structured health records and evaluate clinical trial criteria for heart disease drugs.
How work is evaluated
Evaluates Pydantic output validation compliance, inclusion decision precision, and criterion checks.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
pydantic_schema_strict_test
Validates that the output strictly adheres to Pydantic schema without type errors.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
eligibility_accuracy
Accuracy of trial eligibility determination against expert ground truth. • target: 0.95 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Define strict Pydantic schemas for patient health profiles and trial criteria.
Implement a Pydantic AI agent with dependency injection for database lookups.
Parse OMOP Common Data Model patient tables to evaluate IL-6 biomarker levels.
Output type-safe trial inclusion/exclusion decision objects with justification logs.
Reference links and supporting material
Structured OMOP CDM database tables containing patient demographics, cardiovascular diagnoses, and inflammatory bio-markers.
Challenge setup
Evaluates Pydantic output validation compliance, inclusion decision precision, and criterion checks.
Challenge input
JSON containing OMOP patient clinical record.
Pydantic AI
Guarantees runtime validation and structural type safety for LLM responses.
Evaluated output
JSON strictly matching Pydantic schema for TrialEligibilityResult.
- Validates that the output strictly adheres to Pydantic schema without type errors.
- Accuracy of trial eligibility determination against expert ground truth. • target: 0.95 • range: 0-1
- Benchmark: Clinical Trial Matching Benchmark
- Eligibility Accuracy target: 0.95
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- Pydantic AI · Required
- Upstage · Optional
- Haize Labs · Optional
- Pydantic AI · Required
Evaluation contract
- pydantic_schema_strict_test · Weight 1
- eligibility_accuracy · Weight 1
Metrics
- eligibility_accuracy · Target 0.95
Benchmarks
- Clinical Trial Matching Benchmark
- biotech
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON containing OMOP patient clinical record.
Output: JSON strictly matching Pydantic schema for TrialEligibilityResult.
Execution harness: Python
Evaluation tools
- Pydantic AI · Type-Safe Agent Framework
Recipe freeze marker
No recipe freeze time is recorded.