Pydantic AI Missense Variant Triage with Claude 4 Opus
Interpreting novel missense mutations requires strictly me-structured validation schemas and deep context reasoning. Build a type-safe clinical variant triage agent using Pydantic AI and Claude 4 Opus to classify genomic variants into ACMG tier categories. Achieve 95 percent schema adherence and high accuracy against expert panel annotations.
What you are building
The core problem, expected build, and operating context for this challenge.
Develop a type-safe agentic pipeline leveraging Pydantic AI and Claude 4 Opus for structured genomic missense variant classification based on ACMG guidelines.
How work is evaluated
Measures schema validation accuracy and variant pathogenicity categorization against gold standard ClinVar annotations.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
pydantic_schema_validation
Checks if returned output passes Pydantic schema validation without errors.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
classification_accuracy
Accuracy of Pathogenic/Benign classification vs ClinVar labels. • target: 0.95 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Build type-safe Pydantic AI models for ACMG genomic variant criteria
Integrate Claude 4 Opus model outputs with enforced Pydantic validation
Incorporate variant pathogenicity predictions (CADD, REVEL, PolyPhen-2)
Evaluate classifier performance against ClinVar gold standard benchmark variants
Reference links and supporting material
Curated dataset of 500 missense variants annotated with ClinVar pathogenicity labels, gene context, and in silico predictor scores.
Challenge setup
Measures schema validation accuracy and variant pathogenicity categorization against gold standard ClinVar annotations.
Challenge input
JSON containing chr, pos, ref, alt, gene, and computational scores
Pydantic AI
Required framework for structured type-safe model outputs.
Claude 4 Opus
Provides high-reasoning capabilities for genetic criteria evaluation.
Evaluated output
JSON matching VariantTriageResult Pydantic schema
- Checks if returned output passes Pydantic schema validation without errors.
- Accuracy of Pathogenic/Benign classification vs ClinVar labels. • target: 0.95 • range: 0-1
- Benchmark: ClinVarVariantBench
- Classification Accuracy target: 0.95
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- Pydantic AI · Optional
- Aim · Optional
- Claude 4 Opus · Required
- Pydantic AI · Optional
Evaluation contract
- pydantic_schema_validation · Weight 1
- classification_accuracy · Weight 1
Metrics
- classification_accuracy · Target 0.95
Benchmarks
- ClinVarVariantBench
- biotech
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON containing chr, pos, ref, alt, gene, and computational scores
Output: JSON matching VariantTriageResult Pydantic schema
Execution harness: Python
Evaluation tools
- Pydantic AI · Type-safe agent framework
- Claude 4 Opus · LLM Provider
Recipe freeze marker
No recipe freeze time is recorded.