Skip to main content
Back to challenges
Challenge

Precision Oncology EHR Matching and OMOP Mapping with OpenAI Agents SDK

NIH news emphasizes how patient-derived cancer models drive precision oncology therapies. However, translating patient records into standardized data models like OMOP CDM is a major operational bottleneck. In this challenge, developers will utilize the OpenAI Agents SDK to build an entity extraction and terminology mapping agent that converts unstructured oncology EHR notes into standardized OMOP CDM concept identifiers (ICD-O, RxNorm, SNOMED CT).

Data ScienceHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Build an AI agent using OpenAI Agents SDK to extract oncology EHR entities and map them to standard OMOP CDM concepts.

Delivery guide

How work is evaluated

Evaluation

Evaluates concept mapping precision and OMOP CDM schema conformance.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

check_valid_concept_ids

Ensure concept IDs are integer values

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

concept_mapping_f1

F1 score of extracted and mapped OMOP concept IDs against ground truth clinical annotations • target: 0.88 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Parse clinical oncology narratives into structured medical entities

  • Use OpenAI Agents SDK function calling to query OHDSI Athena terminology APIs

  • Map cancer staging, histology, and treatment regimens to OMOP CDM concept IDs

  • Validate schema compliance against OMOP CDM v5.4 tables

Resources and assets

Reference links and supporting material

Dataset notes

Synthetic clinical oncology discharge summaries annotated with ground-truth OMOP CDM concept IDs.

Challenge setup

Evaluates concept mapping precision and OMOP CDM schema conformance.

Challenge input

JSON containing clinical_note_text

OpenAI Agents SDK

Mandatory OpenAI Agents SDK

Evaluated output

JSON containing condition_concept_id (int), drug_concept_id (int), staging_code (string)

Evaluation checks
  • Ensure concept IDs are integer values
  • F1 score of extracted and mapped OMOP concept IDs against ground truth clinical annotations • target: 0.88...
Configured evaluation evidence
  • Benchmark: OMOP-Map-Bench
  • Concept Mapping F1 target: 0.88
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • OpenAI · Required
  • Zed · Optional
  • Replicate · Optional

Evaluation contract

  • check_valid_concept_ids · Weight 1
  • concept_mapping_f1 · Weight 1

Metrics

  • concept_mapping_f1 · Target 0.88

Benchmarks

  • OMOP-Map-Bench
  • biotech

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON containing clinical_note_text

    Output: JSON containing condition_concept_id (int), drug_concept_id (int), staging_code (string)

    Execution harness: Python

Evaluation tools

  • OpenAI Agents SDK · Agent Framework

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Precision Oncology EHR Matching and OMOP Mapping with OpenAI Agents SDK