EHR¶
openbtk.data.ehr loads FHIR R4, OMOP and HL7 v2 into one PatientRecord schema.
It must be imported explicitly (it registers loader.ehr.fhir, loader.ehr.omop
and loader.ehr.hl7v2). Reading needs the ehr extra: pip install "openbtk[ehr]".
See the EHR guide for a worked example.
Schemas¶
openbtk.data.ehr.schemas.PatientRecord ¶
Bases: BaseModel
One patient's structured clinical record, source-agnostic.
Example
record = PatientRecord( ... patient_id="pt-1", ... demographics=Demographics(gender="female"), ... source_system="fhir-r4", ... ) record.deid_status
openbtk.data.ehr.schemas.Demographics ¶
Bases: BaseModel
A patient's demographic attributes, as loaded -- before or after
de-identification (PatientRecord.deid_status governs that, not this
type).
race/ethnicity are free text here, not an enum: FHIR's US Core
extensions and OMOP's race_concept_id/ethnicity_concept_id both
ultimately resolve to a categorical value from their own vocabularies
(OMB categories, in practice), but Openbtk does not bundle either
vocabulary (docs/09_CODING_STANDARDS.md section 7 -- openbtk.terminology
is the right home for that resolution, and it is not built yet, M7).
Loaders populate the human-readable display text they already have
rather than inventing a premature enum.
Example
Demographics(gender="female").gender 'female'
openbtk.data.ehr.schemas.Encounter ¶
Bases: BaseModel
A single care encounter (visit, admission, ...).
encounter_id is the join key CodedEvent.encounter_ref and
Measurement.encounter_ref point back to -- loaders are responsible
for using the same, bare identifier on both sides (docs/05_DATA_MODALITY_SPEC.md
section 2.1's own worked timeline example shows every event carrying its
encounter inline for exactly this reason).
Example
Encounter(encounter_id="enc-1", encounter_type="inpatient").encounter_id 'enc-1'
openbtk.data.ehr.schemas.CodedEvent ¶
Bases: BaseModel
A single coded clinical event: a condition, medication or procedure.
Example
from openbtk.core.schemas import CodeSystem event = CodedEvent(code="385093006", system=CodeSystem.SNOMED) event.system.value 'SNOMED'
openbtk.data.ehr.schemas.Measurement ¶
Bases: BaseModel
A single observation or lab result.
value is float | str | None because not every observation is
numeric -- a qualitative result ("Positive", "Detected") is real,
common lab data, not an edge case to reject.
display is a deliberate, disclosed addition beyond
docs/05_DATA_MODALITY_SPEC.md section 2.1's original field list (which
otherwise matches exactly) -- PatientTimelineSerializer (task 6.5)
needs a human-readable name ("WBC") to render a lab result at all; a
bare LOINC code is not something a clinician or an LLM reading a
rendered timeline can interpret. CodedEvent already had this field;
Measurement lacking it was an oversight in the original spec, fixed
here rather than shipping a serializer that can only ever show raw
codes.
Example
m = Measurement(code="6690-2", display="WBC", value=14.2, unit="10*3/uL") m.system.value 'LOINC'
Loaders¶
openbtk.data.ehr.fhir.FHIRLoader ¶
Bases: BaseLoader[str, PatientRecord]
Load one PatientRecord per FHIR R4 Bundle JSON file in a
directory. See this module's own docstring for the loader's exact,
disclosed scope.
Example
import contextlib, io, json, tempfile from pathlib import Path bundle = { ... "resourceType": "Bundle", ... "type": "collection", ... "entry": [{"resource": { ... "resourceType": "Patient", "id": "pt-1", "gender": "female", ... }}], ... } with tempfile.TemporaryDirectory() as d: ... _ = (Path(d) / "pt-1.json").write_text(json.dumps(bundle)) ... with contextlib.redirect_stdout(io.StringIO()): ... records = list(FHIRLoader().load(d)) records[0].patient_id 'pt-1'
load ¶
load(source: str) -> Iterator[PatientRecord]
LoaderError: If source is not a directory, a file is not
valid JSON, is not a FHIR Bundle, or a bundle contains no
Patient resource.
Source code in src/openbtk/data/ehr/fhir.py
296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 | |
openbtk.data.ehr.omop.OMOPLoader ¶
OMOPLoader(*, batch_size: int = 10000, condition_system: CodeSystem = CodeSystem.SNOMED, drug_system: CodeSystem = CodeSystem.RXNORM, procedure_system: CodeSystem = CodeSystem.CPT, measurement_system: CodeSystem = CodeSystem.LOINC)
Bases: BaseLoader[str, PatientRecord]
Load PatientRecords from a directory of OMOP CDM v5.4 Parquet
tables (person.parquet required; visit_occurrence,
condition_occurrence, drug_exposure, procedure_occurrence
and measurement are each read if present, skipped if absent). See
this module's own docstring for real, disclosed scope and memory
trade-offs.
Example
import contextlib, io, tempfile from pathlib import Path import pyarrow as pa import pyarrow.parquet as pq with tempfile.TemporaryDirectory() as d: ... table = pa.table({ ... "person_id": [1], ... "gender_concept_id": [8532], ... "year_of_birth": [1980], ... "month_of_birth": [5], ... "day_of_birth": [1], ... }) ... _ = pq.write_table(table, Path(d) / "person.parquet") ... with contextlib.redirect_stdout(io.StringIO()): ... records = list(OMOPLoader().load(d)) records[0].patient_id '1' records[0].demographics.gender 'female'
Source code in src/openbtk/data/ehr/omop.py
162 163 164 165 166 167 168 169 170 171 172 173 174 175 | |
load ¶
load(source: str) -> Iterator[PatientRecord]
LoaderError: If source is not a directory, lacks
person.parquet, or a Parquet table fails to parse.
Source code in src/openbtk/data/ehr/omop.py
177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 | |
openbtk.data.ehr.hl7v2.HL7v2Loader ¶
HL7v2Loader(*, default_utc_offset_hours: float = 0.0, encoding: str = 'utf-8')
Bases: BaseLoader[str, PatientRecord]
Load PatientRecords from HL7 v2 messages. See the module docstring for the
exact scope.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
default_utc_offset_hours
|
float
|
The offset assumed for a timestamp that carries none. |
0.0
|
encoding
|
str
|
The text encoding of the files. |
'utf-8'
|
Example
import contextlib, io, tempfile from pathlib import Path message = "\r".join([ ... "MSH|^~\&|APP|FAC|RCV|FAC|20240314101500||ADT^A01|M1|P|2.5", ... "PID|1||PT0001^^^HOSP^MR||DOE^JANE||19800601|F", ... ]) with tempfile.TemporaryDirectory() as d: ... _ = (Path(d) / "a.hl7").write_text(message) ... with contextlib.redirect_stdout(io.StringIO()): ... records = list(HL7v2Loader().load(d)) (records[0].patient_id, records[0].demographics.gender) ('PT0001', 'female')
Source code in src/openbtk/data/ehr/hl7v2.py
237 238 239 240 241 | |
load ¶
load(source: str) -> Iterator[PatientRecord]
LoaderError: If source is neither a file nor a directory, a file cannot
be read, or a message cannot be parsed. The message names the file and
the message's position, never its content.
Source code in src/openbtk/data/ehr/hl7v2.py
243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 | |
Timelines and text¶
openbtk.data.ehr.temporal.TemporalNormalizer ¶
TemporalNormalizer(*, anchor_missing_timestamps: bool = True)
Bases: BasePreprocessor[PatientRecord]
Anchor missing event timestamps to their encounter and sort every event list chronologically.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
anchor_missing_timestamps
|
bool
|
When True (default), an event with no
timestamp of its own but a resolvable |
True
|
Example
from datetime import datetime, UTC from openbtk.core.schemas import CodeSystem from openbtk.data.ehr.schemas import ( ... CodedEvent, Demographics, Encounter, PatientRecord, ... ) record = PatientRecord( ... patient_id="pt-1", ... demographics=Demographics(), ... encounters=[Encounter( ... encounter_id="enc-1", ... start=datetime(2024, 3, 14, 8, tzinfo=UTC), ... )], ... conditions=[CodedEvent( ... code="385093006", system=CodeSystem.SNOMED, encounter_ref="enc-1", ... )], ... source_system="fhir-r4", ... ) normalized = TemporalNormalizer().process(record) normalized.conditions[0].timestamp datetime.datetime(2024, 3, 14, 8, 0, tzinfo=datetime.timezone.utc)
Source code in src/openbtk/data/ehr/temporal.py
83 84 | |
openbtk.pipelines.timeline.PatientTimelineSerializer ¶
PatientTimelineSerializer(*, encounter_label: str = 'Encounter', condition_label: str = 'Condition', medication_label: str = 'Medication', procedure_label: str = 'Procedure', observation_label: str = 'Lab', encounter_template: Callable[[Encounter], str] = _default_encounter_template, condition_template: Callable[[CodedEvent], str] = _default_coded_event_template, medication_template: Callable[[CodedEvent], str] = _default_coded_event_template, procedure_template: Callable[[CodedEvent], str] = _default_coded_event_template, observation_template: Callable[[Measurement], str] = _default_observation_template, source: str = 'ehr-timeline')
Render a PatientRecord as a chronological plain-text timeline.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
encounter_label
|
str
|
Column label for encounter rows. |
'Encounter'
|
condition_label
|
str
|
Column label for condition rows. |
'Condition'
|
medication_label
|
str
|
Column label for medication rows. |
'Medication'
|
procedure_label
|
str
|
Column label for procedure rows. |
'Procedure'
|
observation_label
|
str
|
Column label for observation/lab rows. |
'Lab'
|
encounter_template
|
Callable[[Encounter], str]
|
Overrides the default one-line encounter
description. Same idea for the other four |
_default_encounter_template
|
source
|
str
|
|
'ehr-timeline'
|
Example
from datetime import datetime, UTC from openbtk.core.schemas import CodeSystem from openbtk.data.ehr.schemas import CodedEvent, Demographics, PatientRecord record = PatientRecord( ... patient_id="pt-1", ... demographics=Demographics(), ... conditions=[CodedEvent( ... code="385093006", system=CodeSystem.SNOMED, ... display="Community-acquired pneumonia", ... timestamp=datetime(2024, 3, 14, tzinfo=UTC), ... )], ... source_system="fhir-r4", ... ) text_record = PatientTimelineSerializer().serialize(record) "Community-acquired pneumonia (SNOMED 385093006)" in text_record.text True text_record.patient_ref 'pt-1'
Source code in src/openbtk/pipelines/timeline.py
148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 | |
serialize ¶
serialize(patient: PatientRecord) -> ClinicalTextRecord
Render every encounter/condition/medication/procedure/ observation as one chronological, compact plain-text timeline.
Returns:
| Type | Description |
|---|---|
ClinicalTextRecord
|
A |
ClinicalTextRecord
|
|
ClinicalTextRecord
|
for the same de-identification/chunking/guardrail pipeline |
ClinicalTextRecord
|
any other clinical text record goes through. |
Source code in src/openbtk/pipelines/timeline.py
179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 | |
Cohorts¶
openbtk.data.ehr.cohort.CohortBuilder ¶
CohortBuilder(records: Iterable[PatientRecord])
Compose include/exclude predicates over a stream of
PatientRecords -- never materialises the source (ADR-0004): a
cohort over a million patients has bounded memory, the same guarantee
every other streaming component in this project makes.
Example
from openbtk.core.schemas import CodeSystem from openbtk.data.ehr.schemas import CodedEvent, Demographics patients = [ ... PatientRecord( ... patient_id="pt-1", demographics=Demographics(), ... conditions=[CodedEvent(code="385093006", system=CodeSystem.SNOMED)], ... source_system="fhir-r4", ... ), ... PatientRecord( ... patient_id="pt-2", ... demographics=Demographics(), ... source_system="fhir-r4", ... ), ... ] cohort = CohortBuilder(patients).include(has_condition("385093006")) [p.patient_id for p in cohort] == ["pt-1"] True
Source code in src/openbtk/data/ehr/cohort.py
141 142 143 144 | |
exclude ¶
exclude(predicate: Predicate) -> CohortBuilder
Add a predicate that, if satisfied, drops a candidate (logical
OR across every exclude() call). Returns self for
chaining.
Source code in src/openbtk/data/ehr/cohort.py
152 153 154 155 156 157 | |
include ¶
include(predicate: Predicate) -> CohortBuilder
Add a predicate every candidate must satisfy (logical AND
across every include() call). Returns self for chaining.
Source code in src/openbtk/data/ehr/cohort.py
146 147 148 149 150 | |
openbtk.data.ehr.cohort.has_condition ¶
has_condition(code: str, *, system: CodeSystem | None = None) -> Predicate
A predicate matching patients with a condition of code (any
coding system, unless system narrows it).
Source code in src/openbtk/data/ehr/cohort.py
60 61 62 63 64 65 66 67 | |
openbtk.data.ehr.cohort.has_medication ¶
has_medication(code: str, *, system: CodeSystem | None = None) -> Predicate
See :func:has_condition; matches patient.medications.
Source code in src/openbtk/data/ehr/cohort.py
70 71 72 73 74 75 76 | |
openbtk.data.ehr.cohort.has_procedure ¶
has_procedure(code: str, *, system: CodeSystem | None = None) -> Predicate
See :func:has_condition; matches patient.procedures.
Source code in src/openbtk/data/ehr/cohort.py
79 80 81 82 83 84 85 | |
openbtk.data.ehr.cohort.age_between ¶
age_between(min_age: int, max_age: int, *, as_of: date | None = None) -> Predicate
A predicate matching patients whose age -- computed from
Demographics.birth_date as of as_of -- falls in
[min_age, max_age] inclusive.
as_of defaults to date.today() evaluated the moment the
returned predicate is called, not when this factory is, so a
predicate built once and reused across a long-running process stays
correct. A patient with no known birth_date never matches (age is
genuinely unknown, not assumed to be anything).
Source code in src/openbtk/data/ehr/cohort.py
88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 | |
openbtk.data.ehr.cohort.quasi_identifiers ¶
quasi_identifiers(records: Iterable[PatientRecord], fields: Sequence[str] = ('birth_year', 'gender', 'race', 'ethnicity'), *, as_of: date | None = None, include_id: bool = False) -> Iterator[dict[str, object]]
Yield one row per patient with the attributes a person could be linked on.
The bridge from a cohort to openbtk.deid.kanonymity.k_anonymity_report and
anonymise_to_k. Streaming: one row is built per record and nothing is retained.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
records
|
Iterable[PatientRecord]
|
The cohort. |
required |
fields
|
Sequence[str]
|
Which attributes to read: |
('birth_year', 'gender', 'race', 'ethnicity')
|
as_of
|
date | None
|
The date |
None
|
include_id
|
bool
|
Also emit |
False
|
Raises:
| Type | Description |
|---|---|
ValueError
|
On an unknown field, or |
Example
from openbtk.data.ehr.schemas import Demographics patient = PatientRecord( ... patient_id="pt-1", ... demographics=Demographics(birth_date=date(1980, 6, 1), gender="female"), ... source_system="fhir-r4", ... ) list(quasi_identifiers([patient], ["birth_year", "gender"])) [{'birth_year': 1980, 'gender': 'female'}]
Source code in src/openbtk/data/ehr/cohort.py
184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 | |