The open-source layer between
biomedical data and modern AI.
Openbtk turns EHR data and clinical text into model-ready, de-identified, auditable inputs — and wraps LLM outputs in clinical guardrails. Runs on a laptop or behind a hospital firewall, with any LLM provider.
"Ask questions of the discharge summaries" costs six weeks of plumbing.
Every team re-derives the same undifferentiated work — a note parser, a de-identifier that actually catches MRNs and admission-relative dates, chunking that doesn't split "Assessment" from "Plan," entity linking, an LLM that cites codes which don't exist — and then an IRB or an EU AI Act auditor asks exactly what data, model, and config produced a given output. Most teams have nothing to show.
De-identification you can audit
Not "runs Presidio." An ensemble of rule-based recognizers, clinical NER, and optional LLM adjudication, targeting entity-level F1 ≥ 0.95 on i2b2 2014 with a reproducible, one-command benchmark harness. Every run emits a DeidReport — entity counts, confidence, method attribution — that never contains the PHI itself.
Run provenance as a primitive
Every pipeline execution emits a RunManifest: config hash, component versions, input data digests, guardrail outcomes, timings. Content-addressable, diffable, and designed to satisfy EU AI Act Article 12 record-keeping obligations. openbtk replay reconstructs a run byte-for-byte.
These two things, together, are what nothing else in open source has today.
A framework-free, layered core
Strict downward-only imports, mechanically enforced in CI. A registry-mediated plugin model — every extension point is a base class you register against, not a fork you maintain.
graph TD
L6["L6 · Applications & Recipes
clinical_text_rag · ehr_qa · deidentify_corpus"]
L5["L5 · Interfaces
CLI (Typer) · optional LangChain adapter"]
L4["L4 · Orchestration
Streaming DAG pipeline executor · RunManifest"]
L3["L3 · Cross-cutting Services
llms · embeddings · retrieval · guardrails · terminology · eval · de-id"]
L2["L2 · Modality Modules
clinical_text · ehr — imaging/biosignals/genomics gated"]
L1["L1 · Core
registry · base classes · config · schemas · errors · provenance"]
L6 --> L5 --> L4 --> L3 --> L2 --> L1
classDef l6 fill:#131c33,stroke:#60a5fa,stroke-width:2px,color:#f1f5f9;
classDef l5 fill:#131c33,stroke:#7c9bfb,stroke-width:2px,color:#f1f5f9;
classDef l4 fill:#131c33,stroke:#967dfa,stroke-width:2px,color:#f1f5f9;
classDef l3 fill:#131c33,stroke:#a06ff8,stroke-width:2px,color:#f1f5f9;
classDef l2 fill:#131c33,stroke:#aa62f6,stroke-width:2px,color:#f1f5f9;
classDef l1 fill:#131c33,stroke:#b455f4,stroke-width:2px,color:#f1f5f9;
class L6 l6
class L5 l5
class L4 l4
class L3 l3
class L2 l2
class L1 l1
Domain-aware by default
Built for clinical text and structured EHR data from day one, not bolted on.
Wrap, don't reinvent
MONAI for imaging, medspaCy for clinical NLP. Openbtk won't out-build the specialists.
Streaming by default
Loaders and chunkers stream — target: 10M notes under 4GB RSS.
Safe by construction
PHI-redacting logs on by default. Offsite LLM providers refused by default.
Everything between raw data and a trustworthy prompt
De-identification
Ensemble rules + NER + optional LLM verifier. All 18 HIPAA Safe Harbor categories.
- 5 transform modes: redact, surrogate, hash, tag, date-shift
- Per-patient consistent replacement
- Output-side PHI leakage guardrail
Clinical Text
Loaders for plain text, MIMIC III/IV, JSONL, CDA/CCD.
- Section-aware chunking — never splits Assessment from Plan
- ConText-style negation & uncertainty detection
- PubMedBERT, BioBERT, ClinicalBERT, SapBERT, MedCPT presets
EHR & Terminology
FHIR R4, OMOP CDM, HL7v2 into a unified PatientRecord.
- Structured → narrative timeline serialization
- Streaming, composable cohort builder
- SNOMED CT, LOINC, RxNorm, ICD-10/11, UMLS resolution
Providers
OpenAI, Anthropic, HF local, vLLM, Azure, Bedrock, Vertex.
- Biomedical model presets (MedGemma, Meditron, BioGPT)
- Every provider declares
sends_data_offsite - Structured output validated against a Pydantic schema
Retrieval
FAISS, Chroma, Qdrant vector stores.
- Concept-overlap reranking by shared UMLS CUIs
- Clinical cross-encoder reranking, hybrid dense+BM25
- Source provenance to exact character offsets
Guardrails, Pipelines & CLI
Composable, never-raise guardrail checks.
- Groundedness, terminology validity, dose plausibility
- Config-as-data YAML pipelines, not agent control flow
openbtk run · validate · deid · replay · doctor
Designed to read like plain Python
from openbtk.deid import DeidEngine, DeidMode
engine = DeidEngine(mode=DeidMode.SURROGATE, recall_bias="high")
result = engine.deidentify(text, patient_id="hashed-123")
result.text # "Robert Chen was seen on [DATE-SHIFTED] for..."
result.report # DeidReport — audit trail, never contains the PHI itselfPlanned API — target design, not yet shipped.
from openbtk.pipelines import Pipeline, Step
pipeline = (
Pipeline("clinical-rag")
.add(Step("load", "loader.clinical_text.plain_text", path="./notes"))
.add(Step("deid", "preprocessor.general.deidentify", mode="surrogate"))
.add(Step("chunk", "chunker.clinical_text.section_aware", max_tokens=512))
.add(Step("embed", "embedding.clinical_text.pubmedbert"))
.add(Step("index", "vectorstore.general.faiss", path="./index"))
.guard("guardrail.general.phi_leakage", at="after:deid", on_violation="block")
)
manifest = pipeline.run()
print(manifest.status, manifest.run_id) # everything an IRB/auditor will ask forPlanned API — target design, not yet shipped.
A toolkit for builders, not a finished product
Not for clinicians wanting a diagnostic tool. Built for the people wiring biomedical data into AI systems.
Building discharge-summary Q&A on a hospital partner's data — needs de-id reliable enough to survive a security review.
Studying readmission signals across notes and labs on MIMIC-IV, without needing 300GB of RAM.
Weighing build-vs-buy against a six-figure commercial license — needs on-prem execution and an audit trail QA accepts.
Adding a new modality by implementing base classes and registering — without patching the framework core.
Why not just use X?
Openbtk doesn't try to out-build category leaders. It fills the gap between them.
| Tool | Camp | Where Openbtk differs |
|---|---|---|
| PyHealth 2.0 | Predictive DL health toolkit | Complementary, not identical — PyHealth predicts, Openbtk is the LLM/agent-era data + safety layer. |
| MONAI | Medical imaging | Not reimplemented — wrapped in a future imaging module. |
| LangChain / LlamaIndex | Generic orchestration & RAG | Domain-blind: no FHIR resource, DICOM series, or lab-panel concepts. One isolated optional adapter instead of a fork. |
| Guardrails AI / NeMo Guardrails | Generic output validation | No clinical semantics — no dose plausibility, code validity, or chart-groundedness checks. |
| John Snow Labs | Commercial clinical NLP | Closed source, license cost excludes academia/startups, no public reproducible eval harness. |
| AWS / GCP / Azure health AI | Hyperscaler platforms | Vendor lock-in and usage-scaled cost. Openbtk competes on portability, auditability, and running on a laptop. |
Milestone M0 complete. M1 — the core framework — is next.
Openbtk is pre-alpha today. Follow the milestone-by-milestone build in public on GitHub.
Follow progress on GitHub →