Radiology Reports Pipeline
Building the infrastructure that makes longitudinal questions askable.
The problem
In IBD, imaging is often sparse by necessity. A patient may accumulate three or four MR enterographies across six years, each reported at a different stage of their disease, each sitting in a folder as a free-text Word document. The clinical information is there: bowel wall measurements, activity descriptors, structural findings noted and then effectively lost to follow-up. Much of the data needed for longitudinal questions already exists. It is simply not in a form that research can use.
That gap, between data that exists and data that is reachable, is the kind of problem I find difficult to leave alone.
The corpus
The dataset is approximately 9,800 abdominal radiology reports from a single institution, spanning nearly a decade. Modalities include MR enterography for IBD and Crohn’s disease, MRCP for primary sclerosing cholangitis, liver and biliary MRI, rectal cancer staging, and gynaecological pelvic MRI. One Word document per report. Reports follow a consistent free-text structure: modality, clinical information, optional prior-imaging reference, findings, and impression.
Pipeline
Where the difficulty was
Before any research use, the corpus had to be parsed, de-identified, and structured. That is not the intellectually exciting part of the work. It is, however, the part that determines whether anything built later is trustworthy.
De-identification turned out to be harder than anticipated, for reasons specific to this corpus. The reports span nearly a decade, and Persian names transliterated into English accumulate variation over time: different spellings, different name ordering, entries that are consistent within one reporting period and inconsistent across another. A generic named-entity approach would not hold. The pipeline matches each report only against name tokens derived from its own filename, after a normalisation step that strips modality, anatomy, and date fragments from the filename before any matching begins.
What complicated this further is that numbers in radiology reports carry multiple roles simultaneously. A date fragment, a bowel wall measurement, and a reference to a prior study interval are all numeric, and all appear in close proximity. Collapsing them indiscriminately would destroy clinical content. The de-identification works through ordered passes that distinguish these categories deliberately, flagging ambiguous cases for review rather than resolving them silently.
Current state
Paused. Phases 0, 1, and 1b are complete on the full corpus. Phase 2 de-identification has been built, run on the full corpus, and passed two manual validation layers.
The public repository holds the code with synthetic-only tests. No patient data is stored in it.
Pipeline built in Python, with python-docx, pandas, and pytest. Paused after Phase 2.
Last updated: October 2026.