Interoperability of standardised electronic healthcare records facilitates transfer learning
1Queen Mary University of London, London, United Kingdom
*Corresponding author; email: r.henkin@qmul.ac.ukAbstract
Electronic healthcare records (EHR) use codes from different vocabularies to describe medical occurrences, often varying by type of care and country. Common data models (CDM) such as the Observational Medical Outcomes Partnership (OMOP) have been developed to enable the combination and comparison of heterogeneous datasets. We use the OMOP Standard Vocabularies to standardise two English EHR datasets and assess their interoperability through a BERT-based transformer model via pretraining and fine-tuning. Our results show the potential for standardisation to empower transfer learning, with tradeoffs related to data loss.
Article notes
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
ER is funded by the Wellcome Trust Health Data in Practice (HDiP) Programme (218584/Z/19/Z)
1Introduction
Electronic healthcare records (EHR) use codes from different vocabularies, often tailored to different countries’ healthcare realities, to describe diseases, medications and other kinds of occurrences in clinical practice. Within the same country or even care system, there is considerable heterogenity between coding systems used in primary and secondary care, for drug prescriptions and surgical procedures. To navigate these ontological challenges and standardise clinical data, common data models (CDM) such as the Observational Medical Outcomes Partnership (OMOP) have been developed [8]. CDMs include data structures and vocabularies to facilitate combining and comparing datasets to unlock health data analysis on a global level [7].
The standardisation of clinical ontologies has also opened doors for transfer learning, where knowledge gained from one dataset can be applied to another without requiring signficant effort to align vocabularies or retrain models [4]. Previous work has successfully trained foundation models on OMOP CDM data from one hospital and applied these models to other out-of-domain datasets across different prediction tasks [2, 9, 6].
In this study, we map the medical codes from two UK datasets of routinely collected data from Clinical Practice Research Datalink (CPRD), GOLD and Aurum, to standard concepts in the OMOP Standardized Vocabularies (SV). To assess interoperability, we pretrain a transformer model on CPRD Aurum, and fine-tune on both CPRD Aurum and GOLD. We report on the quality of the mapping, model performance and impact of transfer learning.
2Methodology
2.1Data
We used EHRs from two datasets, CPRD GOLD and Aurum, and include: individuals ≥ 16 years, registered with an English general practice between January 1, 2010, and December 31, 2020, eligible for linkage to secondary care data and diagnosed with at least 2 long-term conditions (LTC) defined by prevalidated codelists [1]. In Aurum, only general practices based in London were included. The primary care datasets were linked to the Hospital Episode Statistics (HES) for secondary care data, Office for National Statistics (ONS) mortality data and Index of Multiple Deprivation (IMD) data.
We identified individuals within these larger datasets with complex multi-morbidity (CMM), defined as the diagnosis of at least three LTCs that impact three separate body systems [3] using the same code lists [1]. We extracted all data from the date of CMM, which formed our baseline, until deregistration, death or study end (31/12/2020). This resulted in 185,798 (13%) cases and 1,174,173 (87%) controls in CPRD GOLD and 57,328 (8%) cases and 659,691 (92%) controls in CPRD Aurum. This data was used for the pre-training and fine-tuning of the transformer model. See Appendix for more details.
2.2Mapping pipeline
We took a pragmatic approach to mapping codes. First, we downloaded the standard set of vocabularies from the Athena library 1 along with additional vocabularies required for the CPRD datasets. We then mapped internal CPRD codes medcodid to external coding systems: SNOMED CT, OPCS, ICD10, Read and BNF, which are the source vocabularies. Next we mapped source codes, which are referred to as concept codes in OMOP, to concept IDs in the downloaded vocabularies. If codes are found in the SVs these can be classified as non-standard or standard. Unique concept IDs can map non-standard concepts to standard concepts using the OMOP Concept Relationship table. For example, 271649006 is a concept code from SNOMED, which is mapped to the standard concept ID 4152194. The use of standard concepts also enables consolidation of codes. For example, multiple SNOMED codes to record blood pressure can be mapped to a single blood pressure standard concept.
We assessed the total volume of data and calculated the total number of concept codes with no mapping to SVs, non-standard concepts and standard concepts. We report on mapping of the larger datasets, prior to the selection of our CMM cohort.
2.3Model
We pretrained and fine-tuned a BERT-based model from scratch on our clinical corpus consisting of standard concept IDs from our CMM cohort. We used concept IDs as they are unique. We built chronologically ordered sequences of concept IDs for each patient, e.g. 4152194 4334559 4086275, keeping only patients with sequences longer than 3 concepts within the study period.
We trained a domain-specific WordPiece tokenizer from scratch on the whole CMM cohort data from CPRD Aurum. The vocabulary size was set to 20,000 and only codes that appeared over 3 times in the dataset were kept, resulting in 57% coverage of the Aurum vocabulary. We pretrained a 107M parameter BERT model on a masked language modelling task, where 15% of tokens are randomly masked. The data was split 80/10/10 for train, test and validation datasets. All sequences were truncated from left, with a maximum context length of 512. We fine-tuned the pretrained models separately on Aurum and GOLD, on a downstream clinical prediction task: 5-year all-cause mortality in our CMM cohort. For fine-tuning, we kept only data from the date of CMM and up to 5 years before a death date for cases. For controls, we kept all data until end of study or deregistration.
We report area under the precision-recall curve (AUPRC) and area under the receiver operating characteristic curve (AUROC). All experiments were conducted on 4 A100 GPUs in the Queen Mary’s Apocrita HPC facility, supported by QMUL Research-IT [5].
3Results
3.1Concept Mapping
In both GOLD and Aurum, the majority of codes and rows in the data were successfully mapped to standard concepts. In GOLD the majority of concepts were mapped via other vocabularies, particularly as Read and ICD10 codes are not SVs in OMOP. In GOLD, the greatest loss of data was in prescription data, where 5.5% of the data could not be mapped to OMOP concept IDs (unmapped codes), and Read codes where 3.8% of the data was mapped to a non-standard concept ID which was not matched to a standard concept ID. In Aurum, the largest loss of data was again through unmapped SNOMED source codes (11.2%) and non-standard concepts (8.7%) in the pirmary care data. See Appendix for more details for the complete mapping results.
3.2Model performance
We evaluated our pretrained model by fine-tuning on both same-database scenarios AURUM → AURUM and cross-database transfer scenarios AURUM → GOLD. Model performance was higher within the same-database, but demonstrated good performance between different datasets with a 0.10 difference between models (see table 1). The results may reflect the contents of the tokenizer as we trained our custom tokenizer on concept IDs from CPRD Aurum only. The tokenizer covered 68% of the concept IDs in the Aurum fine-tuning dataset and only 37% in GOLD. We report on the AUROC and AUPRC plots in the Appendix.
4Conclusion and future work
Standardisation enabled us to reuse models and lower the time and computational expense required to train multiple models from scratch. Mapping to OMOP also helped reduce the size of the datasets, as multiple source codes (from SNOMED, Read, etc.) can map to one OMOP concept ID.
Standardisation is a step towards interoperability, but does not guarantee it as the SVs include coding systems from different countries. For example, the EHRSHOT model [9] was trained on SV-compatible data, but predominantly includes codes from the LOINC system for test measurements – these tend to be recorded as standard concepts in SNOMED in UK datasets.
Among the limitations of our work, our pragmatic download strategy of standardised vocabularies led to some codes from Aurum and GOLD not being mapped because the correct vocabulary was not downloaded. We also did not calculate or explore the amount of data lost per patient; future work should understand where and for whom data is lost in the conversion to OMOP. Future research should also assess the capacity for zero or few-shot learning in both the same and across datasets.
We present an initial exploration of the possibilities of interoperability between two EHR datasets that rely on different clinical ontologies. Whilst the mapping steps were largely successful, data loss was experienced across both datasets. Models performed better when fine-tuning on the same dataset, but there was evidence of some transfer learning across databases, demonstrating potential to enable the sharing of large healthcare models.
Data Availability
The data underlying this article is provided by the UK CPRD electronic health record database, which is only accessible to researchers with protocols approved by CPRD Research Data Governance.
Acknowledgments
ER is funded by the Wellcome Trust Health Data in Practice (HDiP) Programme (218584/Z/19/Z). This work uses data provided by patients and collected by the NHS as part of their care and support. The data used in this study was under licence.
Disclosure of Interests
The authors have no competing interests to declare that are relevant to the content of this article.
Appendix Group
Appendix
Data Processing
CPRD GOLD and Aurum contain four primary coding systems: primary care (Read or SNOMED), prescriptions (dm+d), hospitalisations (ICD10) and procedures (OPCS4). For the purpose of this study, we did not map the CPRD data structures to the OMOP CDM and limited our work to code standardisation. In CPRD GOLD, we combined the tables of clinical findings, referrals and test results to represent primary care data.
Before mapping datasets to OMOP CDM, we removed all rows without valid dates, those that were empty, occurred before birth or after death of the patient, or occurred after the end of the study.
BERT model
Pretraining was conducted using the AdamW optimizer, with a learning rate of 5e-5 and warm-up ratio of 0.1. We trained the entire train dataset of CPRD Aurum for 5 epochs with early stopping.
The model was fine tuned for 3 epochs with early stopping with a learning rate of 2e-5. We used weighted cross entropy due to class imbalance. To assess variations in model performance we employed a bootstrap resampling method on our test set using 1000 iteractions and report a 95% confidence interval.
Mapping results
Tables 2 and 3 show the mapping results for the four source vocabularies in GOLD and the four source vocabularies in Aurum. Table 4 summarises the distributions of cases and controls between GOLD and Aurum, as well as characteristics of input sequences to the model. The Aurum dataset is slightly more imbalanced than the GOLD dataset. Tables 5 and 6 show the top 30 concepts that, after standardisation, are common between cases and controls in GOLD and Aurum, respectively. Although many concepts have similar distributions, Aurum contorls have an overwhelming amount of administrative codes including SMS messages and general labels such as “History of event”.