The COVID-19 pandemic has lasted for over two years, with multiple waves across the world. Although hospital systems have been cyclically overwhelmed by patients seeking care for COVID-19, as healthcare systems began to open up before the second wave, elective surgeries were again performed starting in the later part of 2020, and especially in the second quarter of 2021 many have approached the healthcare system for health issues (e.g., accidents, strokes, etc.) while incidentally infected with SARS-CoV-2.[32] This, along with the high false positive rate of SARS-CoV-2 PCR tests in some situations,[33–36] has led to increasing numbers of misclassified patients in analyses of COVID-19 characteristics and severity. This could be creating significant detection and reporting bias, leading to erroneous conclusions.[10–13] This study presents a multi-institutional characterization of 1,123 hospitalized patients either incidentally infected with SARS-CoV-2 or specifically hospitalized for COVID-19 in four healthcare systems across multiple waves using consensus-based chart review criteria.

We applied an itemset-mining approach and established Hospital System Dynamics (HSD) principles to phenotype SARS-CoV-2 PCR-positive patients who were admitted specifically for COVID-19, using data on charting patterns (e.g., presence of laboratory tests within 48 hours of admission) rather than results (e.g., laboratory results).[16,37] HSD examines healthcare process data about a hospitalization, such as ordering/charting patterns, rather than the full dataset. For example, to study severely ill patients, HSD might select patients with a high total number of labs for a patient on the day of admission. This could be an indirect measure of clinical suspicion of disease complexity or severity. Previous work shows that proxies such as total number of laboratory tests on the day of admission or the time of day of laboratory tests can be highly predictive of disease course.[24,37] Our methods sorted out who was treated for COVID-19 automatically, over time, with specificities above 0.70 even for some phenotypes discovered at a single site and applied to all four. We focused on specificity because the goal was to remove false positives (i.e., incidental SARS-CoV-2) from the cohort.

Our chart review protocol illustrates that patients who were admitted and have a positive SARS-CoV-2 PCR test were more likely to be admitted specifically for COVID-19 when disease prevalence was high (at least prior to Omicron). However, during periods in which healthcare systems were less restrictive (i.e., resumed routine surgeries), a secondary measure/phenotype is critical for accurately classifying admissions specifically for SARS-CoV-2 infection.

As expected, we observed a lower proportion of hospitalizations specifically for COVID-19 in the summer months when disease prevalence was lower (Figure 3). One would expect this, because there were fewer overall admissions as hospitals were recovering from the previous wave.

As expected, the top chart review criteria (Table 3) were respiratory insufficiency in admissions specifically for COVID-19 and other for incidental and uncertain admissions with SARS-CoV-2. Surprisingly, 10-20% of patients admitted with incidental SARS-CoV-2 were diagnosed with pneumonia, respiratory failure, or acute kidney injury (Table 4). This could reflect data collection issues, where some systems might repeat past problems automatically at hospital admission. In the case of codes for acute kidney injury, further investigation is needed to determine whether SARS-CoV-2-associated acute kidney injury (including COVID-19-associated nephropathy) occurs in patients we otherwise classified as having incidental admissions.[38]

Healthcare systems are beginning to explore phenotyping feature sets to report admissions specifically for COVID-19. Starting January 2022 in Massachusetts, hospitals began reporting the number of for-COVID hospitalizations as the count of admitted patients with both a SARS-CoV-2 positive test and a medication order for dexamethasone.[22,23] This simple phenotype was designed by the Massachusetts Department of Public Health as a first attempt, and it was based only on treatment recommendations for moderate-to-severe COVID-19 with hypoxia. It was not validated against a gold standard. Nonetheless, it illustrates the interest in EHR-based phenotyping for COVID-19.

Phenotypes with diagnosis codes tended to be the best performing predictors of admissions specifically for COVID-19. This could be because diagnosis codes represent either a clinically informed conclusion or a justification for ordering a test (implying the clinician suspected COVID-19). However, diagnoses are less prevalent in the population than laboratory tests and might not cover the entire population of admissions for COVID-19. Further, diagnoses early in hospitalization also do not always reflect the patient’s eventual diagnosis or hospital-related complications that are more accurately reflected in discharge diagnoses. There was also some heterogeneity in the diagnoses used at different sites (e.g., B97.29 “other COVID disease” was a top predictor only at Site B). In addition, presence of laboratory tests are useful for real-time detection systems because diagnosis codes usually are assigned after discharge. Clusters of tests for inflammatory markers (e.g., LDH, CRP, and ferritin) appeared across most sites as predictive of hospitalizations specifically for COVID-19, which fits intuitively because one of the underlying systemic pathophysiological mechanisms of SARS-CoV-2 is thought to be an inflammatory process,[39,40] and guidelines therefore have encouraged health care providers to check inflammatory markers on COVID-19 admissions.[41,42] Many of these inflammatory labs are not routinely ordered on all hospitalized patients and would therefore be expected to help distinguish COVID-19 from other patients. However, laboratory protocol differences across sites may have reduced generalizability for this metric.

Our methods generated pairs of items using OR and groups of up to 4 using AND logical operators. Our feature sets were somewhat vulnerable to the problem that specificity decreases when multiple elements are combined with OR although, in general, OR feature sets performed better across sites because they could be designed to choose the top performing elements at each site.

In addition to site differences, we also found changing disease management patterns over time. At the start of the pandemic, the only predictive phenotype was a pneumonia diagnosis. As standard COVID-19 order recommendations began to appear, laboratory orders became more consistent and predictive. Next, remdesivir began to be administered regularly. Finally, COVID-specific ICD-10 codes began to appear.

Overall, we found that an informatics-informed phenotyping approach successfully improved classification of for-COVID vs. incidental SARS-CoV-2 positive admissions, though generalizability was a challenge. Although some transfer learning is apparent (i.e., a few phenotypes performed well across sites), local practice and charting patterns reduced generalizability. Specifically, phenotypes involving only laboratory tests did not perform well at Site C, because the prevalence of these labs was low in the overall EHR data. This could be due to a data extraction or mapping issue in the underlying data warehouse. BIDMC had lower performance than other sites on the cross-site rules but not on the site-specific rules, perhaps highlighting less typical clinician treatment patterns.