Population level impact of a pulse oximetry remote monitoring programme on mortality and healthcare utilisation in the people with covid-19 in England: a national analysis using a stepped wedge design
Patient Safety Translational Research Centre, Institute of Global Health Innovation, Imperial College London, London, SW7 2AZ, United Kingdom
Department of Primary Care and Public Health, Imperial College London, London, W6 8RP, United Kingdom
Centre for Mathematics of Precision Healthcare, Department of Mathematics, Imperial College London, London, SW7 2AZ, United Kingdom
NHS England and Improvement, London, SE1 6LH, United Kingdom
NHS Digital, 7-8 Wellington Place, Leeds, West Yorkshire, LS1 4AP, United Kingdom
National Heart and Lung Institute, Imperial College London, London, SW7 2AZ United Kingdom
*Corresponding Author: Dr Thomas Beaney, Patient Safety Translational Research Centre, Institute of Global Health Innovation, Imperial College London, London, SW7 2AZ, United Kingdom, Email: thomas.beaney@imperial.ac.ukAbstract
Objectives
To identify the population level impact of a national pulse oximetry remote monitoring programme for covid-19 (COVID Oximetry @home; CO@h) in England on mortality and health service use.
Design
Retrospective cohort study using a stepped wedge pre- and post-implementation design.
Setting
All Clinical Commissioning Groups (CCGs) in England implementing a local CO@h programme.
Participants
217,650 people with a positive covid-19 polymerase chain reaction test result and symptomatic, from 1st October 2020 to 3rd May 2021, aged ≥65 years or identified as clinically extremely vulnerable. Care home residents were excluded.
Interventions
A pre-intervention period before implementation of the CO@h programme in each CCG was compared to a post-intervention period after implementation.
Main outcome measures
Five outcome measures within 28 days of a positive covid-19 test: i) death from any cause; ii) any A&E attendance; iii) any emergency hospital admission; iv) critical care admission; and v) total length of hospital stay.
Results
Implementation of the programme was not associated with mortality or length of hospital stay. Implementation was associated with increased health service utilisation with a 12% increase in the odds of A&E attendance (95% CI: 6%-18%) and emergency hospital admission (95% CI: 5%-20%) and a 24% increase in the odds of critical care admission in those admitted (95% CI: 5%-47%). In a secondary analysis of CO@h sites with at least 10% or 20% of eligible people enrolled, there was no significant association with any outcome measure. However, uptake of the programme was low, with enrolment data received for only 5,527 (2.5%) of the eligible population.
Conclusions
At a population level, there was no association with mortality following implementation of the CO@h programme, and small increases in health service utilisation were observed. Low enrolment of eligible people may have diluted the effects of the programme at a population level.
Article notes
Competing Interest Statement
JC has received fees from Philips UK Limited for consultancy services outside of the submitted work. SE has received fees for an educational lecture sponsored by Astra Zeneca and is co-clinical director for the NHS England and Improvement London Respiratory Network. All other authors report no conflicts of interest.
Funding Statement
This work was funded by NHS England and supported by the National Institute for Health Research (NIHR) Imperial Patient Safety Translation Research Centre. Infrastructure support was provided by the NIHR Imperial Biomedical Research Centre (BRC). JC acknowledges support from the Wellcome Trust (215938/Z/19/Z).
Background
Since the start of the covid-19 pandemic, asymptomatic (‘silent’) hypoxia has complicated the assessment and care of patients with covid-19.[1] Hypoxia has been shown to be an important predictor of mortality and the need for hospital admission in patients with covid-19, yet those patients with asymptomatic hypoxia may be unaware of dangerously low blood oxygen saturations.[2,3] Pulse oximetry allows patients and clinicians to regularly monitor a patient’s oxygen saturation and promptly initiate escalation of treatment should deterioration occur.[1] Health systems across the world introduced remote monitoring pathways, including the use of pulse oximetry, to support the care of patients with Covid-19 outside hospital.[4,5] In November 2020, NHS England and Improvement (NHSEI) introduced the COVID Oximetry @home (CO@h) programme, recommending that all Clinical Commissioning Groups (CCGs) in England provide services to monitor patients with a diagnosis of covid-19 at home using pulse oximetry.[6] This service builds upon local remote monitoring services provided by individual CCGs in earlier stages of the pandemic. CCGs are responsible for the establishment of services in their area, although CO@h services may be shared between CCGs, and more than one service may operate within a single CCG.[7]
Patients are eligible for enrolment into a CO@h programme if they have symptomatic covid-19 and are aged 65 years or older or are identified as clinically extremely vulnerable (CEV) based on concurrent medical conditions and/or treatments.[8] Additionally, clinical judgement can be applied to consider other individual risk factors, including pregnancy, learning disability, and socioeconomic deprivation. Those enrolled are encouraged to record three oximetry readings daily and advised to attend or call emergency services if the reading is 92% or less, or to contact primary care services for readings of 93-94%. The programme is intended to accept patients from primary care, NHS Test and Trace, ambulance services and accident and emergency departments, in contrast to ‘COVID Virtual Wards’ which aim to support discharge of covid-19 patients from hospital.[9]
The clinical effectiveness of the CO@h programme on mortality and secondary care utilisation remains unknown. The primary aim of this analysis is to identify the impact of the CO@h programme on mortality and use of healthcare services at a population level amongst people eligible for the programme. A secondary aim is to identify the impact of the programme in sites with a high uptake of the programme amongst the eligible population.
Methods
This study used a retrospective cohort of people eligible for the CO@h programme, comparing outcomes at a CCG level using a stepped wedge pre- and post-implementation design. Eligibility was defined as the population resident in England, with a positive covid-19 Polymerase Chain Reaction (PCR) test result, who were symptomatic at the time of testing, from 1st October 2020 to 3rd May 2021. Of these individuals, a subset of people who would have remained eligible throughout the course of the programme, aged 65 years or older, or who were Clinically Extremely Vulnerable (CEV) was selected. Care home residents were excluded from the analysis, as previous work has suggested significantly higher mortality in this group.[10]
Five primary outcomes were selected, defined as occurring within 28 days of the date of a positive covid-19 test:
- Death from any cause
- One or more Accident and Emergency department (A&E) attendances
- One or more emergency hospital admissions
- One or more critical care admissions (of those admitted to hospital)
- Total hospital length of stay in days, of those admitted who did not die within 28 days
Data sources and processing
Covid-19 testing data was provided through the Public Health England Second Generation Surveillance System[11], which collates positive covid-19 test results conducted in laboratories across England. Data was available from 1st October 2020 to 3rd May 2021. This analysis used PCR tests, with symptoms documented at the time of testing.[12] Where more than one test was recorded, only the date of first test was used. Data on the number of patients enrolled (‘onboarded’) onto the CO@h programme were submitted from CO@h sites via NHS Digital’s Strategic Data Collection Service (SDCS).[13] Primary care data was sourced from the General Practice Extraction Service (GPES) Data for Pandemic Planning and Research (GDPPR).[14] Hospital Episode Statistics (HES) data[15] and the Emergency Care Data Set[16] provided data on hospital admissions and A&E attendances up to 31st May 2021. Data on registration of deaths was sourced from the Office for National Statistics (ONS), with data available up to 5th July 2021. Datasets were linked using a deidentified NHS patient ID.
The study population were assigned to a CCG based on a person’s CCG of residence when the test was performed. Patient demographic data, including age, sex, ethnicity, lower layer super output area (LSOA) of residence were derived from GDPPR, or, if missing, derived from HES or ECDS. LSOA was linked to measures of deprivation based on the Index of Multiple Deprivation (IMD) 2019.[17] Data on care home residence, CEV status, body mass index (BMI) and smoking status were available from GDPPR only. Information on the following chronic conditions were included, derived from GDPPR: hypertension, chronic cardiac disease, chronic kidney disease, chronic respiratory disease, dementia, diabetes, chronic neurological disease (including epilepsy), learning disability, malignancy/immunosuppression, severe mental illness, peripheral vascular disease and stroke/transient ischaemic attack (TIA). For demographics and chronic conditions, the most recent codes prior to the date of a positive covid-19 test were used to exclude those which may have resulted from covid-19 infection. If no data were available prior to the date of a positive test for age, sex and ethnicity only, then the earliest data following the positive test was used. Full details of the datasets and cleaning approach are provided in Appendix A, with a link to the Systematized Nomenclature of Medicine Clinical Terms (SNOMED-CT) codes for each condition in appendix A.
Statistical analysis
A pre- and post-implementation period were defined for each CO@h site, with implementation start dates for each site provided by NHS England @home. A stepped wedge design was used. All eligible people in each CCG before and after implementation of the CO@h programme were allocated to the control group and intervention group, respectively. Two-level hierarchical regression models were run for each outcome, incorporating random intercepts for CCG. Logistic regression was used for the four binary endpoints and negative binomial regression models were used for the single continuous outcome (length of stay).
Total length of stay was capped at 28 days where a patient was discharged after the 28-day window, and analyses of length of stay excluded patients who died within the 28-day time window.
To account for possible changes in the baseline risk of each outcome over time, the primary models for each outcome incorporated fixed effects for the month of positive covid-19 test. To account for potential differences in the at-risk population before and after implementation, the primary models adjusted patient-level risk factors. Final covariates in each model included age category (years), sex, ethnicity, IMD score, BMI category, month of covid-19 test, CEV status, and clinical co-morbidities. Intra-class correlation coefficients were calculated for each model.
To test the sensitivity of results to time-varying effects and differences in patient risk factors, a series of sensitivity analyses were run:
- A naïve model, not accounting for time or patient-level covariates
- A time-adjusted model, without adjusting for patient-level covariates
- A time-adjusted model, incorporating a random interaction between time and CCG
- A time-adjusted model incorporating a random interaction between time and CCG, and adjusting for patient-level covariates
A secondary analysis was performed on the subset of sites with a higher proportion of eligible people enrolled. Two thresholds were defined a priori, at 10% or more (accounting for sixteen CCGs and 9.4% of the eligible population) and 20% more (accounting for five CCGs and 2.4% of the eligible population) across the whole study period.
Analyses were conducted in the Big Data and Analytics Unit Secure Environment (BDAU SE), Imperial College. Python v3.9.5 and Pandas v1.2.3 were used in data manipulation. Regression models were conducted in Stata v17.0, using the melogit and menbreg commands.
Patient and public involvement
Patients or the public were not involved in the design, conduct or reporting of our research.
Results
A total of 1,714,182 people resident in 106 CCGs in England had a positive PCR covid-19 test between 1 October 2020 and 3rd May 2021, with documented symptoms at the time of the test. Eligibility for the CO@h programme was defined as a person being at least 65 years of age or CEV, resulting in an eligible population of 223,429 (13.0%). Of these, 5,779 (2.6%) were living in a care home at the time of their positive test and were excluded from the analysis. A total of 217,650 people were included in the final analysis (Figure 1).
Most participants were women (54.4%) and of White ethnic background (72.5%), with 125,102 (57.5%) in the five most deprived deciles. More of the population were obese (39.4%) than overweight (33.6%) or had a healthy weight (20.8%). Just over half (54.9%) were never smokers. Hypertension (40.6%), diabetes (31.4%; type 1 and type 2) and chronic respiratory disease (29.3%) were the most common co-morbidities. A total of 5,616 (2.6%) of the study population died within 28 days of a positive covid-19 test, 19.9% attended A&E at least once, 12.2% were admitted at least once, and of those admitted, 16.1% required critical care. There were significant differences in distributions of most of the predictor and outcome variables in the eligible population before and after implementation in each site (Table 1).
Data was received via submissions from CO@h sites for 5,527 patients onboarded to the programme, giving an overall uptake rate based on submitted data of 2.5%. There was considerable variation in uptake across CCGs, ranging from 0.0% to 33.0%, with a median of 2.2% (Figure 2). The earliest date a site became operational was 20th November 2020, with all sites operational from 10th January 2021.
Mixed effects logistic regression was run separately for each outcome, with CCG of residence as a random intercept. Table 2 shows the results for the primary analysis for each outcome, adjusted for month of covid-19 test and patient-level covariates. There was no significant difference in the adjusted odds of 28-day mortality following implementation of the CO@h programme (aOR=1.06, p=0.405). There was evidence of a small increase in both A&E attendances (aOR=1.12, p<0.001) and emergency hospital admissions (aOR=1.12, p<0.001) within 28 days. Of those patients admitted to hospital after implementation there was a 24% increase in the adjusted odds of requiring critical care (aOR 1.24, p=0.012). There was no significant difference in the length of stay of those admitted (p=0.588).
Sensitivity analyses
Sensitivity analyses comparing alternative model specifications are given in the Appendix (Tables A1-5). Naïve models, unadjusted for time, showed significant increases in 28-day mortality, ED attendance and admissions associated with the programme. In contrast, in naïve models, the programme was weakly associated with lower odds of critical care admission. Little meaningful difference was seen in the estimates between models unadjusted or adjusted for patient-level covariates, or with the addition of random time by CCG interactions. The intraclass coefficients for both CCG and CCG by time interactions for mortality, ED attendance and hospital admission models were all <1%, suggesting minimal variation between CCGs that might be accounted for by time-varying CCG level factors.
Secondary analysis of high uptake CCGs
Two secondary analyses using the same stepped wedge designed were performed for sixteen CCGs with 10% or more uptake (Table 4), and for five CCGs with 20% or more uptake (Table 5). In the 10% uptake group, there was a 9% lower odds or mortality, 10% higher odds of A&E attendance and 23% higher odds of critical care admission after implementation, but all effects were statistically non-significant. There was evidence of 27% higher odds of admission (p=0.046). In the 20% uptake CCGs, effect sizes were larger, but none were statistically significant.
Discussion
At a population level, there was no impact on mortality after implementation of the CO@h programme. Implementation was associated with statistically significant increases in A&E attendances and emergency hospital admissions, with a 12% increase in the odds of both A&E attendance or admission within 28 days of a positive test. There was some evidence of an increase in the odds of receiving critical care in patients admitted after implementation of the CO@h programme. In a secondary analysis of sites with at least 10% or at least 20% of eligible people onboarded, there was a trend towards lower odds of death and higher odds of A&E attendance, hospital admission and critical care after implementation, but effects were non-significant.
Increases in A&E attendances, emergency hospital admissions and critical care use following implementation of CO@h may reflect early recognition of silent hypoxia in covid-19. The magnitude of increase in A&E attendances was similar to the magnitude of increase in hospital admissions, suggesting that implementation did not cause a large increase in A&E attendances not requiring admission. However, early intervention might be expected to decrease length of stay and mortality, which was not found here. Despite this, the finding of no significant difference in mortality after implementation suggests that the CO@h pathway is safe and does not cause harm. Uptake of the programme was lower than expected, with data received from only 2.5% of the eligible population. This may be due to incomplete data submissions or may reflect genuinely low uptake of the programme. It could also reflect additional patient selection criteria used by sites, for example, if sites were focussing on patients with higher acuity disease. If uptake were only 2.5%, then there is likely to be a dilutional effect when evaluating results at a population level and we cannot infer a causal relationship at an individual level.
Remote monitoring technologies have been widely used in the management chronic diseases, but with mixed evidence of their clinical effectiveness[18] and there is limited evidence for their use in covid-19 with which to compare our findings (Alboksmaty, A et al. Effectiveness and safety of pulse oximetry in remote home monitoring of COVID-19 patients: a systematic review. Unpublished, under review at The Lancet Digital Health). A previous pilot study of four NHS covid-19 pulse oximetry programmes in England indicated the pathway was safe, but did not include a control group to compare outcomes.[19] A study in the US of patients with covid-19 referred to a remote patient monitoring pathway after discharge from hospital found lower odds of emergency department or hospital reattendance in those enrolled, but did not assess mortality.[20] Furthermore, the characteristics of patients discharged from hospital with covid-19 may be different to people in the community eligible for CO@h, who are likely to be at an earlier stage of disease.
In a separate study of CO@h programme by the same authors using a retrospective matched cohort design, patients with covid-19 enrolled after assessment in A&E were found to have lower odds of mortality and critical care use, and higher odds of subsequent A&E use and admission, compared with matched controls.[21] Collectively, results from the two studies suggest that although there was no impact on mortality at a population level, there is evidence for the effectiveness of the programme at an individual level (albeit within a narrower eligibility cohort of patients assessed in A&E), but indicate that the programme could not be scaled up to provide a benefit to all eligible people nationally. Crucially, neither of the studies indicate a concern over the clinical safety of the programme or that it causes harm. There is a need for further research into the barriers and facilitators to implementation of the programme and the obstacles to wider adoption which may aid policy-makers and commissioners in implementing remote monitoring programmes in the future. There is also a need to understand the equity of access to CO@h, and whether low enrolment nationally is reflected in lower numbers onboarded in certain sociodemographic groups, and to evaluate user experience and the cost implications of the programme.
Strengths and limitations
A strength of this study is the availability of comprehensive data on covid-19 testing, defining a robust denominator population. Through linkage to primary and secondary care records, we identified the eligible population and adjusted for underlying risk factors differing between the population pre- and post-implementation. Eligibility for the programme was not absolute, with NHS England recommending an extended age threshold of 50 years and over from February 2021. Our analysis focussed on a narrower subgroup of people aged 65 or over or CEV, who would have remained eligible over the full study period, but as a result, findings might not be generalisable to all those included in the programme.
The stepped wedge design was chosen in part as it is robust to biases in decisions regarding onboarding of patients (such as systematic onboarding of higher or lower risk patients), which would impact individual level study designs. The effect estimates are also not impacted by lack of submission of patient-level programme data on patients onboarded, which may not be complete. However, incompleteness of received data does impact on our ability to judge the degree to which the study was adequately powered to detect a difference in mortality should one exist. The primary analysis adjusted for patient covariates, for which data were incomplete. However, the impact on the estimates is unlikely to be significant, as sensitivity analyses adjusted for patient-level covariates produced very similar estimates to the primary analysis.
Results may also be confounded by events external to the programme. A separate study using the same data sources showed significant increases in the case hospitalisation and fatality risk from covid-19 from October 2020 to January 2021 at the time CO@h sites were becoming operational.[22] These trends may relate to winter effects on mortality, new treatments, hospital pressures and spread of the alpha variant which became the dominant covid-19 strain in England during December 2020.[23,24] Despite the stepped wedge analysis incorporating time as an adjusting covariate, there may be residual confounding between sites becoming operational, and the underlying increase in mortality rates, particularly given the low numbers onboarded. Incorporation of additional CCG-level and hospital-level covariates was considered but estimates of the residual variation explained by clustering at the CCG-level (intra-class correlation coefficient) were less than 1%, suggesting these would have limited impact. The primary analysis considered time-varying CCG level effects in additional sensitivity analyses for each outcome, with almost identical effect measures compared to the primary analysis, suggesting little impact of time-varying differences between CCGs.
Across England, CO@h sites implemented different types of model, run by different sectors of the healthcare system, and with different recommendations for the frequency of monitoring.[4] National population effect estimates as presented here may therefore mask variation in the effectiveness between programmes and so not be representative of individual sites. Our analysis incorporated outcomes measured up to 28 days from the date of a positive covid-19 test, and different effects may be seen over longer time periods. Longer term outcomes, such as ‘Long Covid’ might also be affected by early recognition of low oxygen saturations but were outside of the scope of this study.
Conclusion
Implementation of the CO@h programme across England had no impact on mortality at a population level and was associated with small increases in A&E attendances, hospitalisations and critical care use in people with covid-19 aged 65 years or over or CEV, which may indicate early recognition of hypoxia and escalation. Lower than expected enrolment of eligible people may have diluted the effects of the programme at a population level. There is a need for further research into the uptake and effectiveness of remote monitoring programmes for covid-19.
Supporting information
Data Availability
The patient level data used in this study are not publicly available but are available to applicants meeting certain criteria through application of a Data Access Request Service (DARS) and approval from the Independent Group Advising on the Release of Data.
Acknowledgements
The authors would like to thank Hutan Ashrafian, Gianluca Fontana, Saira Ghafur, Melanie Leis and Mahsa Mazidi for their input and support. Data management was provided by the Big Data and Analytical Unit (BDAU) at the Institute of Global Health Innovation (IGHI), Imperial College London. This work was funded by NHS England and supported by the National Institute for Health Research (NIHR) Imperial Patient Safety Translation Research Centre. Infrastructure support was provided by the NIHR Imperial Biomedical Research Centre (BRC). JC acknowledges support from the Wellcome Trust (215938/Z/19/Z).
Data availability statement
The patient level data used in this study are not publicly available but are available to applicants meeting certain criteria through application of a Data Access Request Service (DARS) and approval from the Independent Group Advising on the Release of Data.
Ethics approval
The work was conducted as a national service evaluation of the CO@h programme, approved by Imperial College Health Trust on 3rd December 2020. Data access was approved by the Independent Group Advising on the Release of Data (IGARD; DARS-NIC-421524-R0Y3P) on 15th April 2021.