In just a few weeks, the whole world has been upended by the outbreak of COVID-19, an acute respiratory disease caused by a new coronavirus called SARS-CoV-2. The virus is highly contagious: it is easily transmitted from person to person via respiratory droplet nuclei and can persist on surfaces for days (1, 2). As a result, COVID-19 has spread rapidly—classified by the World Health Organization as a public health emergency on January 30, 2020 and as a pandemic on March 11. As of mid-May, over 4.5 million cases and 300,000 deaths have beenreported globally (3).

Because no treatment is currently available, healthcare providers and policy makers are wrestling with unprecedented challenges. Hospitals and other care facilities are facing shortages of beds, ventilators and personal protective equipment— raising hard questions on how to treat COVID-19 patients with scarce supplies and how to allocate resources to prevent further shortages. At the policy level, most countries have imposed “social distancing” measures to slow the spread of the pandemic. These measures allow strained healthcare systems to cope with the disease by “flattening the curve” (4) but also come at a steep economic price (5, 6). Nearly all governments are now confronted to difficult decisions balancing public health and socio-economic outcomes.

This paper proposes a comprehensive data-driven approach to understand the clinical characteristics of COVID-19, predict its mortality, forecast its evolution, and ultimately alleviate its impact. We leverage a broad range of data sources, which include (i) our own cohort-level data aggregating hundreds of clinical studies, (ii) patient-level data obtained from electronic health records, and (iii) census reports on the scale of the pandemic. We develop an integrated approach spanning descriptive analytics (to derive a macroscopic understanding of the disease), predictive analytics (to forecast the near-term impact and longer-term dynamics of the pandemic), and prescriptive analytics (to support healthcare and policy decision-making).

Specifically, our approach comprises four steps (Figure 1):

• Aggregating and visualizing the most comprehensive clinical database on COVID-19 (Section 1). We aggregate cohort-level data on demographics, comorbidities, symptoms and lab values from 160 clinical studies. These data paint a broad picture of the disease, identifying common symptoms, disparities between mild and severe patients, and geographic disparities—insights that are hard to derive from any single study and can orient future clinical research on COVID-19, its mutations, and its disparate effects across ethnic groups.
• Providing personalized indicators to assess the risk of mortality and infection (Section 2). Using patient-level data, we develop machine learning models to predict mortality and infection risk, as a function of demographics, symptoms, comorbidities, and lab values. Using gradient boosting methods, the models achieve strong predictive performance—with an out-of-sample area under the curve above 90%. These models yield personalized calculators that can (i) guide triage, treatment, and care management decisions for strained healthcare systems, and (ii) serve as pre-screening tools for patients before they visit healthcare or testing facilities.
• Developing a novel epidemiological model to forecast the evolution of the disease and assess the effects of social distancing (Section 3). We propose a new compartmental model called DELPHI, which accounts for COVID-19 features such as underdetection and government response. The model estimates the disease’s spread with high accuracy; notably, its projections from as early as April 3 have matched the number of cases observed in the United States up to mid-May. We also provide a data-driven assessment of social distancing policies, showing that the pandemic’s spread is highly sensitive to the stringency and timing of mitigating measures.
• Proposing an optimization model to support ventilator allocation in response to the pandemic (Section 4). We formulate a mixed-integer optimization model to allocate ventilators efficiently in a semi-collaborative setting where resources can be shared both between healthcare facilities or through a central authority. In the United States, this allows us to study the trade-offs of managing the federal ventilator stockpile in conjunction with inter-state transfers. Results show that limited ventilator transfers could have eliminated shortages in April 2020.

A major contribution of our work is to treat these different questions as interdependent challenges raised by the pandemic—as opposed to a series of isolated problems. Indeed, clinical decision-making depends directly on patient inflows and available supplies, while resource planning and government responses react to patient-level outcomes. By combining various data sources into descriptive, predictive and prescriptive methods, this paper proposes an end-to-end approach to design a comprehensive and cohesive response to COVID-19.

Ultimately, this paper develops analytical tools to inform clinical and policy responses to the COVID-19 pandemic. These tools are available to the public on a dedicated website.^* They have also been deployed in practice to combat the spread of COVID-19 globally. Several hospitals in Europe have used our risk calculators to support pre-triage and post-triage decisions, and a major financial institution in South America is applying our infection risk calculator to determine how employees can safely return to work. A major hospital system in the United States planned its intensive care unit (ICU) capacity based on our forecasts, and leveraged our optimization results to allocate ventilators across hospitals when the number of cases was rising. Our epidemiological predictions are used by a major pharmaceutical company to design a vaccine distribution strategy that can contain future phases of the pandemic. They have also been incorporated into the US Center for Disease Control’s forecasts (7).

1.Descriptive Analytics: Clinical Outcomes Database

Early responses to the COVID-19 pandemic have been inhibited by the lack of available data on patient outcomes. Individual centers released reports summarizing patient characteristics. Yet, this decentralized effort makes it difficult to construct a cohesive picture of the pandemic.

To address this problem, we construct a database that aggregates demographics, comorbidities, symptoms, laboratory blood test results (“lab values”, henceforth) and clinical outcomes from 160 clinical studies released between December 2019 and May 2020—made available on our website for broader use. The database contains information on 133,600 COVID-19 patients (3.13% of the global COVID-19 patients as of May 12, 2020), spanning mainly Europe (81, 207 patients), Asia (19, 418 patients) and North America (23, 279 patients). To our knowledge, this is the largest dataset on COVID-19.

A.Data Aggregation

Each study was read by an MIT researcher, who transcribed numerical data from the manuscript. The appendix reports the main transcription assumptions.

Each row in the database corresponds to a cohort of patients—some papers study a single cohort, whereas others study several cohorts or sub-cohorts. Each column reports cohort-level statistics on demographics (e.g., average age, gender breakdown), comorbidities (e.g., prevalence of diabetes, hypertension), symptoms (e.g., prevalence of fever, cough), treatments (e.g., prevalence of antibiotics, intubation), lab values (e.g., average lymphocyte count), and clinical outcomes (e.g., average hospital length of stay, mortality rate). We also track whether the cohort comprises “mild” or “severe” patients (mild and severe cohorts are only a subset of the data).

Due to the pandemic’s urgency, many papers were published before all patients in a cohort were discharged or deceased. Accordingly, we estimate the mortality rate from discharged and deceased patients only (referred to as “Projected Mortality”).

B.Objectives

Our main goal is to leverage this database to derive a macroscopic understanding of the disease. We break it down into the following questions:

• Which symptoms are most prevalent?
• How do “mild” and “severe” patients differ in terms of symptoms, comorbidities, and lab values?
• Can we identify epidemiological differences in different parts of the world?

C.Descriptive Statistics

Table 1 depicts the prevalence of COVID-19 symptoms, in aggregate, classified into “mild” or “severe” patients, and classified per geographic region. Our key observations are that:

• Cough, fever, shortness of breath, and fatigue are the most prevalent symptoms of COVID-19.
• COVID-19 symptoms are much more diverse than those listed by public health agencies. COVID-19 patients can experience at least 15 different symptoms. In contrast, the US Center for Disease Control and Prevention lists seven symptoms (cough, shortness of breath, fever, chills, myalgia, sore throat, and loss of taste/smell) (8); the World Health Organization lists three symptoms (fever, cough, and fatigue) (9); and the UK National Health Service lists two main symptoms (fever and cough) (10). This suggests a lack of consensus among the medical community, and opportunities to revisit public health guidelines to capture the breadth of observed symptoms.
• Shortness of breath and elevated respiratory rates are much more prevalent in cases diagnosed as severe.
• Symptoms are quite different in Asia vs. Europe or North America. In particular, more than 75% of Asian patients experience fever, as compared to less than half in Europe and North America. Alternatively, shortness of breath is much more prevalent in Europe and North America.

Using a similar nomenclature, Figure 2A reports demographics, comorbidities, lab values, and clinical outcomes (an extended version is available in the appendix). In terms of demographics, severe populations of patients have a higher incidence of male subjects and are older on average. Severe patients also have elevated comorbidity rates. Figures 2B and 2C visually confirm the impact of age and hypertension rates on population-level mortality—consistently with (11–13). In terms of lab values, CRP, AST, BUN, IL-6 and Protocalcitonin are highly elevated among severe patients.

D.Discussion and Impact

Our database is the largest available source of clinical information on COVID-19 assembled to date. As such, it provides new insights on common symptoms and the drivers of the disease’s severity. Ultimately, this database can support guidelines from health organizations, and contribute to ongoing clinical research on the disease.

Another benefit of this database is its geographical reach. Results highlight disparities in patients’ symptoms across regions. These disparities may stem from (i) different reporting criteria; (ii) different treatments; (iii) disparate impacts across different ethnic groups; and (iv) mutations of the virus since it first appeared in China. This information contributes to early evidence on COVID-19 mutations (14, 15) and on its disparate effects on different ethnic groups (16, 17).

Finally, the database provides average values of key parameters into our epidemiological model of the disease’s spread and our optimization model of resource allocation (e.g., average length of stay of hospitalizations, average fraction of hospitalized patients put on a ventilator).

The insights derived from this descriptive analysis highlight the need for personalized data-driven clinical indicators. Yet, our population-level database cannot be leveraged directly to support decision-making at the patient level. We have therefore initiated a multi-institution collaboration to collect electronic medical records from COVID-19 patients and develop clinical risk calculators. These calculators, presented in the next section, are informed by several of our descriptive insights. Notably, the disparities between severe patients and the rest of the patient population inform the choice of the features included in our mortality risk calculator. Moreover, the geographic disparities suggest that data from Asia may be less predictive when building infection or mortality risk calculators designed for patients in Europe or North America—motivating our use of data from Europe.

B.Results

Study Population

The mortality study population comprises 2,831 patients, 711 (25.1%) of whom died during hospitalization while the remaining ones were discharged. The infection study population comprises 3,135 patients, 1,661 (53.0%) of whom tested positive for COVID-19. The full distributions of patient characteristics are reported in the appendix.

Performance Evaluation

All models achieve strong out-of-sample performance. Our mortality risk calculator has an AUC of 93.8% with lab values and 90.5% without lab values. Our infection risk calculator has an AUC of 91.8% with lab values and 83.1% without lab values. These values suggest a strong discriminative ability of the proposed models. We report in the appendix average results across all random data partitions.

We also report in the appendix threshold-based metrics, which evaluate the discriminative ability of the calculators at a fixed cutoff. Using cutoff to ensure a sensitivity of at least 90% (motivated by the high costs of false negatives), we obtain an accuracy spanning 65%–80%.

The mortality model achieves better overall predictive performance than the infection model. As expected, both models have better predictive performance with lab values than without lab values. Yet, the models without lab values still achieve strong predictive performance.

Model Interpretation

Figure 3 plots the SHAP importance plots for all models. The figures sort the features by decreasing significance. For each one, the row represents its impact on the SHAP value, as the feature ranges from low (blue) to high (red). Higher SHAP values correspond to increased likelihood of a positive outcome (i.e. mortality or infection). Features with the color scale oriented blue to red (resp. red to blue) from left to right have increasing (resp. decreasing) risk as the feature increases. For example, “Age” is the most important feature of the mortality score with lab values (Figure 3A), and older patients have higher predicted mortality.

3.Predictive and Prescriptive Analytics: Disease Projections and Government Response

We develop a new epidemiological model, called DELPHI (Differential Equations Leads to Predictions of Hospitalizations and Infections). The model first provides a predictive tool to forecast the number of detected cases, hospitalizations and deaths—we refer to this model as “DELPHI-pred”. It then provides a prescriptive tool to simulate the effect of policy interventions and guide government response to the COVID-19 pandemic—we refer to this model as “DELPHI-presc”. All models are fit in each US state (plus the District of Columbia).

A.DELPHI-pred: Projecting Early Spread of COVID-19

A.1.Model Development

DELPHI is a compartmental model, with dynamics governed by ordinary differential equations. It extends the standard SEIR model by defining 11 states (Figure 4A): susceptible (S), exposed (E), infectious (I), undetected people who will recover (U_R) or decease (U_D), detected hospitalized people who will recover (DH_R) or decease (DH_D), quarantined people who will recover (DQ_R) or decease (DQ_D), recovered (R) and deceased (D). The separation of the U_R/U_D, DQ_R/DQ_D and DH_R/DH_D states enables separate fitting of recoveries and deaths from the data.

As opposed to other COVID-19 models (see, e.g., 33), DEL-PHI captures two key elements of the pandemic:

• Underdetection: Many cases remain undetected due to limited testing, record failures, and detection errors. Ignoring them would underestimate the scale of the pandemic. We capture them through the U_R and U_D states.
• Government Response: “Social distancing” policies limit the spread of the virus. Ignoring them would over-estimate the spread of the pandemic. We model them through a decline in the infection rate over time. Specifically, we write: , where – is a constant baseline rate and “(t) is a time-dependent function characterizing each state’s policies, modeled as follows:

The inverse tangent function provides a concave-convex relationship, capturing three phases of government response. In Phase I, most activities continue normally as people adjust their behavior. In Phase II, the infection rate declines sharply as policies are implemented. In Phase III, the decline in the infection rate reaches saturation. The parameters t₀ and k can be respectively thought of as the start date and the strength of the response.

Ultimately, DELPHI involves 13 parameters that define the transition rates between the 11 states. We calibrate six of them from our clinical outcomes database (Section 1). Using non-linear optimization, we estimate seven parameters for each US state from the data to minimize in-sample error. This training procedure leverages historical data on the number of cases and deaths per US county (34). We include each state as soon as it records more than 100 cases. We provide details on the fitting procedure in the appendix.

A.2.Validation

DELPHI was created in late March and has been continuously updated to reflect new observed data. Figure 4B shows our projections made on three different dates, and compares them against historical observations. This plot focuses on the number of cases, but a similar plot for the number of deaths is reported in the appendix.

In addition to providing aggregate validation figures, we also evaluate the model’s out-of-sample performance quantitatively, using a backtesting procedure. To our knowledge, this represents the first attempt to assess the predictive performance of COVID-19 projections. Specifically, we fit the model’s parameters using data up to April 27, build projections from April 28 to May 12, and evaluate the resulting Mean Absolute Percentage Error (MAPE). Figure 4C reports the results in each US state.

A.3.Discussion and Impact

Results suggest that DELPHI-pred achieves strong predictive performance. The model has been consistently predicting, with high accuracy the overall spread of the disease for several weeks. Notably, DELPHI-pred was able to anticipate, as early as April 3rd, the dynamics of the pandemic in the United States up to mid-May. At a time where 200,000–300,000 cases were reported, the model was predicting 1.2M–1.4M cases by mid-May—a prediction that became accurate 40 days later.

Our quantitative results confirm the visual evidence. The MAPE is small across US states. The median MAPE is 8.5% for the number of cases—the 10% and 90% percentiles are equal to 1.9% and 16.7%. The median MAPE is 7.8% for the number of deaths—the 10% and 90% percentiles are equal to 3.3% and 25.1%. Given the high level of uncertainty and variability in the disease’s spread, this level of accuracy is suggestive of excellent out-of-sample performance.

As Figure 4C shows, a limitation of our model is that the relative error remains large for a small minority of US states. These discrepancies stem from two main reasons. First, errors are typically larger for states that have recorded few cases (WY) or few deaths (AK, KS, NE). Like all SEIR-derived models, DELPHI performs better on large populations. Moreover, the MAPE metric emphasizes errors on smaller population counts. Second, our model is fitted at the state level, implicitly assuming that the spread of the pandemic is independent from one state to another—thus ignoring inter-state travel. This limitation helps explain the above-median error in a few heartland states which were confronted to the pandemic in later stages (MN, TN, IA).

In summary, DELPHI-pred is a novel epidemiological model of the pandemic, which provides high-quality estimates of the daily number of cases and deaths per US state. This model has been incorporated to the forecasts used by the US Center for Disease Control to chart and anticipate the spread of the pandemic (7). It has also been used by the Hartford HealthCare system—the major hospital system in Connecticut, US—to plan its ICU capacity, and by a major pharmaceutical company to design a vaccine distribution strategy that can most effectively contain the next phases of the pandemic.

B.DELPHI-presc: Toward Re-opening Society

To inform the relaxation of social distancing policies, we link policies to the infection rate using machine learning. Specifically, we predict the values of “(t), obtained from the fitting procedure of DELPHI-pred. For simplicity and interpretability, we consider a simple model based on regression trees (35) and restrict the independent variables to the policies in place. We classify policies based on whether they restrict mass gatherings, school and/or other activities (referred to as “Others”, and including business closures, severe travel limitations and/or closing of non-essential services). We define a set of seven mutually exclusive and collectively exhaustive policies observed in the US data: (i) No measure; (ii) Restrict mass gatherings; (iii) Restrict others; (iv) Authorize schools, restrict mass gatherings and others; (v) Restrict mass gatherings and schools; (vi) Restrict mass gatherings, schools and others; and (vii) Stay-at-home.

We report the regression tree in the appendix, obtained from state-level data in the United States. This model achieves an out-of-sample R² of 0.8, suggesting a good fit to the data. As expected, more stringent policies lead to lower values of “(t). The results also provide comparisons between various policies—for instance, school closures seem to induce a stronger reduction in the infection rate than restricting “other” activities. More importantly, the model quantifies the impact of each policy on the infection rate. We then use these results to predict the value of “(t) as a function of the policies (see appendix for details), and simulate the spread of the disease as states progressively loosen social distancing policies.

Figure 4D plots the projected case count in the State of New York (NY), for different policies (we report a similar plot for the death count in the appendix). Note that the stringency of the policies has a significant impact on the pandemic’s spread and ultimate toll. For instance, relaxing all social distancing policies on May 12 can increase the cumulative number of cases in NY by up to 25% by September.

Using a similar nomenclature, Figure 4E shows the case count if all social distancing policies are relaxed on May 12 vs. May 26. Note that the timing of the policies also has a strong impact: a two-week delay in re-opening society can greatly reduce a resurgence in NY.

The road back to a new normal is not straightforward: results suggest that the disease’s spread is highly sensitive to both the intensity and the timing of social distancing policies. As governments grapple with an evolving pandemic, DELPHI-presc can be a useful tool to explore alternative scenarios and ensure that critical decisions are supported with data.

4.Prescriptive Analytics: Ventilator Allocation

COVID-19 is primarily an acute respiratory disease. The World Health Organization recommends that patients with oxygen saturation levels below 93% receive respiratory support (9). Following the standard Acute Respiratory Distress Syndrome protocol, COVID-19 patients are initially put in the prone position and then put in a drug induced paralysis via a neuromuscular blockade to prevent lung injury (36). Patients are then put on a ventilator, which delivers high concentrations of oxygen while removing carbon dioxide (37). Early evidence suggests that ventilator intubation reduces the risk of hypoxia for COVID-19 patients (38).

As a result, hospitals have been facing ventilator shortages worldwide (39). Still, local shortages do not necessarily imply global shortages. For instance, in April 2020, the total supply of ventilators in the United States exceeded the projected demand from COVID-19 patients. Ventilator shortages could thus be alleviated by pooling the supply, i.e., by strategically allocating the surge supply of ventilators from the federal government and facilitating inter-state transfers of ventilators.

We propose an optimization model to support the allocation of ventilators in a semi-collaborative setting where resources can be shared both between healthcare facilities or through a central authority. Based on its primary motivation, we formulate the model to support the management of the federal supply of ventilators and inter-state ventilator transfers in the United States. A similar model has also been used to support inter-hospital transfers of ventilators. The model can also support inter-country ventilator allocation during the next phases of the pandemic. This model leverages the demand projections from DELPHI-pred (Section 3) to prescribe resource allocation recommendations—with the ultimate goal of alleviating the health impact of the pandemic.

A.Model

Resource allocation is critical when clinical care depends on scarce equipment. Several studies have used optimization to support ventilator pooling. A time-independent model was first developed for influenza planning (40). A time-dependent stochastic optimization model was developed to support transfers to and from the federal government for COVID-19, given scenarios regarding the pandemic’s spread (41). In this section, we propose a deterministic time-dependent model, leveraging the projections from DELPHI-pred.

We model ventilator pooling as a multi-period resource allocation over S states and D days. The model takes as input ventilator demand in state s and day d, denoted as v_s,d, as well as parameters capturing the surge supply from the federal government and the extent of inter-state collaboration. We formulate an optimization problem that decides on the number of ventilators transferred from state s to state s^′ on day d, and on the number of ventilators allocated from the federal government to state s on day d. We propose a bi-objective formulation. The first objective is to minimize ventilator-day shortages; for robustness, we consider both projected shortages (based on demand forecasts) and worst-case shortages (including a buffer in the demand estimates). The second objective is to minimize inter-state transfers, to limit the operational and political costs of inter-state coordination. Mixed-integer optimization provides modeling flexibility to capture spatial-temporal dynamics and the trade-offs between these various objectives. We report the mathematical formulation of the model, along with the key assumptions, in the appendix.

B.Results

We implemented the model on April 15, a time of pressing ventilator need in the United States. We estimate the number of hospitalizations from DELPHI-pred as the sum of DH_R and DH_D. From our clinical outcomes database in Section 1, we estimate that 25% of hospitalized patients are put on a ventilator, which we use to estimate the demand for ventilators. We also obtain the average length of stay from our clinical outcomes database (Figure 2).

Figure 5A shows the evolution of ventilator shortages with and without ventilator transfers from the federal government and inter-state transfers. These results indicate that ventilator pooling can rapidly eliminate all ventilator shortages. Figure 5C shows ventilator transfers recommended in the US North-east on April 15 (with inter-state transfers only), overlaid on a map displaying the predicted shortage without transfers.

There are different pathways toward eliminating ventilator shortages. Figure 5B shows the trade-off between shortages and transfer distance—each line corresponds to the maximal fraction of its own ventilators that each state can pool. Overall, states do not have to share more than 10% of their supply at any time to efficiently eliminate shortages. States can largely meet their needs with help from neighboring states, with cross-country transfers only used as a last resort. Broadly, results underscore trade-offs between ventilator shortages, the extent of inter-state transfers, the number of ventilators allocated from the federal government, and the robustness of the solution. We discuss these trade-offs further in the appendix.

C.Discussion and Impact

Our main insight is that ventilator shortages could be eliminated altogether through inter-state transfers and strategic management of the federal supply. Results also underscore (i) the benefits of inter-state coordination and (ii) the benefits of early coordination. First, ventilator shortages can be eliminated through inter-state transfers alone: leveraging a surge supply from the federal government is not required, though it may reduce inter-state transfers. Under our recommendation, the most pronounced transfers occur from states facing no shortages (Ohio, Pennsylvania, and North Carolina) to states facing strong shortages (New York, New Jersey). Second, most transfers occur in early stages of the pandemic. This underscores the benefits of leveraging a predictive model like DELPHI-pred to align the ventilator supply with demand projections as early as possible.

A similar model has been developed to support the redistribution of ventilators across hospitals within the Hartford HealthCare system in Connecticut—using county-level forecasts of ventilator demand obtained from DELPHI-pred. This model has been used by a collection of hospitals in the United States to align ventilator supply with projected demand at a time where the pandemic was on the rise.

Looking ahead, the proposed model can support the allocation of critical resources in the next phases of the pandemic— spanning ventilators, medicines, personal protective equipment etc. Since epidemics do not peak in each state at the same time, states whose infection peak has already passed or lies weeks ahead can help other states facing immediate shortages at little costs to their constituents. Inter-state transfers of ventilators occurred in isolated fashion through April 2020; our model proposes an automated decision-making tool to support these decisions systematically. As our results show, proactive coordination and resource pooling can significantly reduce shortages—thus increasing the number of patients that can be treated without resorting to extreme clinical recourse with side effects (such as splitting ventilators).