Among approximately 88 million delivery hospitalizations, 15 SMM indicators identified 80% of in-hospital mortality in 1993–2015. The overall prevalence of this 15-indicator ranked SMM index increased by approximately 80% from 1993–2015. In contrast, while the prevalence of the current 20 SMM indicators, excluding blood transfusion, only increased 37% from 1993– 2015. Using predicted estimates for 2017–2019, the prevalence of this 15-indicator ranked SMM index is 54.6 per 10,000 delivery hospitalizations or approximately 22,000 SMM cases per year.

A major strength of this study is the use of NIS, a large, representative national all-payer database widely used in the US to provide generalizable estimates and trends. NIS provided a unique opportunity to determine which SMM indicators, though rare outcomes, identified the most in-hospital mortality at delivery hospitalization. Our study has several limitations. First, the SMM indicators include both diagnosis and procedure-based indicators from administrative data, for which coding rules change over time. Procedure-based indicators may be a proxy for multiple diagnosis-based indicators. For example, ventilation may be an indirect indicator of acute respiratory distress syndrome, pulmonary edema, and/or pulmonary embolism. Second, we used pre-defined conditions or procedures to ascertain SMM indicators from the hospital discharge records. The indicators identified 82% of in-hospital mortality, therefore approximately 18% is attributable to conditions not identified by these analyses. Conditions not included as indicators of severe morbidity that identify in-hospital mortality include those that clinicians encounter in the Intensive Care Unit (ICU; e.g., motor vehicle crashes, poisoning, suicides, malignancies). The NIS does not include ICU admission status for all reporting states.[16] Additionally, the SMM indicators do not capture information on comorbidities such as mental health conditions or substance use disorders which may also contribute to in-hospital mortality. Third, our study only examines in-hospital mortality at delivery hospitalization and does not look at antepartum or postpartum hospitalizations. This study does not capture the full spectrum of pregnancy related deaths that occur during pregnancy or up to one year postpartum. Fourth, we did not incorporate length of stay (LOS) in our algorithm. The SMM indicators with and without LOS considerations demonstrated excellent reliability (kappa statistic=0.927) in a study from California [21]. Hence, its inclusion may add methodological complexity and may not contribute to meaningful interpretation of the association between SMM and in-hospital mortality. Fifth, the quality of documentation by physicians, coders’ training, and ongoing initiatives may affect coding accuracy [22]. Several validity studies reported high to moderate specificity but low sensitivity for some ICD-9 codes for SMM [11–13]. Although limited studies on the validity of ICD-10 codes are available, the sensitivity of these codes may be lower than for ICD-9 codes [13], though it may vary by indicator. Sixth, we were limited in examining pulmonary edema and acute congestive heart failure separately due to interchangeable coding practices for pulmonary edema and heart failure [23]. Pulmonary edema is labeled as a left ventricular failure in the coding manual and described as a left ventricular failure or heart failure in the medical chart [23], though ambiguity of medical diagnoses like this are inherent when using administrative data sets for analyses rather than individual medical records. Finally, we combined the top 15 SMM indicators into a ranked index for further analysis because these indicators identified 80% of in-hospital mortality and the remaining indicators only identified 2%; however, this threshold of 15 was arbitrary.

Our study offers an index for SMM population-level surveillance. While there are limitations to using administrative data to identify SMM, the ranked SMM index provides a measurement that is associated with the most severe outcome, in-hospital mortality, and can be used as a marker for maternal health.

The top three ranked indicators (restoration of cardiac rhythm, cardiac arrest, mechanical ventilation), all markers of acute cardiac and respiratory compromise, identified approximately 69% of in-hospital mortality. While cardiac arrest may result from a variety of cardiovascular and non-cardiovascular conditions, cardiovascular conditions are among the leading causes of maternal death [24]. This highlights the importance of rapid response teams, checklists, huddles, and debriefs to ensure successful and prompt resuscitation [25,26].

The most prevalent indicators were the recoded severe type of hypertension of pregnancy indicator, the current blood transfusion indicator, and the recoded blood transfusion without DIC indicator; however, none of these indicators ranked among the top 15 indicators. Hypertensive disorders in pregnancy and postpartum hemorrhage are significant contributors to maternal morbidity and mortality and have increased in prevalence over the last two decades [24,27]. In our analysis, we recoded the eclampsia indicator to include severe pre-eclampsia and HELLP syndrome in order to capture a broader spectrum of hypertensive disorders in pregnancy. While the severe pre-eclampsia, HELLP syndrome, or eclampsia indicator was last on our mutually exclusive, ranked SMM index, it is possible that an SMM code indicating a consequence of severe pre-eclampsia, HELLP syndrome, or eclampsia that is more strongly associated with in-hospital mortality (e.g., a puerperal cerebrovascular disorder; mechanical ventilation) would also be present. This would produce a lower hierarchical ranking of this indicator individually. The current blood transfusion indicator and the recoded blood transfusion without DIC indicator were the second and third most prevalent indicators. Obstetric transfusions are primarily associated with postpartum hemorrhage, and we sought to capture severe cases of postpartum hemorrhage by recoding the blood transfusion and DIC indicators. The NIS data do not quantify transfusion of blood products in units which might serve as a proxy for severity [28]. However, hospital discharge records with severe blood loss resulting in hemorrhagic shock are expected to have ICD codes for blood transfusion and multiple organ failure, including DIC. Thus, most hospitalizations with severe hemorrhage in our study should be captured by indicators of organ failure. The recoded indicator for both DIC and blood transfusion was among the 15 top ranked indicators. By making indicators mutually exclusive during the ranking process, we identified how many additional SMM deaths an indicator would capture, accounting for indicators already ranked. This process does not account for the true burden of the condition which informs population-based initiatives for prevention through organizations such as perinatal quality collaboratives [29].

Our methodology allowed the ranking of rare indicators with high in-hospital mortality and requires further testing. The strength and quality of evidence generated by its use could provide the basis for further refinement of SMM indicators, increasing potential applicability. Validation studies of the indicators used for the ranked SMM index are needed and may inform index utility.

Our results have important public health and surveillance implications. While this ranked SMM index results in a lower rate and total number of SMM cases compared with estimates of SMM using the combined 21 current indicators, it directly addresses the criticism of the combined 21 current SMM indicators: a lack of specificity for disease severity [11,21]. While “severe” is difficult to define, linking diagnosis and procedure codes to in-hospital mortality underscores that selected conditions and events precede the most severe event, death. However, a ranking of indicators for the purpose of determining how many deaths could be prevented by sequentially eliminating indicators would need to use a different definition of PAF than we used (e.g., Benichou 2007) [30]; in particular, our hPAF is not appropriate for this purpose.

The ranked SMM index is for surveillance at the population level and was not developed for individual facility reviews [8]. Criteria for facility review of SMM are outlined by clinical organizations such as the American College of Obstetricians and Gynecologists and the Society for Maternal and Fetal Medicine [31]. The indicators included in the ranked SMM index could facilitate further refinement of SMM indicators for other uses (e.g., facility-based identification and review of SMM) and improve overall utility.

The ranked SMM index is unsuitable for comparison of individual health care institutions for multiple reasons. Health care institutions have access to the medical record and detailed and reliable data on comorbidities and severity, unavailable in the NIS, yielding more accurate, informative, and comparable case results. Comorbid conditions are strong predictors of overall SMM [32], but comorbid conditions are under-reported in hospital discharge data [33], with limited studies of reliability of ICD codes among hospitals [34], especially among delivery hospitalizations. The NIS data lack information on prenatal care and antepartum risk factors that are important determinants of SMM [35]. These factors are not considered in this ranked SMM index to adjust for differences in case-mix among institutions. Facilities with higher levels of maternal care are expected to have more high-risk deliveries and may have higher SMM rates compared with lower-level facilities [32]. Our approach of creating mutually exclusive indicators through iterative ranking may not be appropriate for care management goals in individual health care institutions. For instance, the risk of in-hospital death and thus, the condition severity and complexity of clinical management, increases with the growing number of SMM indicators [36]. Clinical reviews allow identification of preventable causes of SMM including determinants of clinical practices associated with SMM [31]. Only clinical reviews can guide design of hospital-level interventions and quality improvement initiatives to reduce SMM. A recent study of ICD-10 coding for blood transfusion, DIC, and acute renal failure confirmed wide variability of code sensitivity among hospitals in administrative discharge data and cautioned against using this type of data to compare inter-hospital safety and quality improvement initiatives at the facility level [13].