Jacob V. Spertus, Sharon-Lise T. Normand, Robert Wolf, Matt Cioffi, Ann Lovett and Sherri Rose
Circulation: Cardiovascular Quality and Outcomes. 2016;CIRCOUTCOMES.116.002826
Originally published online November 8, 2016.
Although risk adjustment remains a cornerstone for comparing outcomes across hospitals, optimal strategies continue to evolve in the presence of many confounders. We compared conventional regression-based model to approaches particularly suited to leveraging big data.
We assessed hospital all-cause 30-day excess mortality risk among 8952 adults undergoing percutaneous coronary intervention between October 1, 2011, and September 30, 2012, in 24 Massachusetts hospitals using clinical registry data linked with billing data. We compared conventional logistic regression models with augmented inverse probability weighted estimators and targeted maximum likelihood estimators to generate more efficient and unbiased estimates of hospital effects. We also compared a clinically informed and a machine-learning approach to confounder selection, using elastic net penalized regression in the latter case. Hospital excess risk estimates range from −1.4% to 2.0% across methods and confounder sets. Some hospitals were consistently classified as low or as high excess mortality outliers; others changed classification depending on the method and confounder set used. Switching from the clinically selected list of 11 confounders to a full set of 225 confounders increased the estimation uncertainty by an average of 62% across methods as measured by confidence interval length. Agreement among methods ranged from fair, with a κ statistic of 0.39 (SE: 0.16), to perfect, with a κ of 1 (SE: 0.0).
Of 10,301 patients in the surgical cohort, 804 (7.8%) incident cancers developed over 5 years of follow-up, whereas 245.7 incident cancers were expected. This yielded an SMR of 3.27 (95% CI, 3.05-3.51; P<0.0001). In the PCI cohort comprising 13,001 patients, 1029 (7.9%) incident cancers developed over 5 years, resulting in an SMR of 3.53 (95% CI, 3.32-3.75; P<0.0001). Excluding respiratory cancers from the analysis (to limit smoking-related cancers) reduced risk estimates only slightly. For the surgical cohort: SMR=2.80; 95% CI, 2.59-3.01; P<0.0001. For the PCI cohort: SMR=2.97; 95% CI, 2.78-3.18; P<0.0001.
Modern causal inference techniques should be more frequently adopted to leverage big data while minimizing bias in hospital performance assessments.