Study: Publicly reporting hospital outcomes may not improve patients care
Publicly reporting hospital quality data and outcomes does not assist Medicare beneficiaries in choosing hospital facilities that report better outcomes for high-risk surgeries, according to the results of a report published in the Oct. 19 issue of the Archives of Surgery.
Currently, the Centers for Medicare & Medicaid Services (CMS) require hospital facilities to report data from the national Surgical Care Improvement Project (SCIP), and submit it to the Hospital Compare website. “This reporting is believed to aid patients and payors in choosing high-quality hospitals and to stimulate quality improvement among reporting hospitals. It is unclear whether these efforts will translate into better outcomes for surgical patients,” the authors wrote.
To compare hospital compliance with processes of care and operative mortality rates, Lauren H. Nicholas, PhD, of the University of Michigan and Michigan Surgical Collaborative for Outcomes Research and Evaluation, in Ann Arbor, Mich., and colleagues assessed Medicare inpatient claims data for 325,052 patients undergoing one of six high-risk surgical procedures at 2,189 hospitals in 2005 or 2006.
The patients underwent abdominal aortic aneurysm repair, aortic valve repair, CABG, esophageal resection, mitral valve repair or pancreatic resection. Researchers then used the Hospital Compare data to calculate a score for each facility based on the number of times a hospital complied with recommended measures for each patient per year.
The results showed that hospital compliance scores had a wide range—from 53.7 percent in low-compliance hospitals to 91.4 percent in high-compliance hospitals. However, the researchers found that the compliance scores only accounted for 3.3 percent of the variation in hospital death rates.
Additionally, Nicholas and colleagues reported that the 95,387 hospitals that did not report data all had similar rates of death compared to the one-fifth of hospitals that reportedly had the highest compliance scores. The researchers also found that the risk-adjusted outcomes did not vary at high-compliance hospitals compared to medium-compliance hospitals for surgical site infections or venous thromboembolisms; these rates also did not vary significantly at low-compliance hospitals.
“We found little evidence of a consistent relationship between hospital compliance with processes of care and operative mortality rate,” Nicholas and colleagues wrote. “In univariate analysis, mortality rates in the lowest compliance hospitals were statistically indistinguishable from those in the highest quintile of compliance for all procedures studied except aortic valve replacement, in which the highest compliance hospitals had lower mortality rates.”
The researchers speculated that the reason reporting may not be beneficial may stem from the fact that the SCIP only reports measures that have secondary, less important outcomes. “Even when processes are tied to important outcomes such as pulmonary embolism, these events are rare and offer insufficient variation to differentiate between high- and low-quality hospitals,” the authors wrote.
They concluded: “The CMS needs to identify higher leverage process measures and devote greater attention to profiling hospitals based on outcomes for improved public reporting and pay-for-performance programs. Future research should ascertain whether process measures become more useful as indicators of surgical quality as public reporting programs mature.”
Currently, the Centers for Medicare & Medicaid Services (CMS) require hospital facilities to report data from the national Surgical Care Improvement Project (SCIP), and submit it to the Hospital Compare website. “This reporting is believed to aid patients and payors in choosing high-quality hospitals and to stimulate quality improvement among reporting hospitals. It is unclear whether these efforts will translate into better outcomes for surgical patients,” the authors wrote.
To compare hospital compliance with processes of care and operative mortality rates, Lauren H. Nicholas, PhD, of the University of Michigan and Michigan Surgical Collaborative for Outcomes Research and Evaluation, in Ann Arbor, Mich., and colleagues assessed Medicare inpatient claims data for 325,052 patients undergoing one of six high-risk surgical procedures at 2,189 hospitals in 2005 or 2006.
The patients underwent abdominal aortic aneurysm repair, aortic valve repair, CABG, esophageal resection, mitral valve repair or pancreatic resection. Researchers then used the Hospital Compare data to calculate a score for each facility based on the number of times a hospital complied with recommended measures for each patient per year.
The results showed that hospital compliance scores had a wide range—from 53.7 percent in low-compliance hospitals to 91.4 percent in high-compliance hospitals. However, the researchers found that the compliance scores only accounted for 3.3 percent of the variation in hospital death rates.
Additionally, Nicholas and colleagues reported that the 95,387 hospitals that did not report data all had similar rates of death compared to the one-fifth of hospitals that reportedly had the highest compliance scores. The researchers also found that the risk-adjusted outcomes did not vary at high-compliance hospitals compared to medium-compliance hospitals for surgical site infections or venous thromboembolisms; these rates also did not vary significantly at low-compliance hospitals.
“We found little evidence of a consistent relationship between hospital compliance with processes of care and operative mortality rate,” Nicholas and colleagues wrote. “In univariate analysis, mortality rates in the lowest compliance hospitals were statistically indistinguishable from those in the highest quintile of compliance for all procedures studied except aortic valve replacement, in which the highest compliance hospitals had lower mortality rates.”
The researchers speculated that the reason reporting may not be beneficial may stem from the fact that the SCIP only reports measures that have secondary, less important outcomes. “Even when processes are tied to important outcomes such as pulmonary embolism, these events are rare and offer insufficient variation to differentiate between high- and low-quality hospitals,” the authors wrote.
They concluded: “The CMS needs to identify higher leverage process measures and devote greater attention to profiling hospitals based on outcomes for improved public reporting and pay-for-performance programs. Future research should ascertain whether process measures become more useful as indicators of surgical quality as public reporting programs mature.”