A large study of intensive care patients in California found that public reporting of patient outcomes did not reduce mortality, but did result in reduced admission of the sickest patients to the ICU and increased transfer of critically ill patients to other hospitals.
“Public reporting is designed to reduce mortality by steering patients towards high-quality hospitals and creating incentives for hospitals to adopt quality improvement programs,” said Lora Reineck, MD, a postdoctoral scholar at the University of Pittsburgh. “But the reality does not necessarily meet the expectation.”
In fact, said Reineck, the study demonstrated that large policy initiatives like California’s can have unintended consequences. Whether those consequences left patients worse or better off is not a question the study was designed to answer.
“It could be that some hospitals didn’t take certain surgical cases, fearing that the patients were at high risk of dying in intensive care after surgery,” Dr. Reineck explained. “Or, it could be that the very sickest patients were not admitted to the ICU because they were not going to get better, and instead were transitioned to care that emphasized comfort rather than prolonging life.”
In their retrospective cohort study, presented at the 2014 American Thoracic Society International Conference, Reineck and Pitt colleagues took advantage of a natural experiment created in 2007 when California required every ICU in the state to report severity-adjusted mortality rates. (California has since stopped collecting and reporting this information, and no other state has launched a similar initiative.)
As a control, the Pitt researchers looked at Arizona, Nevada, and Texas, which did not have a public report requirement. In all four states, the group gathered Medicare fee-for-service data from 2005 to 2009, two years before California initiated its program and continuing two years after the program was launched.
In all, they reviewed the records of 936,063 patients admitted to 646 hospitals. To determine the effect of public reporting, they compared changes over time in California ICUs to control ICUs. The researchers analyzed admission patterns in each ICU by determining the proportion of patients with 3 or greater comorbidities and the proportion of patients needing mechanical ventilation in each ICU. They reviewed discharge patterns by examining where ICU patients were sent when they left the hospital. And they assessed mortality rates, including deaths that occurred in the hospital and those that occurred within 30 days of hospital admission.
In their analysis, the team adjusted for patient and hospital characteristics, as well as regional characteristics that could affect patient outcomes. These regional factors included the age and racial distribution of residents and the number of hospitals in the region, a statistic that could affect how often critically ill patients were transferred to another ICU. In the first year after public reporting began, California ICUs experienced virtually identical changes in in-hospital and 30-day mortality rates as controls (OR 1, p=0. 97; OR 1, p=0 .84, respectively).
However, in the first year, both the admission and discharge patterns showed statistically significant changes. Patients with 3 or more comorbidities were less likely to be admitted to California ICUs than control ICUs (OR .98, p=0.03). They were also more likely to be transferred to another acute care facility (OR 1.08, p=0.03).
In the second year of pubic reporting, these patterns continued. Changes in mortality rates in California were similar to controls (in-hospital mortality OR 0.99, p = 0.72; 30-day mortality OR 0.99, p = 0.55). Admission of patients with 3 or more comorbidities was reduced compared to controls, but this was not statistically significant (OR 0.98, p=0. 13). And critically ill patients were more likely to be transferred to another hospital than in control states – a pattern more pronounced in year two than in year one (OR 1.43, p<0.001).
The Pitt researchers also looked at the number of patients who were transitioned to skilled nursing homes or long-term care facilities. They hypothesized that hospitals would try to improve their mortality rates by sending patients to these facilities.
“We expected that the proportion of patients discharged to these post-acute care facilities would have increased, but it didn’t,” Reineck said.
Reineck noted that a similar, but much smaller, study conducted in Cleveland, OH, supported their original hypothesis. The study, which analyzed the effect of publicly reporting ICU in-hospital mortality rates during the 1990s, found that discharges to post-acute care facilities increased while in-hospital mortality decreased. (Sirio CA, Shepardson LB, Rotondi AJ, et al. Community-wide assessment of intensive care outcomes using a physiologically based prognostic measure: implications for critical care delivery from Cleveland Health Quality Choice. Chest 1999; 115:793-801.) Reineck also noted that initial media stories in California erroneously credited public reporting of ICU mortality rates with reducing mortality.
“That turned out not to be true,” she said. “We determined that mortality rates decreased in the control states at the same rate as California, meaning that the quality of care of all ICU patients improved during the study period irrespective of the public reporting initiative in California.”
* Please note that numbers in this release may differ slightly from those in the abstract. Many of these investigations are ongoing; the release represents the most up-to-date data available at press time.
The Impact Of Publicly Reporting Intensive Care Unit (ICU) In-Hospital Mortality On ICU Case-Mix And Outcomes Type: Scientific Abstract
Category: 04.07 – ICU Management/Outcome (CC)
Authors: L.A. Reineck, T.Q. Le, C.W. Seymour, A.E. Barnato, D.C. Angus, J.M. Kahn; University of Pittsburgh – Pittsburgh, PA/US
Rationale: Public reporting of performance measures is a common strategy to accelerate quality improvement in health care. However, these programs may carry unintended consequences. We examined whether California’s recent public reporting program of hospitals’ severity-adjusted mortality rates for intensive care unit (ICU) patients changed ICU admission patterns, discharge patterns, and mortality.
Methods: We performed a retrospective cohort study leveraging a natural experiment in which California, but not other states, publicly reported each hospital’s severity-adjusted mortality rates between 2007-2012. We used Medicare fee-for-service data from the 2 years before and 2 years after the introduction of public reporting in California and three control states (Arizona, Nevada, and Texas). We assessed admission patterns by measuring the proportion of patients with three or greater comorbidities and the proportion of patients who underwent mechanical ventilation, both measures of ICU case-mix. We assessed discharge patterns using post-acute care utilization rates and acute care hospital transfer rates. We assessed mortality using in-hospital mortality rates and 30-day mortality rates (to account for death regardless of transfer). We used multivariable logistic regression to compare the case-mix and outcomes of ICU patients admitted in California versus control states over time adjusting for key patient, hospital, and regional characteristics using a differences-in-differences approach.
Results: The study cohort included 936,063 patients admitted to 646 hospitals. Compared to admission to a non-California ICU, admission to a California ICU after the initiation of public reporting was associated with reduced odds of having ? 3 comorbidities, reduced odds of post-acute care utilization, and increased odds of transfer to another acute care hospital (Table). There were no significant differences in mortality, either in the hospital or at 30 days.
Outcome Measure Post-reform Year 1 OR p-value Post-reform Year 2 OR p-value
? 3 Comorbidities 0.98 0.03 0.98 0.13
Use of Mechanical Ventilation 1.01 0.54 0.99 0.59
Post-acute Care Utilization 0.98 0.15 0.94 <0.001
Transfer to Another Acute Care Hospital 1.08 0.03 1.43 <0.001
In-hospital Mortality 1.00 0.95 0.99 0.72
30-day Mortality 1.00 0.86 0.99 0.55
Conclusion: Public reporting of hospital’s ICU risk-adjusted mortality was associated with lower admission rates for patients with greater comorbidity and increased transfers to other hospitals. Despite differences in case-mix and discharge patterns, there was no change in risk-adjusted mortality. These data suggest that public reporting of ICU outcomes changes ICU admission and discharge practices without improving critical care quality.