Studies find no association between participation in surgical quality improvement program and improvement in outcomes, complications, risk of death
Participation by hospitals in the American College of Surgeons National Surgical Quality Improvement Program has not been associated with an improvement in surgical outcomes, serious complications, hospital readmissions, risk of death or lower Medicare payments, according to two studies in JAMA.
Increased scrutiny of hospital performance has led to an increase of clinical registries used to benchmark outcomes. One of the most visible national quality reporting programs is the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP). The program provides hospitals with reports that include a detailed description of their risk-adjusted outcomes (e.g., mortality, specific complications, and length of stay). These reports allow hospitals to benchmark their performance relative to all other ACS NSQIP hospitals. Participating hospitals are encouraged to focus improvement efforts on areas in which they perform poorly. The extent to which participation in ACS NSQIP improves outcomes is unclear.
In one study, Nicholas H. Osborne, M.D., M.S., of the University of Michigan, Ann Arbor, and colleagues evaluated the association of participation in the ACS NSQIP with surgical outcomes and payments among Medicare patients. The researchers used national Medicare data (2003-2012) for a total of 1,226,479 patients undergoing general and vascular surgery at 263 hospitals participating in ACS NSQIP and 526 nonparticipating (control) hospitals.
The authors found that although there were slight trends toward improved outcomes in ACS NSQIP hospitals before vs after enrollment (year 1, year 2, and year 3), there were similar trends in control hospitals, with no significant improvements in outcomes after enrollment in ACS NSQIP. For example, in analyses comparing outcomes at 3 years after (vs before) enrollment, there were no significant differences in risk-adjusted 30-day mortality (4.3 percent vs 4.5 percent), serious complications (11.1 percent vs 11.0 percent), reoperations (0.49 percent vs 0.45 percent), or readmissions (13.3 percent vs 12.8 percent).
There were also no differences at 3 years after (vs before) enrollment in average total Medicare payments, or payments for the index admission or hospital readmission.
“Enrollment in a national surgical quality reporting program was not associated with improved outcomes or lower payments among Medicare patients. Feedback of outcomes alone may not be sufficient to improve surgical outcomes,” the authors write.
In another study, David A. Etzioni, M.D., M.S.H.S., of Mayo Clinic Arizona, Phoenix, and colleagues compared rates of any complications, serious complications, and death during a hospitalization for elective general/vascular surgery at hospitals that did vs did not participate in the NSQIP.
Data from the University HealthSystem Consortium from January 2009 to July 2013 were used to identify elective hospitalizations representing a broad spectrum of elective general/vascular operations in the United States. The study group included 345,357 hospitalizations occurring in 113 different academic hospitals; 172,882 (50.1 percent) hospitalizations were in NSQIP hospitals. Hospitalized patients were predominantly female (62 percent), with an average age of 56 years. The types of procedures performed most commonly were hernia repairs (15.7 percent), bariatric (10.5 percent), mastectomy (9.7 percent), and cholecystectomy (9.0 percent).
Over the course of the study period, risk-adjusted rates of postoperative complications, serious complications, and mortality decreased for hospitalizations at both NSQIP and nonNSQIP hospitals. After accounting for patient risk, procedure type, underlying hospital performance, and temporal (transient) trends, the researchers found no significant differences over time between NSQIP and nonNSQIP hospitals in terms of likelihood of inpatient complications, serious complications, or risk of death.
The authors suggest that the “failure of this and other studies to demonstrate an association between outcomes-oriented reporting systems and improved surgical outcomes may be related to difficulties translating outcomes reports into evidence-based approaches to quality improvement.”
“This study has implications for hospitals and health care systems considering the role of programs that monitor surgical outcomes. Among hospitals providing care to patients undergoing general and vascular surgical procedures, our findings suggest that a surgical outcomes reporting system does not provide a clear mechanism for quality improvement.”
Editorial: Measuring Surgical Outcomes for Improvement
“The most likely explanation for the findings of these 2 studies is that end-results information, although necessary for improvement, is not sufficient, and that the skills necessary to make effective changes in processes and cultures do not yet pervade U.S. hospitals, to say the least. Both research groups speculate about that as a reason for their results,” writes Donald M. Berwick, M.D., M.P.P., of the Institute for Healthcare Improvement, Cambridge, Mass., in an accompanying editorial.
“ACS NSQIP and its champions and proponents should take these important studies as prompts, not to decrease investment in the careful analysis and reporting of surgical results but rather to link that information more energetically to processes of learning, skill building, and change within participating hospitals. Hospitals not enrolled in ACS NSQIP should not use these studies as an excuse to avoid measuring and investigating their own surgical results systematically, one way or another. But all hospitals should take note that measurement, alone, is not enough for improvement.”
Study 1: Association of Hospital Participation in a Quality Reporting Program With Surgical Outcomes and Expenditures for Medicare Beneficiaries, Nicholas H. Osborne, MD, MS; Lauren H. Nicholas, PhD; Andrew M. Ryan, PhD; Jyothi R. Thumma, MPH; Justin B. Dimick, MD, MPH, JAMA, doi:10.1001/jama.2015.25, published 3 February 2015.
This study was supported by a grant from the National Institute on Aging. Please see the article for additional information, including other authors, author contributions and affiliations, financial disclosures, etc.
Study 2: Association of Hospital Participation in a Surgical Outcomes Monitoring Program With Inpatient Complications and Mortality, David A. Etzioni, MD, MSHS; Nabil Wasif, MD, MPH; Amylou C. Dueck, PhD; Robert R. Cima, MD; Samuel F. Hohmann, PhD; James M. Naessens, ScD; Amit K. Mathur, MD, MS; Elizabeth B. Habermann, PhD, MPH, JAMA, doi:10.1001/jama.2015.90, published 3 February 2015.
Please see the article for additional information, including other authors, author contributions and affiliations, financial disclosures, funding and support, etc.
Editorial: Measuring Surgical Outcomes for Improvement : Was Codman Wrong? Donald M. Berwick, MD, MPP, JAMA, doi:10.1001/jama.2015.4, published 3 February 2015.
Please see the article for additional information, including financial disclosures, funding and support, etc.