Agency for Healthcare Research and Quality Patient Safety Indicators and Mortality in Surgical Patients

Patient safety indicators (PSI), developed by the Agency for Healthcare Research and Quality, use administrative billing data to measure and compare patient safety events at medical centers. We retrospectively examined whether PSIs accurately reflect patients' risk of mortality, hospital length of stay, and intensive care unit (ICU) requirements at an academic medical center. Surgical patient records with PSIs were reviewed between October 2011 and September 2012 at our urban academic medical center. Primary outcomes studied included mortality, hospital length of stay, and ICU requirements. Subset analysis was performed for each PSI and its association with the outcome measures. PSIs were more common among surgical patients who died as compared with those alive at discharge (35.3 vs 2.7 PSIs/100 patients, P < 0.01). Although patients who died with PSIs had shorter hospital courses, they had a significantly greater ICU requirement than those without a PSI (96.0 vs 61.1%, P < 0.01) and patients who were alive at discharge (96.0 vs 48.0%, P < 0.01). The most frequently associated PSIs with mortality were postoperative metabolic derangements (41.7%), postoperative sepsis (38.5%), and pressure ulcers (33.3%). PSIs occur at a higher frequency in surgical patients who die and are associated with increased ICU requirements.

PATIENT SAFETY INDICES are generating increasing attention as patient safety becomes a central component in hospital performance benchmarking, comparison, and reimbursement. The federal Agency for Healthcare Research and Quality (AHRQ), in concert with the Centers for Medicare and Medicaid Services (CMS), is increasingly using the system of patient safety indicators (PSI) to track and compare patient safety performance.1-3 The AHRQ PSIs use administrative International Classifications of Dis- eases, 9th Revision, Clinical Modification (ICD-9-CM) billing codes generated at the time of patient discharge to retrospectively measure patient safety events that occur in the course of a patient's hospitalization. Spe- cific combinations of ICD-9-CM billing codes are used as PSIs for 20 safety events (Fig. 1).

To pilot this system, 113 academic medical centers continue to participate in the University Health System Consortium (UHC) and submit their billing codes for AHRQ PSI analysis. These data are used to generate reports that indicate how a particular academic center performs in comparison to national means. Since its inception, numerous studies have demonstrated that the AHRQ PSI system suffers from poor predictive value and sensitivity, largely because it is a purely administrative data collection tool that relies heavily on coding performed for purposes unrelated to patient safety analysis.4-16

As use and endorsement of the PSIs by government agencies and payers grow, it is necessary to understand what the clinical relevance of PSIs is to patients in our healthcare systems. In this study, we investigate the association between PSIs and patient morbidity and mortality.

Methods

At our urban academic medical center, we analyzed surgical patients hospitalized between October 2011 and September 2012 for the presence of PSIs or mor- tality. Among these groups, we further investigated outcomes including hospital length of stay (HLOS), need for intensive care unit (ICU) admission, and ICU length of stay (ICU LOS).

Surgical patients were defined as those admitted to, discharged from, or operated on by a surgical service. The 10 surgically related PSIs reviewed in this study are starred in Figure 1 and were queried from the UHC database. Standard definitions of PSIs with AHRQ software Version 3.1 were used.1

The study was approved by our Institutional Review Board.

Results

During the study period, there were 9879 surgical patients and 358 PSIs (3.6 PSI/100 patients). The most common PSIs were postoperative pulmonary embolus or deep vein thrombosis (24.8%), accidental puncture or laceration (20.9%), and postoperative respiratory failure (15.6%). Over one-fourth (26.1%) of patients who died during their hospitalization had at least one PSI at a rate of 1.35 PSI/patient. Compared with pa- tients discharged alive, patients who had in-hospital mortality had a 13-fold higher rate of PSIs and almost a twofold greater ICU requirement (Table 1). Among those patients who died during their hospitalization, having a PSI was associated with increased ICU re- quirement but similar hospital and ICU stay as com- pared with patients discharged alive (Table 1).

The highest mortality rate was observed with pa- tients sustaining the PSI corresponding to postoper- ative physiologic or metabolic derangement followed by those with PSIs for postoperative sepsis and for pressure ulcers. Mean HLOS was longest in those with PSIs for postoperative wound dehiscence (108.1 days), pressure ulcer (48.0 days), and postoperative physio- logic or metabolic derangement (40.8 days). All patients with postoperative sepsis and wound dehiscence re- quired ICU admission, and over 90 per cent of patients with postoperative physiologic or metabolic derange- ment and respiratory failure required ICU admission (Table 2).

Discussion

The AHRQ PSI system has been endorsed by CMS and will likely become a significant factor in future determinations of hospital quality rating, public report- ing, and in reimbursement determination.3 In a previous analysis, we demonstrated that the AHRQ PSIs were associated with an overall positive predictive value of only 83 per cent with predictive values as low as 67 per cent for catheter-related bloodstream in- fections and 71 per cent for postoperative respiratory failure.4 Similarly, poor validity of the PSI was found for metabolic and physiologic derangements.11 In this current study we sought to establish what as- sociations exist between PSIs and patient morbidity and mortality.

Compared with patients who were discharged alive, patients who experienced in-hospital mortality expe- rienced a greater rate of PSIs, ICU requirement, and shorter length of ICU and hospital stay. The shorter LOS are likely a direct reflection of their in-hospital mortality. Among the patients who had a PSI and died, there was no statistically significant increase in length of hospital stay or ICU stay as compared with the patients who were discharged alive. We also found that dying without a PSI was associated with decreased ICU need and shorter ICU and hospital stay as com- pared with dying with a PSI. This is likely a reflection of the primary illness that caused early unavoidable death, as seen for example in trauma patients with severe head injuries. Thus, our findings suggest that having a PSI was more common in patients who died, and PSIs may be associated with increased ICU needs but not with increased hospital or ICU LOS.

PSIs corresponding to postoperative sepsis and postoperative metabolic or physiologic derangements were more closely associated with mortality and in- creased ICU requirement. Postoperative metabolic or physiologic derangements encompass acute kidney injury requiring dialysis and diabetes-related coma, ketoacidosis, and hyperosmolarity. The association between sepsis and mortality is well documented as are the associations between acute kidney injury and mortality.17-20 Therefore, one might speculate that PSIs may play a role in redress of such complications and help track and monitor the progress of improve- ment initiatives.

Significant attention has been directed at the limi- tations in accuracy and efficacy of administrative data collection systems. However, less demonstrated are the limitations that centrally collected and analyzed ad- ministrative data impose on quality improvement programs at an institutional level. Central administra- tive databases such as the AHRQ PSI system generate reports for participating institutions that include per- formance statistics and comparisons to national means. To truly understand the implications of such a report, however, process measures and/or additional individual factors of patients affected are imperative. As seen in our data set, we have noted an association among PSI, mortality, and ICU need, yet we were unable to ac- count for patient comorbidities and illness through metrics like the Acute Physiology and Chronic Health Evaluation II score.

Previous comorbidities and conditions of patients be- fore hospitalization are important, and despite increased incorporation of codes for ''present-on-admission'' in the AHRQ PSI system, the full impact of preadmission comorbidities cannot be completely appreciated.21-23 Patients with recalcitrant chronic surgical illness often have malnutrition, deconditioning, cachexia, increased lifetime hospital exposure, and are thus more suscepti- ble to in-hospital complications like sepsis, respiratory failure, and metabolic derangement.24 Currently, the AHRQ PSI system does not integrate any known, val- idated system for assessing patient illness and severity other than severity coding for billing purposes. Aggre- gate AHRQ PSI reports, in addition to the lack of pro- cess measures, preclude centers from crossreferencing their data to calculate illness scores thereby gaining a deeper understanding of the contributory factors to PSIs. Moreover, pre-existing comorbidities of patients have been shown to increase PSIs and thus not con- trolling for comorbidities likely negatively affects accurate public reporting.24 This issue is particularly meaningful for the financial viability of academic medical centers that deal disproportionately with such patients, often a result of such patients having lost or been unable to obtain private insurance.

In conclusion, the AHRQ PSIs are associated with increased mortality and increased ICU needs. How- ever, the known poor validity makes this a suboptimal tool for hospital safety benchmarking comparison, public reporting, and reimbursement determination. Further larger studies with normalization to patient illness are needed to continue to analyze the compar- ative effectiveness and potential meaning of the AHRQ PSI system.

REFERENCES

1. Agency for Healthcare Research and Quality. AHRQ Quality Indicators: Guide to Patient Safety Indicators. Rockville, MD: Agency for Healthcare Research and Quality; 2003.

2. Romano PS, Geppert JJ, Davies S, et al. A national profile of patient safety in U.S. hospitals. Health Aff (Millwood) 2003;2: 154-66.

3. Centers for Medicare and Medicaid Services (CMS), HHS. Medicare program; changes to the hospital inpatient prospective payment systems and fiscal year 2008 rates. Fed Regist 2007;162: 47129-8175.

4. Ramanathan R, Leavell P, Stockslager G, et al. Validity of Agency for Healthcare Research and Quality Patient Safety In- dicators at an academic medical center. Am Surg 2013;6:578-82.

5. Henderson KE, Recktenwald A, Reichley RM, et al. Clinical validation of the AHRQ postoperative venous thromboembolism patient safety indicator. Jt Comm J Qual Patient Saf 2009;7:370-6.

6. Chen Q, Rosen AK, Cevasco M, et al. Detecting patient safety indicators: how valid is 'foreign body left during procedure' in the Veterans Health Administration? J Am Coll Surg 2011;6: 977-83.

7. Utter GH, Cuny J, Sama P, et al. Detection of postoperative respiratory failure: how predictive is the Agency for Healthcare Research and Quality's Patient Safety Indicator? J Am Coll Surg 2010;3:347-54, e1-29.

8. Rosen AK, Rivard P, Zhao S, et al. Evaluating the patient safety indicators: how well do they perform on Veterans Health Administration data? Med Care 2005;9:873-84.

9. Rosen AK, Loveland S, Shin M, et al. Examining the impact of the AHRQ Patient Safety Indicators (PSIs) on the Veterans Health Administration: the case of readmissions. Med Care 2013; 1:37-44.

10. White RH, Sadeghi B, Tancredi DJ, et al. How valid is the ICD-9-CM based AHRQ patient safety indicator for postoperative venous thromboembolism? Med Care 2009;12:1237-43.

11. Zrelak PA, Sadeghi B, Utter GH, et al. Positive predictive value of the Agency for Healthcare Research and Quality Patient Safety Indicator for central line-related bloodstream infection ('selected infections due to medical care'). J Healthc Qual 2011;2: 29-36.

12. Utter GH, Zrelak PA, Baron R, et al. Positive predictive value of the AHRQ accidental puncture or laceration patient safety indicator. Ann Surg 2009;6:1041-5.

13. Cevasco M, Borzecki AM, Chen Q, et al. Positive predictive value of the AHRQ Patient Safety Indicator 'Postoperative Sepsis': implications for practice and policy. J Am Coll Surg 2011;6:954-61.

14. Rivard PE, Luther SL, Christiansen CL, et al. Using patient safety indicators to estimate the impact of potential adverse events on outcomes. Med Care Res Rev 2008;1:67-87.

15. Gallagher B, Cen L, Hannan EL. Advances in patient safety: from research to implementation (Volume 2: concepts and meth- odology). In: Henriksen K, Battles JB, Marks ES, Lewin DI, eds. Validation of AHRQ's Patient Safety Indicator for Accidental Puncture or Laceration. Rockville, MD: Agency for Healthcare Research and Quality; 2005.

16. Kaafarani HM, Borzecki AM, Itani KM, et al. Validity of selected Patient Safety Indicators: opportunities and concerns. J Am Coll Surg 2011;6:924-34.

17. Angus DC, Linde-Zwirble WT, Lidicker J, et al. Epidemiol- ogy of severe sepsis in the United States: analysis of incidence, out- come, and associated costs of care. Crit Care Med 2001;7:1303-10.

18. Martin GS, Mannino DM, Eaton S, et al. The epidemiology of sepsis in the United States from 1979 through 2000. N Engl J Med 2003;16:1546-54.

19. Ramanathan R, Leavell P, Stockslager G, et al. Validity of ICD-9-CM screening for sepsis in surgical mortalities. Surg Infect (Larchmt). doi:10.1089/sur.2013.089.

20. Thakar CV. Perioperative acute kidney injury. Adv Chronic Kidney Dis 2013;1:67-75.

21. Naessens JM, Huschka TR. Distinguishing hospital com- plications of care from pre-existing conditions. Int J Qual Care 2004;26(suppl 1)i27-35.

22. Naessens JM, Scott CG, Huschka TR, et al. Do complica- tion screening programs detect complications present at admis- sion? Jt Comm J Qual Saf 2004;3:133-42.

23. Houchens RL, Elixhauser A, Romano PS. How often are potential patient safety events present on admission? Jt Comm J Qual Patient Saf 2008;3:154-63.

24. Naessens JM, Campbell CR, Shah N, et al. Effect of illness severity and comorbidity on patient safety and adverse events. Am J Med Qual 2012;1:48-57.

RAJESH RAMANATHAN, M.D.,* PATRICIA LEAVELL, B.S.N., R.N.,[dagger] LUKE G. WOLFE, M.S.,* THERESE M. DUANE, M.D.[double dagger]

From the *Department of Surgery, [dagger]Performance Improvement, and the [double dagger]Division of Trauma, Critical Care & Emergency Surgery, Virginia Commonwealth University Medical Center, Richmond, Virginia

Presented at the Annual Scientific Meeting and Postgraduate Course Program, Southeastern Surgical Congress, Savannah, GA, February 22-25, 2014.

Address correspondence and reprint requests to Therese M. Duane, M.D., 1200 E. Broad Street, P.O. Box 980454, Richmond, VA 23219. E-mail: tduane@mcvh-vcu.edu.