A study of adherence to the AAO-HNS “Clinical Practice Guideline: Adult Sinusitis”

Abstract

A retrospective study was conducted to determine if physicians in otolaryngology practice adhered to the clinical practice guideline for adult sinusitis that had been issued by the American Academy of Otolaryngology-Head and Neck Surgery (AAO-HNS) 3 years earlier. We analyzed data obtained from the charts of 90 adults who had presented to an otolaryngology outpatient department with a diagnosis of chronic rhinosinusitis (CRS), acute bacterial rhinosinusitis (ABRS), or acute viral rhinosinusitis (AVRS); there were 76 cases of CRS, 11 cases of ABRS, and 3 cases of AVRS. Our goal was to ascertain how closely the treating physician had adhered to the AAO-HNS recommendations with respect to diagnosis, treatment, and prevention of these diseases. The study group was made up of 10 otolaryngologists. We evaluated 7 clinical practice metrics for CRS, 7 metrics for ABRS, and 3 for AVRS. We found that individual physician adherence rates for cases of CRS ranged from 0 to 100%; average scores for the 7 metrics ranged from 4 to 88%. For cases of ABRS, adherence scores ranged from 0 to 100%; average scores for the 7 metrics ranged from 0 to 41%. For AVRS, the rate of adherence for all 3 metrics was 0%. This study revealed wide variations in adherence to the AAO-HNS guideline, but overall adherence was generally poor. Adherence appeared to be worse for the acute types of rhinosinusitis than for chronic rhinosinusitis. In view of these findings, a worksheet was developed that clinicians could use to improve compliance with the guidelines.

Introduction

Clinical practice guidelines are "systematically developed statements to assist practitioner and patient decisions about appropriate healthcare for specific clinical circumstances."1

The adoption of clinical practice guidelines was meant to improve quality of care and decrease inappropriate and ineffective therapeutic practices. The impetus for developing such guidelines was a finding that scientific knowledge is not adopted systematically or in a timely fashion by the healthcare system. For example, one investigation found that it took an average of 17 years for new knowledge generated by randomized controlled trials to be put into practice.2

In 2001, the Institute of Medicine urged the healthcare system to become more effective (i.e., by providing services based on scientific knowledge) and more efficient (i.e., by avoiding waste).2 In recommending that healthcare decisions be based on evidence, the Institute wrote that the best way to accomplish this would be to perform analyses of existing medical evidence and use those to develop specific practice guidelines.

The Institute also recommended (1) identification of best practices, (2) dissemination of the guidelines to professional communities, (3) establishment of goals for improvement in care processes and outcomes, and (4) development of measures for assessing quality of care and support tools to help clinicians. The prevailing opinion was that in addition to being evidence-based, these guidelines should be systematic and transparent.3 Guidelines can also be used to evaluate the performance of individual physicians in particular and the healthcare system in general.

Despite all the potential benefits of practice guidelines, many physicians lack awareness and familiarity with the them, and they do not follow them consistently in clinical practice. Cabana et al wrote that a "lack of agreement, self-efficacy, outcome expectancy, and the inertia of previous experience are also a potential barrier."4 Indeed, a study of the quality of healthcare in America published in 2003 by McGlynn et al found that only 54.9% of American patients received the recommended services for acute and chronic conditions, as well as preventive care.5

Sinusitis is one of the most common medical conditions, affecting 1 in 7 adults in the United States; 31 million cases are diagnosed each year, with an estimated annual healthcare expenditure of $5.8 billion.6-7 Outpatient clinic visits, emergency department visits, imaging tests, and prescriptions for antibiotics are a major source of this cost. Approximately 20% of all antibiotics prescribed are for the treatment of sinusitis.8

Because of the heavy economic burden and considerable variations in the treatment of sinusitis-not only within otolaryngology but across other specialties as well-the practical application of a clinical practice guideline for this disease would be of significant benefit to healthcare delivery. The American Academy ofOtolaryngology-Head and Neck Surgery's (AAO-HNS) "Clinical Practice Guideline: Adult Sinusitis" was published in 2007.9 The purpose of the guideline was to improve the diagnostic accuracy of adult sinusitis, to reduce inappropriate antibiotic use and radiographic imaging, and to promote appropriate use of ancillary testing.

The AAO-HNS guideline was created in conjunction with representatives from many other pertinent clinical specialties, including allergy, emergency medicine, family medicine, immunology, infectious disease, internal medicine, pulmonology, and radiology. Experts in nursing, health insurance, and medical informatics were also included in the development of the guidelines. This multidisciplinary panel reviewed the appropriate evidence-based literature and agreed on the guideline that contains nine actionable statements that are evaluated on the evidence of the medical literature (figure 1 ). The quality of the aggregate evidence, benefit-harm assessments, and costs were taken into consideration. Four categories were created by the AAO-HNS based on value judgments, the role of patient preferences, and the strength of the existing scientific literature for each of the nine statements; the statement strengths are strong recommendation, recommendation, option, and recommendation against.

A study was performed to assess adherence to the AAO-HNS adult sinusitis guidelines in an academic otolaryngology practice.

Subjects and methods

This study was based on a retrospective chart review of patients who had presented to the Department of Otolaryngology-Head and Neck Surgery at Henry Ford Hospital, a large tertiary care academic center, over a 3-year period. The otolaryngologists were all board-certified, and some had subspecialty fellowships in rhinology, laryngology, head and neck surgery, sleep medicine, and neurotology. The patients' clinic notes for each physician were evaluated for adherence to the nine rhinosinusitis guideline evidence-based statements during each patient encounter.

Eligibility criteria. ICD-9 codes were obtained from claims data submitted from June 2007 through June 2010 to identify all patients aged 18 years or older who had an initial visit to the otolaryngology department and had been diagnosed with either acute maxillary sinusitis (ICD-9 code: 461.0), acute sinusitis unspecified (461.9), chronic maxillary sinusitis (473.0), chronic ethmoid sinusitis (473.2), chronic sphenoid sinusitis (473.3), chronic sinusitis (473.8), or unspecified sinusitis (chronic) (473.9).

Two Certified Professional Coders (CPCs) randomly selected 10 charts for each otolaryngologist. The CPCs then reviewed the clinic notes to determine if documentation was present that was consistent with the AAO-HNS clinical practice guideline for adult sinusitis. As the CPCs reviewed their own charts, they discussed with each other the information contained in them, especially if there was any confusion regarding documentation .or terminology. The ICD-9 diagnosis was confirmed by reviewing the billing claim submitted at the time of the visit.

The CPCs were instructed to identify the following data in each clinic note as they pertained to the patients' underlying diagnoses (table 1):

* the duration and type of symptoms, which were used to distinguish chronic rhinosinusitis (CRS), acute bacterial rhinosinusitis (ABRS), and acute viral rhinosinusitis (AVRS);

* documentation of pain, including objective pain scale measurements;

* physical examination findings;

* recommendations for symptom relief;

* use of appropriate antibiotics;

* modifying factors;

* diagnostic testing; and

* prevention.

The patients diagnosis was established on the basis of AAO-HNS sinusitis guideline statements la and 7a (figure 1). The otolaryngologist's clinical diagnosis was then compared with the guideline-derived diagnosis. All other guideline adherence measures were based on those applicable to the AAO-HNS Clinical Practice Guideline diagnosis.

Treating physicians. The 10 physicians were boardcertified otolaryngologists who regularly saw patients with "sinus" complaints. The mix of physicians was typical of that found in a tertiary care academic center: 3 general otolaryngologists, 3 fellowship-trained facial plastic and reconstructive surgeons, 1 fellowship-trained rhinologist, 1 fellowship-trained sleep expert with a robust rhinology practice, 1 fellowship-trained laryngologist, and 1 fellowship-trained neurotologist.

Patients. After the initial medical record review, we identified 98 patients who met our inclusion criteria. Of these, 76 patients met the AAO-HNS diagnostic criteria for CRS, 11 for ABRS, 4 for subacute rhinosinusitis, and 3 for AVRS; no patient had recurrent acute rhinosinusitis, and 4 cases were classified as unknown because documentation was insufficient to support any specific type of rhinosinusitis. The 4 patients with subacute rhinosinusitis and the 4 with an unknown type were excluded from the study.

The patient population was made up of 90 patients-31 men (34.4%) and 59 women (65.6%)-aged 22 to 88 years (mean: 54).

Data analysis. Adherence scores were calculated as percentages based on the number of times a statement was followed divided by the number of patient encounters. The analysis provided an evaluation of both the individual otolaryngologist's guideline adherence across all patients and the aggregate compliance score of the department as a whole for each statement. Additionally, where individual statements provided for the inclusion of subrecommenda- tions (i.e., education of patients for prevention to include hand washing, smoking cessation, sinus irrigation, etc.), the statement was scored according to the percentage of additional recommendations completed. As such, it was possible for a physician to receive a 50% compliance score for a statement applied to a single patient.

Ethical considerations. This study was approved by the Henry Ford Health System Institutional Review Board.

Results

CRS. For the 76 cases of CRS, the individual physicians' adherence scores for the seven statements ranged from 0 to 100% (table 2). The average rates ranged from 4 to 88% (table 3).

ABRS. For the 11 cases of ABRS, the individual scores ranged from 0 to 100% (table 4). The average rates ranged from 0 to 41% (table 3).

AVRS. Guideline adherence in the 3 cases of AVRS was 0%, which was primarily attributable to the fact that the physicians misdiagnosed it as ABRS (table 5).

Discussion

This study found wide variations in adherence to the AAO-HNS guideline, with overall adherence being poor. Adherence appeared to be worse in cases of acute rhinosinusitis than in cases of chronic disease, stemming mostly from a failure to adhere to the statement for diagnosis.

Alowpercentageofpatientsmetthecriteriaforadiagnosis of acute rhinosinusitis in our study (15.6% [14/90]), likely because most patients with these conditions are treated initially by their primary care physician or in an emergency department. In 2011, Mattos et al!0 reported that only 6.5% of office visits for acute sinusitis were made to otolaryngologists, which may explain the poor performance in our study.

Given the complete discrepancy (100%) between the AAO-HNSdiagnosesandthephysician-assigned diagnoses in this investigation, it is possible that the otolaryngologists were aware ofthe AAO-HNS guideline and actually applied it too stringently, failing to recognize cases of ABRS and diagnosing them as AVRS instead.

For cases of ABRS, adherence with the AAO-HNS recommendation to avoid radiologic testing in uncomplicated cases was 36%. As a result, more than one-third of patients experienced unnecessary radiation exposure and increased expenditure of healthcare dollars. The compliance rate for prescribing appropriate antibiotics in ABRS was 0%.

On the whole, the otolaryngologists were more compliant with the CRS guideline for using endoscopy and computed tomography in the diagnosis (66 and 71%, respectively). This is of significant importance, as noted by Bhattacharyya and Lee, who showed that adherence to the guidelines endoscopic and symptoms criteria significantly improved the specificity and the positive and negative predictive values for CRS.11

The otolaryngologists did not do as well with the modifying factors and the prevention statement with respect to ( 1 ) ordering testing to evaluate for allergic rhinitis, cystic fibrosis, an immunocompromised state, ciliary dyskinesia, and anatomic variations and (2) offering smoking cessation and hand washing. In patients with CRS, individual physician compliance rates for evaluating the patient for modifying factors, such as an allergy evaluation, ranged from 0 to 56%, with a group average of 28%. For prevention, the range was 13 to 40%, with an average of 30%. Most patients did not receive suggested recommendations that could demonstrate benefit with little risk of harm.

Study limitations. As with any retrospective analysis, this one had several limitations. One obvious limitation was the small size of the patient population. However, as a pilot study of adherence by otolaryngologists to the clinical practice guideline, it is unlikely that a greater number of patients would have made a difference.

The calculated adherence rates were based on a diagnosis supported by clinical documentation, as opposed to the physician-assigned diagnosis. Since it is necessary to make an accurate diagnosis to provide appropriate care, the study methodology was appropriate.

This study may be limited by the physicians' inability to accurately assign an ICD-9 code to a diagnosis of acute rhinosinusitis, since ICD-9 codes do not differentiate between the bacterial and viral forms of sinusitis. It is unclear what effect ICD-10 would have.

Follow-up with the otolaryngologists. In an attempt to improve our department's performance in adhering to the AAO-HNS Clinical Practice Guideline: Adult Sinusitis, we issued "report cards" to the 10 otolaryngologists for their review and feedback. In face-to-face meetings with each, discussions coveredthe wide variations in results amongthe 10 otolaryngologists, as well as their own performance on individual measures. Most of them had believed they had followed the clinical practice guideline "all the time," and they were surprised to learn that the chart review showed otherwise.

Since no single otolaryngologist achieved stellar results in all categories, a worksheet was developed based on the AAO-HNS guideline that all clinicians can use at each encounter with a patient with a presumptive diagnosis of sinusitis (figure 2). This worksheet was circulated for approval and suggestions, and it is now available in all of our clinics. A pocket-sized version is also available.

Other compliance-monitoring tools. Several initiatives have been developed in the public sector to evaluate physician performance on clinical guidelines. The Physician Quality Reporting System (PQRS) provides an incentive payment for physicians who report their performance on quality measures.12 This system has been voluntary since 2007, but it will be mandated for all physicians starting in 2015 as part of the Patient Protection and Affordable Care Act. Under the PQRS, physicians will be penalized 1.5 to 2.0% of their total estimated Medicare Part B Physician Fee Schedule for failing to report quality measures. One problem with the PQRS mandate is that applicable quality measures have not yet been identified for many specialties, including otolaryngology.

In addition, the American Board of Medical Specialties has adopted four measures that are required to fulfill its Maintenance of Certification (MOC) requirement, one of which is practice-based learning. Practice-based learning involves physicians completing a practice performance assessment that (1) demonstrates their use of best evidence and practices and (2) compares their practices to those of their peers and national benchmarks. This is accomplished through the use of self-evaluation tools that guide physicians through collecting data from their own practice, using chart reviews, patient surveys, and a practice system survey to create a multidimensional practice performance assessment. This study represents the first of many steps necessary to implement a process that will fulfill the requirements of MOC.

In conclusion, there were wide variations in adherence rates among individual otolaryngologists and within the department as a whole to the AAO-HNS Clinical Practice Guideline: Adult Sinusitis. While there may be an expectation that an academic department would be more likely to demonstrate high compliance with guidelines, this was not the case. To improve performance in this regard, tools need to be developed for physicians that provide real-time guidance and feedback so that they can improve adherence to clinical practice guidelines.

References

1. Field MJ, Lohr KN, eds. Clinical Practice Guidelines. Directions for a New Program. Washington, D.C.: National Academy Press; 1990.

2. Institute of Medicine. Crossing the Quality Chasm. A New Health System for the 21 st Century. Washington, D.C.: National Academy Press, 2001.

3. Rosenfeld RM, Shiffman RN. Clinical practice guidelines: A manual for developing evidence-based guidelines to facilitate performance measurement and quality improvement. Otolaryngol Head Neck Surg 2006;I35(4 Suppl):Sl-28.

4. Cabana MD, Rand CS, Powe NR, et al. Why don't physicians follow clinical practice guidelines? JAMA 1999;282(15):1458-65.

5. McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med 2003;348(26):2635-45.

6. Anand VK. Epidemiology and economic impact of rhinosinusitis. Ann Otol Rhinol Laryngol Suppl 2004;193:3-5.

7. Lethbridge-Çejku M, Rose D, Vickerie J. Summary Health Statistics for U.S. Adults. National Health Interview Survey, 2004. National Center for Health Statistics. Vital Health Stat 2006;10(228):19-22.

8. Anon JB, Jacobs MR, Poole MD, et al; Sinus and Allergy Health Partnership. Antimicrobial treatment guidelines for acute bacterial rhinosinusitis. Otolaryngol Head Neck Surg 2004; 130(1 Suppl):Sl-45.

9. Rosenfeld RM, Andes D, Bhattacharyya N, et al. Clinical practice guideline: Adult sinusitis. Otolaryngol Head Neck Surg 2007; 137(3 Suppl):Sl-31.

10. Mattos JL, Woodard CR, Payne SC. Trends in common rhinologic illnesses: Analysis of U.S. healthcare surveys 1995-2007. IntForum Allergy Rhinol 2011; 1( 1 ):3-12.

11. Bhattacharyya N, Lee LN. Evaluating the diagnosis of chronic rhinosinusitis based on clinical guidelines and endoscopy. Otolaryngol Head Neck Surg 2010; 143(1): 147-51.

12. Physician Quality Reporting System. Centers for Medicare & Medicaid Services Web site. http://www.cms.gov/Medicare/ Quality-Initiatives-Patient-Assessment-Instruments/pqrs/index. html. Updated April 20, 2014. Accessed May 28,2014.

Ilaaf Darrat, MD; Kathleen Yaremchuk, MD; Spencer Payne, MD; Michelle Nelson, MBA, CPC

From the Department ofOtolaryngology-Head and Neck Surgery, Henry Ford Hospital, Detroit (Dr. Darrat and Dr. Yaremchuk); Managed Care Services, Henry Ford Health System, Detroit (Ms. Nelson); and the Department of Otolaryngology-Head and Neck Surgery, University of Virginia, Charlottesville (Dr. Payne). The study described in this article was conducted at Henry Ford Hospital.

Corresponding author: Kathleen Yaremchuk, MD, Department of Otolaryngology-Head and Neck Surgery, Henry Ford Hospital, 2799 W. Grand Blvd., Detroit, MI 48202. Email: [email protected]