Institutional Review of Compliance With NCCN Guidelines for Breast Cancer: Lessons Learned From Real-Time Multidimensional Synoptic Reporting

Background

All health care stakeholders are seeking affordable, high-quality, patient-centered care for breast cancer. NCCN develops guidelines for this care,¹ which are evidence-based when evidence exists, and expert opinion–based when high-level evidence is lacking. Guideline development by itself is not sufficient to ensure quality care. Hence, adherence to recommended care pathways must be audited, and auditing requires metrics.² A quality metric or measure (QM) attempts to quantify quality in a domain of care with a specific numerator and denominator, each with inclusion and exclusion criteria. A QM can be developed for each of the NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines) by auditing the percentage of patients with care adherent to that guideline.² Once the QM is defined, it can be used locally or nationally for quality improvement initiatives. Multiple centers, including NCI-designated and nondesignated oncology centers, have reported their NCCN Guidelines compliance rates.^3–7 In general, high compliance rates are considered evidence of high-quality care.

Despite international acceptance of NCCN-endorsed care, variation in the adoption of these guidelines exists. This results in disparate patient care, as measured by compliance. Variation of care has been documented based on age, race, geography, insurance, and education status.^8–11 Furthermore, overuse of testing and underuse of treatment occurs. The former is associated with higher costs of care, and the latter with poor patient outcomes.^12–14 The primary objectives of this study were to evaluate real-time compliance with NCCN Guidelines using electronic synoptic templated reporting, document reasons for noncompliance, and assess the potential of local quality results to aid national quality strategy.

Methods

This study was undertaken at a single institution that provides National Accreditation Program for Breast Centers (NAPBC)–accredited interdisciplinary breast oncology care offered at a single site. The structure, processes of care, and quality outcomes at this site have been described elsewhere.^7,15 During the past decade, it has been designated policy at this institution to measure, adhere to, and create a culture emphasizing the importance of NCCN Guidelines. Since 2009, NCCN Guideline compliance has been recorded by surgeons prospectively at the point of patient contact before and after surgery using electronic synoptic templates (EST). The EST is incorporated into the typed or dictated progress note for that patient encounter. Information from the EST is then entered into the institution’s patient registry. Ten percent of the registry is reaudited annually, compliant with the Agency for Healthcare Research and Quality (AHRQ) recommendations for registry accuracy. Furthermore, the audit results are critically analyzed at biannual interdisciplinary NAPBC committee meetings, during which a decision is made whether compliance is satisfactory. Reasons for noncompliance are discussed and then categorized as a quality concern or not. Action plans are generated based on this review. In addition, compliance to NCCN Guidelines for each patient is discussed at a weekly interdisciplinary tumor board in which all members are encouraged to challenge the proposed care if it is not guideline-compliant.

Institutional Review Board approval was obtained for review of the patient registry. The treatment records of all patients with newly diagnosed breast cancer treated between January 2010 and December 2011 were assessed for NCCN Guideline compliance. A standard retrospective chart review of the electronic health record (EHR) was also conducted for NCCN compliance to compare with the real-time EST. Concordance of the prospective synoptic audits with retrospective review was assessed and reasons for noncompliance with NCCN Guidelines were documented. Compliance was based on treatment received within the institution. NCCN Guidelines do not recommend specific surgical care pathways for patients with stage 4 disease, suggesting that surgery should not be routine care for these patients. Therefore, the institution’s compliance with NCCN Guidelines regarding surgery was based primarily on omission of surgery as being compliant. The following factors were analyzed for their influence on compliance with NCCN Guidelines: age, Charlson comorbidity index¹⁶ (CCI), estrogen receptor (ER)/progesterone receptor (PR)/HER2 status by immunohistochemistry and fluorescence in situ hybridization, AJCC stage,¹⁷ year of treatment, surgeon, medical oncologist, and radiation oncologist. Statistical analysis was performed using SAS 9.3 (SAS Institute Inc., Cary, NC). Chi-square and Fisher exact tests were used for comparisons. Statistical significance was reported at α level of .05.

Results

The study group comprised 395 patients and 401 cancers, with 3 men and 392 women. Patient age ranged from 25 to 97 years (mean and median both 63 years). The breast-conserving therapy rate was 56% in 341 patients undergoing surgery at the institution. According to AJCC 7th Edition staging, 56 (14%) were stage 0, 182 (46%) were stage I, 100 (25%) were stage II, 35 (9%) were stage III, and 22 (6%) were stage IV. Routine HER2 testing was not performed in patients with stage 0 disease, thereby excluding 56 patients of 395 from HER2 testing of tumors. In the remainder of patients, ER/PR/HER2 status was ER- and/or PR-positive and HER2-negative in 248 (73%), ER- and/or PR-positive and HER2-positive in 18 (5%), ER/PR/HER2-negative in 46 (14%), ER/PR-negative and HER2-positive in 17 (5%), and “other” in 10 (3%) patients. Most patients in the other category did not have sufficient tissue to undergo HER2 testing.¹⁸ Any attempt to perform HER2 testing that failed because of insufficient tissue was categorized as NCCN-compliant. Compliance with NCCN Guidelines was 94% for the initial staging evaluation, 97% for surgery, 91% for chemotherapy, 89% for hormone therapy, 91% for radiation therapy, 85% for follow-up, and 100% for ER/PR/HER2 status documentation. Reasons for noncompliance are detailed in Table 1. Advanced stage, older age, and higher CCI score were associated with decreased compliance rates (Tables 2 and 3). ER/PR/HER2 status, year of treatment, and individual care provider were not associated with differences in compliance rates. Compliance results stratified by ER/PR/HER2 status are demonstrated in Table 4. Compared with retrospective chart review, EST reporting by surgeons before and after breast cancer operations had an accuracy of 97% and 98% for NCCN Guidelines compliance for breast surgery and management of the axillary nodes, respectively. The audit results of 2010 and 2011 were reviewed

Table 1

NCCN Guideline Compliance and Reasons for Noncompliance

View Table

Discussion

The goals of this study were to determine the quality of care provided to patients as measured by compliance with NCCN Guidelines and to determine whether local auditing could aid national quality strategy by describing justifiable reasons for noncompliance. Previous publications have documented gaps in the quality of breast cancer care as measured by variability of guideline adherence.^{3,8,12–14,19,20} These national audits successfully identified inequities and disparities of care based on patient demographics, such as age, race, zip code, and economic and insurance status. Single-institution audits are unable to perform this

Table 2

Effects of Stage on NCCN Compliance^a

View Table

Other limitations of national audits include potential coding inaccuracies in claims and administrative databases; age restriction to patients older than 65 years in some databases, which may not always be relevant to other age groups; and attribution issues. In national audits, identifying which care provider or system to attribute noncompliance to a guideline can be difficult. For example, if a quality gap is apparent for lack of receipt of postmastectomy radiation therapy in patients with 4 or more positive nodes, is the cause a regional surgeon referral failure, patient refusal, or lack of patient access to a radiation center within 100 miles? Surgeon failure to refer is a quality concern; patient refusal is not. Lack of access to a radiation center is also a quality concern for patient and health care policy stakeholders. If identified, then this quality gap should be addressed by regional health care providers, aided by state and national support. An institutional database is robust, with enough information to decipher whether noncompliance is the result of patient, provider, tumor, institutional, or geographic factors.¹⁹ Reasons for noncompliance are embedded in the

Table 3

NCCN Guideline Compliance Based on Age and Charlson Comorbidity Index

View Table

We have shown that when an institution has a pervasive culture and policy of guideline compliance, a high compliance rate is achievable. With the exception of follow-up care, we have demonstrated compliance rates greater than 90%. As a result, fewer than 10% of patients lacked receipt of recommended therapies. These findings provide evidence of the workability and feasibility of NCCN Guidelines. Understanding reasons for noncompliance can potentially aid a national quality strategy. The nonquality reasons for lack of receipt of recommended therapy were comorbidities, limited life expectancy, patient refusal, patient choice after shared decision-making, and patient preference for complementary and alternative medical therapy. This granularity of information is useful both for risk adjustment before peer performance comparison, and for benchmarking and establishment of target goals for NCCN compliance. Based on the results, target goals for guideline compliance should not be 100%, but high levels are achievable in an institution committed to guideline compliance.

Suboptimal care identified by critical review of local audits is actionable. The audits prompted discussion of preoperative imaging choices and motivated better survivorship efforts. The most common reasons for noncompliance in this study were patient choice after shared decision-making and patient refusal to undertake recommended care. Neither reason would be considered a gap in quality. On the other hand, findings demonstrated noncompliance in some patients with clinical stage I and II disease in terms of extra testing or different testing regarding systemic imaging. Both noncompliant use of systemic imaging and use of an imaging type that was different from what was recommended were documented, with the use of PET imaging instead of the NCCN-recommended CT imaging plus bone scan. Overuse of testing is a quality and cost issue, recently addressed by ASCO.²⁵ A review of 67,847 patients with stage I and II breast cancer from the SEER database showed an increase in the use of systemic imaging from 1992 to 2005 in the form of CT scans, PET scan, and brain MRI.¹³ In a survey of medical oncologists, 34% were noncompliant with NCCN Guidelines because of unnecessary use of PET scanning for initial assessment.²⁶ The value of real-time auditing at the local level is to immediately recognize guideline deviation and implement action plans to address quality issues. The choice to perform and choice of systemic imaging are now more often reviewed during the institution’s tumor boards.

Another cause of guideline noncompliance in this study was withholding therapies because of limited life expectancy and comorbidities. Whether this deviation from recommended care is deemed justifiable is problematic on a national level, because it is not possible to determine whether nonreceipt of therapy is the result of age bias, a quality issue, or actual severe life expectancy limiting comorbidity, which is a justifiable reason. Efforts to improve local survivorship planning have also been accelerated based on audit results showing our follow-up compliance to be less than 90%. This effort was made more feasible by the recent addition of NCCN Guidelines for Survivorship (available at NCCN.org).

Guideline compliance has been noted by

Table 4

Effects of ER/PR/HER2 Status on NCCN Compliance

View Table

The use of EST reporting has been described elsewhere to document patient care. Synoptic reporting may be more accurate, complete, efficient, and actionable compared with narrative reports and chart review.^29–31 NCCN compliance with multiple care pathways was documented in a few minutes, increasing feasibility of data entry. The accuracy of the templates was compared with standard burdensome retrospective chart review, yielding a discrepancy of only 3% and 2% for breast and lymph node surgery compliance, respectively.

The final argument for the value of local audits of compliance with evidence-based guidelines is self-evident: all quality improvement must occur at a local level before there can be an expectation of national improvement. This is inherent in the methodology of quality improvement endorsed by AHRQ.³² Local trends in performance can also be tracked. Our compliance rates have improved compared with our prior report, especially for radiation and medical oncology compliance.⁷

Conclusions

High levels of compliance with NCCN Guidelines for Breast Cancer care are achievable, supporting the utility of incorporating NCCN care pathways into daily patient care. Electronic synoptic templates, populated by care providers at the point of patient contact are quick, feasible, and accurate, allowing real-time guideline compliance assessment. This method facilitates early recognition of local quality gaps. Local studies may also potentially assist the development of a national quality strategy by aiding risk-adjusted peer performance comparison and benchmarking, because reasons for noncompliance can be documented.