NCCN Guidelines and Quality Cancer Care: Where Have We Come From, and Where Should We Be Going?

I have a clear and vivid recollection of the initial meeting of our NCCN Melanoma Panel 20 years ago. Roughly 25 national experts in melanoma, all from NCI-designated comprehensive cancer centers across the country, including surgeons, medical oncologists, dermatologists, radiologists, and pathologists, gathered together in a room in New York City, in a group chaired by Alan Houghton, MD, and overseen by Rodger Winn, MD. The stated mission of the then-fledgling organization, the National Comprehensive Cancer Network, was to “improve the quality, effectiveness, and efficiency of cancer care.”1 Our panel's charge sounded straightforward at the time: to derive a set of evidence-based or, when evidence was insufficient, consensus-based treatment guidelines for physicians treating patients with melanoma of all stages. We were to develop recommendations for the diagnosis, workup, initial treatment including adjuvant therapy, follow-up, and management of recurrent disease.

Over the ensuing 2 decades, these guidelines have evolved substantially, in part in response to evolving new information (clinical trials of local, regional, and systemic therapy; newer imaging modalities; and seminal retrospective reviews, series, and meta-analyses), but also in large part in response to our continuously evolving understanding of the role of guidelines as they are applied to achieve quality in clinical cancer care. This evolving understanding involves several components, including the challenge of effectively defining quality care, the task of identifying which elements of a treatment guideline best support quality care, and determining how best to measure the impact of guidelines on quality care.

The Elusive Definition of Quality Care

It is nearly impossible to arrive at a concise, comprehensive, and reproducible definition of quality medical care. To paraphrase an old saw, “Although it is difficult to define, we all know when we do (or don't) see it.” In 2001, the Institute of Medicine (IOM) made an initial attempt to define the 6 key elements of quality medical care. These elements included care that was effective, efficient, safe, timely, patient-centered, and equitable.2 Although few would argue with these tenets of quality, each describes a very subjective continuum with no absolute reference point, each quite difficult to measure in a strictly quantifiable fashion. This posed a challenge, as, in the words attributed to Sir William Thomson (Lord Kelvin), “If you cannot measure it, you cannot improve it.” In 2013, the IOM made another more detailed attempt to define a list of components of quality care relevant to patients with cancer, including components that should be discussed as part of a cancer care plan, such as treatment goals; initial plan for treatment and proposed duration; treatment benefits and harms; and estimated total and out-of-pocket costs of cancer treatment3 (a more detailed list is available in Box 3-3 of “Delivering High-Quality Cancer Care: Charting a New Course for a System in Crisis”).3 Although not all of these components apply to every patient, as a group they define an interesting list of “talking points” to be addressed with patients in the collaborative pursuit of quality cancer care.

Another challenge in the attempt to define quality medical care is the question of whether quality is an absolute or a relative value. Are there definable thresholds for quality? Is achievement of quality care subjective or objective, and as judged by whom? Are the standards necessary to achieve high-quality cancer care at NCCN Member Institutions the same as those required to achieve high-quality cancer care in rural or inner-city America, or in low-resource settings here or in other countries? And perhaps most controversially, should cost and value ever be included in the definition of quality medical care? If the same result can be achieved at lesser cost, can that be defined as better quality and better value?

NCCN has a strong record of recognition as a leader in quality cancer care. In 2008, the NCCN Drugs & Biologics Compendium was recognized by the Centers for Medicare & Medicaid Services (CMS) (and several major third-party payers) as a “mandated reference for establishment of coverage policy and coverage decisions.”4,5 In 2015, CMS issued a Request for Application for an Oncology Care Model, encouraging reporting of cancer care that is “consistent with clinical guidelines of… the National Comprehensive Cancer Network (NCCN).”6 But can we do better going forward?

Guidelines: A Recipe for Quality?

A cookbook recipe consists of a set of precise instructions to deliberately combine a list of ingredients in specific amounts and order, often with the application of a specific amount and duration of heat, to achieve a predictably reproducible result. In many respects, a recipe represents a physics experiment. Adherence to the very specific conditions defined should always produce an identical result. Quality medical care is anything but a 2-dimensional recipe. Dimensions of resource availability, physician expertise, patient preferences and prejudices, stress, varying educational levels, cost, and value are impossible to capture in a flat pathway or guideline.

Effective guidelines must avoid being too prescriptive, but rather should emphasize appropriate evidence-based options for evaluation and treatment, with strategies for how to arrive at appropriate individual patient recommendations. The cornerstone of this task is to splice anxiety-infused patient preferences into what relevant facts are known. A few examples will illustrate this challenge.

The decision about prophylactic contralateral mastectomy (PCM) for patients with breast cancer and no known genetic predisposition to develop subsequent breast cancer is one often fraught with emotion. Effective guidelines should encourage a patient-physician discussion on the evidence-based quantitative estimate of risk of contralateral breast cancer, what is known about the impact of PCM on the risk of dying of breast cancer in the context of competing health risks, patient anxiety associated with ongoing radiologic screening of the contralateral breast, patient concerns about cosmesis and symmetry, and what is known generally about patient satisfaction after PCM across multiple domains. No one would describe this conversation as efficient, but all would describe it as high-quality medical care.

Although guidelines to support quality medical care cannot be overly prescriptive, they must, if they are to be useful, strive to avoid the trap of being too comfortably vague. For example, when the NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines) for Melanoma address the issue of whether to perform a sentinel lymph node biopsy (SLNB), verbs such as “consider” or “discuss and offer” are used. There is an expectation that the decision will be tempered by “clinical judgment.” Imagine how much more useful these guidelines would be, especially to nonexperts (perhaps the practitioners most likely to benefit from guidelines that help determine whether to perform SLNB on a given patient), if they also included a discussion of factors that contribute to clinical judgment. These might be in the form of specific talking points, including the probability of a positive sentinel node, the fact that SLNB is prognostic but may not be therapeutic, the small but finite morbidity of the procedure, the patient's nonmelanoma comorbidities and competing risks, and even the anticipated incremental cost to the patient for this staging procedure (usually something we know very little about until after the fact).

Another more generic example of how difficult it is to define guideline-based quality care comes from the realm of adjuvant therapy. Again, the NCCN Guidelines for Melanoma list a number of options for adjuvant therapy after complete resection of node-positive disease. Against the background of a very broad range of risk of recurrence in this heterogeneous group, the guidelines list several options, including interferon, biochemotherapy, and high-dose ipilimumab. Although the ranking of these adjuvant therapy options is addressed somewhat in the footnotes and discussion, the guidelines fall short of synthesizing these points into a useful framework for nonexperts in the field. A more interactive guideline structure would empower both practitioner and patient to directly access clinical trial reports, contrasting expert points of view in areas of controversy, consensus summary statements, and NCCN panel recommendations. A bullet list of points to be addressed in a clinical discussion about adjuvant therapy might include addressing the clinical end point most valued by the patient, a sense of the patient's tolerance to treatment-related toxicity, and cost. For skeptics of the routine use of adjuvant therapy, one bullet might acknowledge that, if there is a 10% treatment-related improvement in a specified clinical end point, then necessarily 90% of patients receiving the treatment will experience all of the cost and toxicity with no benefit defined by that end point. Furthermore, in the absence of effective biomarkers predicting clinical benefit, patients need to understand that clinical trial results apply to groups of patients and may not impact them specifically.

Metrics of Guideline-Based Quality Care

There is no single metric that defines quality care for all patients in all situations. High quality in the example of PCM is best characterized as an honest bidirectional discussion about all of the issues enumerated. This interaction is infinitely more difficult to capture as a metric of care than would be the incidence of performing PCM, but it is also considerably more important. The metric of quality in the example of whether to perform SLNB is not the easily measured end point of how often SLNB was performed for a given stage of melanoma, but rather how thorough and understandable was the discussion of the talking points. In the example of whether to administer a specific adjuvant regimen in stage III melanoma, the optimal metric of guideline-based quality care is not whether adjuvant therapy was administered but rather how thorough and understandable was the discussion that informed the final adjuvant therapy decision. Although critical to our understanding of quality, the effectiveness of these discussions is not easily measured. In words that have been attributed to Albert Einstein, “Not everything that counts can be counted; not everything that can be counted counts.”

NCCN Guideline Initiatives

The NCCN Guidelines process is evolving from the simple Categories of Evidence and Consensus stratification of recommendations. Most recently, in an attempt to derive value-based recommendations and further ensure quality, certain disease site panel members have been asked to further categorize the data supporting selected therapeutic interventions with NCCN Evidence Blocks. This grading system incorporates panel member assessment of a treatment option on 5 characteristics: efficacy, safety, quality of evidence, consistency of evidence, and affordability. Using a scale of 1 through 5, these Evidence Blocks result in a graphic display of the panel members' assessments. The resulting Evidence Blocks are being integrated into the guidelines (Figure 1). The methodology behind the Evidence Blocks continues to evolve, and the reliability, reproducibility, and clinical utility of this new scoring system remain undefined.

Another exciting initiative that has the potential to address many of the limitations of the flat format of the current guidelines is being explored. This concept would offer the potential for the current text-based guideline infrastructure to be replaced with a synoptic database architecture, in which each guideline node could be updated as new information becomes available, without requiring publication of an entire new version.

As this new platform matures, the guidelines of tomorrow should be much more useful to both clinicians and patients in their shared goal of improving the overall quality of care. One envisions an interactive multimedia online guideline process, in which every aspect of care—from initial diagnosis and staging through palliative and end of life care—would be represented by a “node” in a pathway. Behind every node, just a mouse click or two away, would be a referenced discussion about the

Figure 1.
Figure 1.

NCCN Evidence Blocks. (A) Categories and definitions. (B) Sample evidence block. (C) NCCN Evidence Blocks as applied to adjuvant therapy options for completely resected stage III melanoma. Modified from Coit DG, Thompson JA, Algazi A, et al. NCCN Clinical Practice Guidelines in Oncology: Melanoma, NCCN Evidence Blocks. (A, B) Page EB-1. (C) Page ME-4. Accessed March 23, 2016.

Citation: Journal of the National Comprehensive Cancer Network J Natl Compr Canc Netw 14, 4; 10.6004/jnccn.2016.0044

most current data relevant to that node, nomogram-based estimates of an individual patient's probability of a specified outcome, contrasting opinions by experts in areas of controversy, and guides on issues to be raised with patients to help them define preferences. We might someday offer a virtual multidisciplinary tumor board and tools to help patients and providers assess the cost of interventions, not only to payers but also, and especially, to individual health care consumers.

If these ideas seem a bit fantastical, they should. Few of us in that room 20 years ago could have envisioned the extraordinary and truly impactful accomplishments NCCN would realize in its first 2 decades. These accomplishments were driven by leaders with the vision to see how NCCN could truly contribute to its mission “to improve the quality, effectiveness, and efficiency of cancer care.” A major challenge in the decades ahead is leveraging new technology to adapt the guideline process to support and inform the infinite complexity of quality clinical care of patients with cancer.

References

  • 1.

    About NCCN. Available at: www.nccn.org. Accessed March 23, 2016.

  • 2.

    Committee on Quality of Health Care in America; Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001.

    • Search Google Scholar
    • Export Citation
  • 3.

    Levit L, Balogh E, Nass S, Ganz PA, eds; Committee on Improving the Quality of Cancer Care: Addressing the Challenges of an Aging Population; Board on Health Care Services; Institute of Medicine. Delivering High-Quality Cancer Care: Charting a New Course for a System in Crisis. Washington, DC: National Academies Press; 2013.

    • Search Google Scholar
    • Export Citation
  • 4.

    BusinessWire.com. CMS Officially Recognizes NCCN Drugs & Biologics Compendium. 2008. Available at: http://www.businesswire.com/news/home/20080605006126/en/CMS-Officially-Recognizes-NCCN-Drugs-Biologics-Compendium. Accessed March 23, 2016.

    • Search Google Scholar
    • Export Citation
  • 5.

    Compendia as Authoritative Sources for Use in the Determination of a “Medically Accepted Indication” of Drugs and Biologicals Used Off-Label in an Anti-Cancer Chemotherapeutic Regimen in MLN Matters. Available at: https://www.cms.gov/Outreach-and-Education/Medicare-Learning-Network-MLN/MLNMattersArticles/downloads/mm6191.pdf

    • Search Google Scholar
    • Export Citation
  • 6.

    Centers for Medicare & Medicaid Services. Oncology Care Model (OCM) Request for Applications (RFA). February 2015. Available at: http://innovation.cms.gov/Files/x/ocmrfa.pdf. Accessed March 23, 2016.

    • Search Google Scholar
    • Export Citation

If the inline PDF is not rendering correctly, you can download the PDF file here.

Daniel G. Coit, MD, is co-leader of the Melanoma Disease Management team at Memorial Sloan Kettering Cancer Center, attending surgeon at Memorial Hospital, and Professor of Surgery at the Weil-Cornell University Medical School.

Dr. Coit lectures extensively and has authored or coauthored over 200 articles and abstracts that have appeared in publications including the Journal of Clinical Oncology, Cancer, the Annals of Surgery, and the Annals of Surgical Oncology.

Dr. Coit is a Fellow of the American College of Surgeons. He is a member of several professional societies, including ASCO, the Society of Surgical Oncology, and the American Surgical Association. He is currently Vice President of the Society of Surgical Oncology and serves on the AJCC Melanoma and Upper GI Staging Committees.

Dr. Coit is Chair of the NCCN Clinical Guidelines Panel for Melanoma and a member of the Editorial Board for JNCCN. He was recognized by NCCN for his dedication to multidisciplinary care and commitment to the development of evidence-based guidelines with the 2015 Rodger Winn Award.

The ideas and viewpoints expressed in this editorial are those of the author and do not necessarily represent any policy, position, or program of NCCN.

  • View in gallery

    NCCN Evidence Blocks. (A) Categories and definitions. (B) Sample evidence block. (C) NCCN Evidence Blocks as applied to adjuvant therapy options for completely resected stage III melanoma. Modified from Coit DG, Thompson JA, Algazi A, et al. NCCN Clinical Practice Guidelines in Oncology: Melanoma, NCCN Evidence Blocks. (A, B) Page EB-1. (C) Page ME-4. Accessed March 23, 2016.

  • 1.

    About NCCN. Available at: www.nccn.org. Accessed March 23, 2016.

  • 2.

    Committee on Quality of Health Care in America; Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001.

    • Search Google Scholar
    • Export Citation
  • 3.

    Levit L, Balogh E, Nass S, Ganz PA, eds; Committee on Improving the Quality of Cancer Care: Addressing the Challenges of an Aging Population; Board on Health Care Services; Institute of Medicine. Delivering High-Quality Cancer Care: Charting a New Course for a System in Crisis. Washington, DC: National Academies Press; 2013.

    • Search Google Scholar
    • Export Citation
  • 4.

    BusinessWire.com. CMS Officially Recognizes NCCN Drugs & Biologics Compendium. 2008. Available at: http://www.businesswire.com/news/home/20080605006126/en/CMS-Officially-Recognizes-NCCN-Drugs-Biologics-Compendium. Accessed March 23, 2016.

    • Search Google Scholar
    • Export Citation
  • 5.

    Compendia as Authoritative Sources for Use in the Determination of a “Medically Accepted Indication” of Drugs and Biologicals Used Off-Label in an Anti-Cancer Chemotherapeutic Regimen in MLN Matters. Available at: https://www.cms.gov/Outreach-and-Education/Medicare-Learning-Network-MLN/MLNMattersArticles/downloads/mm6191.pdf

    • Search Google Scholar
    • Export Citation
  • 6.

    Centers for Medicare & Medicaid Services. Oncology Care Model (OCM) Request for Applications (RFA). February 2015. Available at: http://innovation.cms.gov/Files/x/ocmrfa.pdf. Accessed March 23, 2016.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 114 83 2
PDF Downloads 23 16 0
EPUB Downloads 0 0 0