The Elusive Definition of Quality Care

It is nearly impossible to arrive at a concise, comprehensive, and reproducible definition of quality medical care. To paraphrase an old saw, “Although it is difficult to define, we all know when we do (or don't) see it.” In 2001, the Institute of Medicine (IOM) made an initial attempt to define the 6 key elements of quality medical care. These elements included care that was effective, efficient, safe, timely, patient-centered, and equitable.² Although few would argue with these tenets of quality, each describes a very subjective continuum with no absolute reference point, each quite difficult to measure in a strictly quantifiable fashion. This posed a challenge, as, in the words attributed to Sir William Thomson (Lord Kelvin), “If you cannot measure it, you cannot improve it.” In 2013, the IOM made another more detailed attempt to define a list of components of quality care relevant to patients with cancer, including components that should be discussed as part of a cancer care plan, such as treatment goals; initial plan for treatment and proposed duration; treatment benefits and harms; and estimated total and out-of-pocket costs of cancer treatment³ (a more detailed list is available in Box 3-3 of “Delivering High-Quality Cancer Care: Charting a New Course for a System in Crisis”).³ Although not all of these components apply to every patient, as a group they define an interesting list of “talking points” to be addressed with patients in the collaborative pursuit of quality cancer care.

Another challenge in the attempt to define quality medical care is the question of whether quality is an absolute or a relative value. Are there definable thresholds for quality? Is achievement of quality care subjective or objective, and as judged by whom? Are the standards necessary to achieve high-quality cancer care at NCCN Member Institutions the same as those required to achieve high-quality cancer care in rural or inner-city America, or in low-resource settings here or in other countries? And perhaps most controversially, should cost and value ever be included in the definition of quality medical care? If the same result can be achieved at lesser cost, can that be defined as better quality and better value?

NCCN has a strong record of recognition as a leader in quality cancer care. In 2008, the NCCN Drugs & Biologics Compendium was recognized by the Centers for Medicare & Medicaid Services (CMS) (and several major third-party payers) as a “mandated reference for establishment of coverage policy and coverage decisions.”^4,5 In 2015, CMS issued a Request for Application for an Oncology Care Model, encouraging reporting of cancer care that is “consistent with clinical guidelines of… the National Comprehensive Cancer Network (NCCN).”⁶ But can we do better going forward?

Guidelines: A Recipe for Quality?

A cookbook recipe consists of a set of precise instructions to deliberately combine a list of ingredients in specific amounts and order, often with the application of a specific amount and duration of heat, to achieve a predictably reproducible result. In many respects, a recipe represents a physics experiment. Adherence to the very specific conditions defined should always produce an identical result. Quality medical care is anything but a 2-dimensional recipe. Dimensions of resource availability, physician expertise, patient preferences and prejudices, stress, varying educational levels, cost, and value are impossible to capture in a flat pathway or guideline.

Effective guidelines must avoid being too prescriptive, but rather should emphasize appropriate evidence-based options for evaluation and treatment, with strategies for how to arrive at appropriate individual patient recommendations. The cornerstone of this task is to splice anxiety-infused patient preferences into what relevant facts are known. A few examples will illustrate this challenge.

The decision about prophylactic contralateral mastectomy (PCM) for patients with breast cancer and no known genetic predisposition to develop subsequent breast cancer is one often fraught with emotion. Effective guidelines should encourage a patient-physician discussion on the evidence-based quantitative estimate of risk of contralateral breast cancer, what is known about the impact of PCM on the risk of dying of breast cancer in the context of competing health risks, patient anxiety associated with ongoing radiologic screening of the contralateral breast, patient concerns about cosmesis and symmetry, and what is known generally about patient satisfaction after PCM across multiple domains. No one would describe this conversation as efficient, but all would describe it as high-quality medical care.

Although guidelines to support quality medical care cannot be overly prescriptive, they must, if they are to be useful, strive to avoid the trap of being too comfortably vague. For example, when the NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines) for Melanoma address the issue of whether to perform a sentinel lymph node biopsy (SLNB), verbs such as “consider” or “discuss and offer” are used. There is an expectation that the decision will be tempered by “clinical judgment.” Imagine how much more useful these guidelines would be, especially to nonexperts (perhaps the practitioners most likely to benefit from guidelines that help determine whether to perform SLNB on a given patient), if they also included a discussion of factors that contribute to clinical judgment. These might be in the form of specific talking points, including the probability of a positive sentinel node, the fact that SLNB is prognostic but may not be therapeutic, the small but finite morbidity of the procedure, the patient's nonmelanoma comorbidities and competing risks, and even the anticipated incremental cost to the patient for this staging procedure (usually something we know very little about until after the fact).

Another more generic example of how difficult it is to define guideline-based quality care comes from the realm of adjuvant therapy. Again, the NCCN Guidelines for Melanoma list a number of options for adjuvant therapy after complete resection of node-positive disease. Against the background of a very broad range of risk of recurrence in this heterogeneous group, the guidelines list several options, including interferon, biochemotherapy, and high-dose ipilimumab. Although the ranking of these adjuvant therapy options is addressed somewhat in the footnotes and discussion, the guidelines fall short of synthesizing these points into a useful framework for nonexperts in the field. A more interactive guideline structure would empower both practitioner and patient to directly access clinical trial reports, contrasting expert points of view in areas of controversy, consensus summary statements, and NCCN panel recommendations. A bullet list of points to be addressed in a clinical discussion about adjuvant therapy might include addressing the clinical end point most valued by the patient, a sense of the patient's tolerance to treatment-related toxicity, and cost. For skeptics of the routine use of adjuvant therapy, one bullet might acknowledge that, if there is a 10% treatment-related improvement in a specified clinical end point, then necessarily 90% of patients receiving the treatment will experience all of the cost and toxicity with no benefit defined by that end point. Furthermore, in the absence of effective biomarkers predicting clinical benefit, patients need to understand that clinical trial results apply to groups of patients and may not impact them specifically.

Metrics of Guideline-Based Quality Care

There is no single metric that defines quality care for all patients in all situations. High quality in the example of PCM is best characterized as an honest bidirectional discussion about all of the issues enumerated. This interaction is infinitely more difficult to capture as a metric of care than would be the incidence of performing PCM, but it is also considerably more important. The metric of quality in the example of whether to perform SLNB is not the easily measured end point of how often SLNB was performed for a given stage of melanoma, but rather how thorough and understandable was the discussion of the talking points. In the example of whether to administer a specific adjuvant regimen in stage III melanoma, the optimal metric of guideline-based quality care is not whether adjuvant therapy was administered but rather how thorough and understandable was the discussion that informed the final adjuvant therapy decision. Although critical to our understanding of quality, the effectiveness of these discussions is not easily measured. In words that have been attributed to Albert Einstein, “Not everything that counts can be counted; not everything that can be counted counts.”

NCCN Guideline Initiatives

The NCCN Guidelines process is evolving from the simple Categories of Evidence and Consensus stratification of recommendations. Most recently, in an attempt to derive value-based recommendations and further ensure quality, certain disease site panel members have been asked to further categorize the data supporting selected therapeutic interventions with NCCN Evidence Blocks. This grading system incorporates panel member assessment of a treatment option on 5 characteristics: efficacy, safety, quality of evidence, consistency of evidence, and affordability. Using a scale of 1 through 5, these Evidence Blocks result in a graphic display of the panel members' assessments. The resulting Evidence Blocks are being integrated into the guidelines (Figure 1). The methodology behind the Evidence Blocks continues to evolve, and the reliability, reproducibility, and clinical utility of this new scoring system remain undefined.

Another exciting initiative that has the potential to address many of the limitations of the flat format of the current guidelines is being explored. This concept would offer the potential for the current text-based guideline infrastructure to be replaced with a synoptic database architecture, in which each guideline node could be updated as new information becomes available, without requiring publication of an entire new version.

As this new platform matures, the guidelines of tomorrow should be much more useful to both clinicians and patients in their shared goal of improving the overall quality of care. One envisions an interactive multimedia online guideline process, in which every aspect of care—from initial diagnosis and staging through palliative and end of life care—would be represented by a “node” in a pathway. Behind every node, just a mouse click or two away, would be a referenced discussion about the

View Full Size

NCCN Evidence Blocks. (A) Categories and definitions. (B) Sample evidence block. (C) NCCN Evidence Blocks as applied to adjuvant therapy options for completely resected stage III melanoma. Modified from Coit DG, Thompson JA, Algazi A, et al. NCCN Clinical Practice Guidelines in Oncology: Melanoma, NCCN Evidence Blocks. (A, B) Page EB-1. (C) Page ME-4. Accessed March 23, 2016.

Citation: Journal of the National Comprehensive Cancer Network J Natl Compr Canc Netw 14, 4; 10.6004/jnccn.2016.0044

If these ideas seem a bit fantastical, they should. Few of us in that room 20 years ago could have envisioned the extraordinary and truly impactful accomplishments NCCN would realize in its first 2 decades. These accomplishments were driven by leaders with the vision to see how NCCN could truly contribute to its mission “to improve the quality, effectiveness, and efficiency of cancer care.” A major challenge in the decades ahead is leveraging new technology to adapt the guideline process to support and inform the infinite complexity of quality clinical care of patients with cancer.