NCCN Roundtable: What Are the Characteristics of an Optimal Clinical Practice Guideline?

Authors: David S. Ettinger MD, Michael Kuettel MD, MBA, PhD, Jennifer Malin MD, PhD, Joan S. McClure MS, Mary Lou Smith JD, MBA, Andrew D. Zelenetz MD, PhD, and F. Marc Stewart MD
View More View Less

Much has changed in the treatment of cancer since the first NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines) were rolled out for 8 different tumor types in November 1996. NCCN Guidelines now include involved algorithms often containing multiple treatment alternatives and detailed pathways of care that depend on more-specific patient characteristics and molecular tumor diagnostics. With 47 different individual NCCN panels, all members of the cancer care team are now better informed than ever to guide patients through the often complex decision-making required to improve the odds of successful outcomes. At the NCCN 20th Annual Conference, a distinguished panel assembled to take a closer look at these invaluable clinical practice guidelines, first glancing backward to how it all started and then forward to explore the key ingredients of trustworthy guidelines.

“Clinical guidelines are the heart and soul of the NCCN,” revealed panel moderator F. Marc Stewart, MD, Medical Director of Seattle Cancer Care Alliance; Member of Fred Hutchinson Cancer Research Center; and Professor of Medicine at the University of Washington, Seattle. “It is in part our reason to exist, because ultimately, clinical guidelines help us to deliver the best care to our patients.”

A Look in the Rearview Mirror

First, the panel offered an historical perspective on the NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines) from some of the members who were in the delivery room when they were born. For example, in 1996, the NCCN Guidelines for Lung Cancer were created by a 7-member lung cancer panel and included 40 references, shared David S. Ettinger, MD, Alex Grass Professor of Oncology at the Johns Hopkins University School of Medicine. In 2015, the lung cancer panel has 35 members, including patient advocates, and version 5.2015 contains 718 references. Once the lone patient advocate—on the breast cancer panel—Mary Lou Smith, JD, MBA, co-founder of the Research Advocacy Network and former Director of Patient, Payer, and Government Relations for the Coalition of Cancer Cooperative Groups, noted that patient advocates now sit on many panels. “We have come a long way,” she shared.

The early versions of the guidelines were certainly less complex and sophisticated than their later-day counterparts. “When I first started, the only cancer that we talked about biomarkers for was breast. And that is not true today,” stated Joan S. McClure, MS, Senior Vice President of Clinical Information and Publications at NCCN, who playfully refers to herself as the “stepmother of the guidelines.” About 40 different medical specialties are represented in the 2015 NCCN Guidelines, and principles of chemotherapy, radiation oncology, and surgery have been added.

Dr. Stewart asked the panel whether the scientific method has changed over the past 20 years, and the general consensus was that it has. “There has been a significant evolution into making the guidelines even more evidence based,” declared Andrew D. Zelenetz, MD, PhD, Vice Chair of Medical Informatics in the Department of Medicine at Memorial Sloan Kettering Cancer Center.. In the past, he added, “we did a lot of learning on the job, tackling the most important lymphomas first, for instance.” Dr. Ettinger agreed, suggesting that in the past, we started with rigorous methodology, based on both evidence and consensus. Now, he added, there are many orphan diseases, with few if any phase III randomized trials, resulting in primarily category 2A recommendations for many systemic therapies.

The group also highlighted other differences between past and present guidelines. Among the changes is the current increase in transparency, noted Dr. Ettinger, with conflict of interest statements included for all contributors. Also, a real effort is made to include new U.S. FDA approvals in a timely fashion. In addition, patient versions of the guidelines are now available.

Finally, once only available in print, the NCCN Guidelines are now available online, which Ms. McClure mentioned has made a major change in how they are used. In fact, December 2013 saw a record-high 4.9 million PDF downloads of NCCN Guidelines.

The panel also noted that some things have remained consistent over the years. For instance, the guidelines development process continues to be paid for by membership dues from the NCCN Member Institutions and nothing else, added Dr. Ettinger. And, the guidelines were and continue to be intended as a resource for physicians and now patients, offering both groups more certainty about decision-making.

Ingredients of a Sound Guideline

Dr. Stewart summarized the essential components of a trustworthy guideline, as presented by the Institute of Medicine (IOM) report in January 2015. According to the IOM, the underlying principles include transparency, disclosure of conflicts of interest, multidisciplinary group participation, systematic review of data, determination of the strength of the evidence, and routine review/update of the guidelines. “From the standpoint of these principles, the NCCN is pretty good,” acknowledged Michael Kuettel, MD, MBA, PhD, Professor and Chair of Radiation Oncology at Roswell Park Cancer Institute.

According to Ms. McClure, the IOM and the NCCN concur in most areas but differ in a few. For instance, she noted, the IOM does not recommend the use of experts to develop clinical practice guidelines, but the NCCN believes that experts are essential, given the specialized nature of cancer, and other experts, not the NCCN staff, perform systematic reviews. “If something new and very promising is approved by the FDA, we feel a responsibility to make sure that our panel members review it, decide if and where it fits into the standard of care, and then get it into the compendium, if it does fit, rapidly, so patients have access to these new therapies,” explained Ms. McClure.

‘Not All Data Are Created Equal’

Guidelines offer both clinicians and patients assurance that there is a consensus of expert opinion and a scientific basis for a given treatment approach, particularly with rare cancers, “whether you get your treatment in Des Moines or Los Angeles or Hollywood, Florida,” according to Jennifer Malin, MD, PhD, Medical Director for Oncology Solutions and Innovation at Anthem Inc. and Assistant Professor of Medicine at UCLA School of Medicine. However, the panel agreed that the issue of data quality is a major consideration with any guideline.

In the NCCN Guidelines, approximately 8% of the recommendations are based on a category 1 level of evidence, noted Ms. McClure. However, she added, this is the case “not because there is a high level of data available and we choose to ignore it, but because it is not there.”

Dr. Stewart asked the panel about the role of unpublished data, and the opinions were somewhat mixed. Drs. Malin and Ettinger indicated that it would be nice to have access to unpublished data. “We need to improve access to data that do not make it into the NEJM or JCO,” said Dr. Malin. However, she noted, it is helpful to make a distinction between data and information. “Often we are drowning in data and don't have information.” As for Dr. Ettinger, “any data that we can get, that will enhance our ability to make a recommendation makes sense.”

Dr. Zelenetz was a bit more cautious about the use of unpublished data. “Not all data are created equal,” he clarified. “Richness of data depends on appropriate measurements; if [a study was] done inappropriately, the data may not be useful, and that may be why they were never published.” However, he added, some of the orphan diseases in cancer may never have high-level, randomized trials or adequate data for a high-quality meta-analysis. In such cases, the NCCN Guidelines can offer some help by providing results from small trials and clinical expert experience. “To ignore those guidelines where we don't have that high-level evidence does a disservice to the oncologists and the patients, because it denies patients access to what is expert care.”

Why Focus on Pathways and Biomarkers?

The concept of pathways, which often signal a more selective treatment among a wide range of options in a given guideline, were up next for the panel's consideration. “Guidelines and pathways are tools to improve the quality of care,” said Dr. Malin, but Dr. Zelenetz added that how they are viewed plays a key role in their use. Although pathways are intended to provide quality care, some clinicians may view them as restrictive, with a focus on simply saving money.

With this in mind, Dr. Stewart asked the panel whether payers cared about pathways. The consensus was that some do, although not all payers are using them. In fact, Dr. Kuettel shared the results of an informal survey he took a year ago. “Roughly 25% [of payers] were using them for their organization, and 50% said they planned to incorporate them within the next 2 years,” he revealed.

Turning to the increasing use of biomarkers, Dr. Stewart predicted that more and more branches in the guidelines will center on refined patient subsets, often based on molecular abnormalities. And, according to Dr. Zelenetz, that will depend on the quality of the smaller datasets. For instance, in lymphoma, there is no chance that a patient with a CARD11 mutation will respond to ibrutinib, he added. “When there is a strong parallelism between the biomarker and the biology, then small datasets will be useful,” declared Dr. Zelenetz. “When it is less clear, and there are a bunch of biomarkers that may put a patient at risk, it becomes more of a prognostic type of marker, and larger datasets are needed to know how to use it.”

The New NCCN Evidence Block

Finally, the roundtable briefly addressed the new NCCN evidence block, which will now appear with the systemic therapy recommendations in some of the 2015 Guidelines. Intended to provide more value to the guidelines user and assist in decision-making, the dimensions include efficacy (E), safety (S), quality of evidence (Q), consistency of evidence (C), and affordability (A), and more shading in the tables illustrating these criteria is better.

“The NCCN evidence block puts a lot of detailed information into a very accessible format,” said Dr. Malin. These grids represent a starting point of conversation between clinicians and patients, she added.

“As we move into smaller and smaller subsets of patients, the quality and consistency of the data will become important in the evaluation of treatments,” Ms. McClure predicted. Trials of hundreds and hundreds of patients may become a thing of the past, she said.

Meet the Panelists

F. Marc Stewart, MD, moderator: Medical Director of Seattle Cancer Care Alliance; Member of Fred Hutchinson Cancer Research Center; and Professor of Medicine at the University of Washington, Seattle. Dr. Stewart serves as a member of the Board of Directors of the NCCN and as President of the National Advocate Foundation.

David S. Ettinger, MD: Alex Grass Professor of Oncology at The Johns Hopkins University School of Medicine in Baltimore, Maryland. A member of the Board of Directors of the NCCN, Dr. Ettinger also chairs several NCCN panels, including the Non-Small Cell Lung Cancer, Occult Primary Cancers, and Antiemesis Panels.

Michael Kuettel, MD, MBA, PhD: Professor and Chair of Radiation Oncology at Roswell Park Cancer Institute, Buffalo, New York. In addition to serving on the Health Policy Council of his institution, Dr. Kuettel is a member of the NCCN Board of Directors and a panel member for the NCCN Guidelines for Prostate Cancer.

Jennifer Malin, MD, PhD: Medical Director for Oncology Solutions and Innovation at Anthem Inc. and Assistant Professor of Medicine at UCLA School of Medicine in Los Angeles, California. Dr. Malin serves on the American Society of Clinical Oncology's Quality of Care Committee and the National Quality Forum's Cancer Steering Committee.

Joan S. McClure, MS: Senior Vice President of Clinical Information and Publications at NCCN. Her group develops the NCCN Clinical Practice Guidelines in Oncology, associated NCCN Guidelines for Patients, and the NCCN Drugs & Biologics Compendium. Ms. McClure also serves as an Associate Editor for JNCCN—Journal of the National Comprehensive Cancer Network.

Mary Lou Smith, JD, MBA: Co-Founder of the Research Advocacy Network and former Director of Patient, Payer, and Government Relations for the Coalition of Cancer Cooperative Groups. A two-time breast cancer survivor and ovarian cancer survivor, Ms. Smith serves on the NCCN Breast Cancer Treatment and Screening Guidelines Committees.

Andrew D. Zelenetz, MD, PhD: Vice Chair of Medical Informatics in the Department of Medicine at Memorial Sloan Kettering Cancer Center in New York. Prior to this position, he was Chief of the Lymphoma Service. Dr. Zelenetz serves as the Chair of the NCCN Non-Hodgkin's Lymphomas Panel and a member of the NCCN Acute Lymphoblastic Leukemia Panel.