Clinical decision support (CDS) systems include any electronic system designed to directly aid clinical decision-making by using individual patient characteristics to generate patient-specific assessments or recommendations.1,2 These systems require computable biomedical knowledge, person-specific data, and a reasoning or inferencing mechanism that combines knowledge and data to generate and present information to clinicians as care is being delivered.3 Examples of CDS tools include computerized alerts and reminders, clinical guidelines, condition-specific order sets, focused patient data reports, documentation templates, diagnostic support, and contextually relevant reference information. Thus, CDS has the potential to drive evidence-based standardization of cancer care, improving care delivery and patient outcomes. CDS tools have been incorporated across the patient care spectrum, encompassing prevention, diagnosis, and clinical monitoring. Common roles for CDS include computerized physician order entry (CPOE) and electronic health record (EHR) clinical reminder systems.2
CDS improves healthcare process measures; however, data demonstrating their effectiveness on clinical outcomes and costs are limited.1 Accordingly, real-world uptake of CDS systems has been modest at best.4 Benefits of CDS include improved efficiency and quality of healthcare delivery and access to medical data; enhanced communication; and potential cost savings.5–14 In 2007, the American Medical Informatics Association (AMIA) sounded a call to action regarding CDS implementation that included 3 pillars for fully realizing the promise of CDS: (1) best knowledge available when needed; (2) high adoption and effective use; and (3) continuous improvement of knowledge and CDS methods.4 Furthermore, the Agency for Healthcare Research and Quality stated that the question is not whether CDS systems should be designed and implemented, but rather how to make it easy to do the right thing.15 Nevertheless, effective implementation of a CDS system is a major undertaking, considering the vast amount of clinical data and its variability, availability, and structure across facilities.
Clinical oncology is a dynamic, multidimensional healthcare specialty with complex decision-making and care coordination needs and multiple handoffs between primary and specialty care providers.16 In 2013, the Institute of Medicine reported that the cancer care delivery system was in crisis due to a lack of patient-centric care, palliative care, and evidence-based decision-making.17 CDS systems have the potential to significantly improve cancer care delivery, but there are critical gaps in the availability and use of effective CDS tools.18 To better understand the current landscape of CDS systems in oncology practice, we conducted a systematic review of the literature describing real-world implementation of CDS tools for the diagnosis, treatment, and supportive care of patients with cancer. Our objective was to investigate the reported impact of CDS systems on clinically relevant patient outcomes.
Study Rationale and Definition
We critically appraised and synthesized the published medical literature to answer the objective question, “What evidence supports the use of CDS systems for diagnosis, treatment, and supportive care in clinical oncology?” A CDS system was defined as any electronic system in which characteristics of individual patients are used to generate patient-specific assessments or recommendations that are then presented to clinicians to help with clinical decision-making.1
Inclusion and Exclusion Criteria
Inclusion/exclusion criteria were determined a priori. We performed a systematic review of peer-reviewed, English-language studies published between January 1, 1995, and December 31, 2016, that evaluated the implementation of an electronic CDS system in the field of cancer care. Included studies assessed ≥1 of the following: clinical outcomes (including patient-reported outcomes [PROs]), costs, and care delivery process measures (eg, medication prescribing error rates or compliance with clinical practice guidelines [CPGs]). Studies of decision aids intended primarily for patient use were excluded. Of the 120 studies identified before 1995, none met the predefined inclusion criteria and therefore they were not included in this analysis.
Literature Search Strategy
In a manner consistent with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement, we conducted a comprehensive review of the clinical and scientific literature.19 A trained health sciences librarian (B.A.O.B.) performed a comprehensive search of the literature to identify studies meeting the inclusion criteria. Databases searched were: PubMed, Embase, Academic Search Premier, Web of Science, and Inspec. Relevant narrative reviews, systematic reviews, and meta-analyses were also evaluated for background information, but were not included in this study.
In PubMed, the MeSH terms defined the concepts of cancer and CDS systems, medical order entry systems, or clinical pathways. For optimal retrieval, all terms were supplemented with relevant title and text words. Full PubMed search parameters are available in supplemental eAppendix 1 (available with this article at JNCCN.org). Search strategies for Embase, Academic Search Premier, Web of Science, and Inspec were adjusted for the syntax appropriate for each database. Published reports in the peer-reviewed literature were identified, and all abstracts were reviewed for eligibility. Full-text articles were retrieved and reviewed when additional information was needed. Bibliographies from selected key articles, relevant review articles, and related meta-analyses were reviewed to identify additional publications.
Screening and Full-Text Data Extraction
The study team made every effort to identify all publications meeting the inclusion criteria. Two investigators reviewed each title and abstract for potential inclusion. Titles and abstracts were excluded based on mutual agreement; those not mutually determined to be included or excluded were adjudicated by the third investigator. Each publication that was not excluded after review of the title and abstract was then subjected to a second round of full-text review by 2 members of the study team in the same manner as before, again with adjudication of discordance by the third investigator.
Data from included studies were abstracted from full-text publications using a data abstraction form that included the study objective and design, intervention, number of subjects, study results and conclusions, location of study by country, number of study facilities (single site vs multisite), setting (inpatient vs outpatient), patient diagnosis, subject age range, CDS tool type (stand-alone, EHR-embedded) and name, study or system features, funding source, and conflict of interest declaration.
Electronic database searches yielded 2,439 potentially eligible papers (Figure 1). After review and screening of titles and abstracts, 83 full-text articles were assessed for eligibility, and 24 studies were included in the final analysis.
Table 1 describes the characteristics of the included studies. Most were conducted in adults (96%) in the United States (38%) or Europe (38%) and across any cancer diagnosis (71%), and evaluated a variety of outcomes, including patients, clinicians, and clinician hours. The most common study outcomes were chemotherapy orders (N=76,619) or chemotherapy order sets (N=9,838). Most studies were conducted at a single facility (79%), 87% used quasi-experimental designs, and only 13% (n=3) were randomized controlled trials. Table 2 describes individual study sample sizes, study design, number of facilities, and diagnoses, and supplemental eTable 1 summarizes the objectives and outcomes of each included study.
Characteristics of Studies Assessing CDS Systems in Oncology Clinical Care
Rate of prescription errors was the primary study outcome in 9 of the 12 studies evaluating CPOE.20–31 Although errors were defined differently among studies, prescription errors were reduced in all of them20,23–25,27–31: 1 evaluated medication-related safety events, demonstrating fewer events with use of CDS22; 2 evaluated pharmacy workflow, with 1 showing decreased workflow fragmentation and increased continuous task time with CPOE26; and another study showed increased order review time with CPOE.21
CDS systems that could be classified as clinical pathways for care delivery processes were assessed in 6 of the 24 evaluated studies.32–37 The primary outcome for each of these studies was the association of the clinical pathway CDS system with receipt of guideline-concordant or pathway-recommended care. In general, the outcomes associated with use of clinical pathway systems were compared with usual care; however, one study compared use of an electronic clinical pathway with a paper pathway.34 Most studies were framed as reporting favorable outcomes, including reduced acute care use,32,35 increased guideline concordance and reduced symptoms,33 improved identification of eligible subjects for clinical trial participation,36 and improved hemoglobin levels among patients with anemia.37 The comparison between web- and paper-based pathway systems did not show a significant difference in pathway deviations between them.34
The remaining 6 studies assessed CDS systems in CPGs (n=2),38,39 PROs (n=3),40–42 and oncology-specific prescriber alerts (n=1).43 One study evaluated CPG concordance with CDS and demonstrated a significant increase in guideline adherence (P<.0001) accompanied by treatment plan modification in 31% of cases, and a 50% increase in clinical trial accrual.38 Furthermore, adherence to CPGs with CDS system advice was >90% when the CDS tool included CPG recommendations.39 CDS tools for obtaining PROs and reporting the results to clinicians showed increased discussion of symptoms and quality-of-life issues (P=.03) and symptom monitoring during routine clinical care.40–42 Patients reported a high ease of use and minimal time required; however, patient satisfaction was similar between intervention and control groups. One study evaluated prescriber alerts with CDS tools and demonstrated that hard stops for hepatitis B screening before chemotherapy treatment were associated with increased screening (99.3% vs 40.2%; P<.001) and chemoprophylaxis rates (95.8% vs 39.2%; P<.001) and a reduction in severe exacerbations of liver disease.43
This systematic review assessed evidence supporting the use of CDS systems in cancer care delivery from studies evaluating cancer diagnosis, treatment, and supportive care. Importantly, most of our findings were consistent with those of previous non–oncology-focused studies showing that CDS systems improve care process measures. However, one study did not demonstrate favorable findings, and some of the findings were not of significant magnitude.28 Overall, these data demonstrate the potential value that CDS systems can bring to oncology clinical practices.
The largest category of studies included in our review comprises studies evaluating the use of CPOE systems in oncology care. These studies demonstrated a positive impact on prescriber errors, safety events, and workflow.20–31 CDS tools implementing clinical pathways,32–37 CPGs,38,39 PROs,40–42 and prescriber alerts43 were also primarily associated with positive outcomes, including reduced hospital stays,35 increased guideline use and concordance,33,38,39 enhanced identification of trial-eligible patients,36 and improvements in symptom management.41 Overall, the studies suggest that clinical oncology–focused CDS systems appear to be well accepted and are associated with potentially meaningful improvements in patient care.
Two systematic reviews that assessed CPOE for inpatients in medicine or intensive care units12 and at the point of care in any clinical setting13 demonstrated a clear benefit to implementing CDS. However, one study in our analysis showed that there was an increase in pharmacist order review time without any impact on intervention rates.21 This finding underlines the potential for CDS systems to have a negative impact on care delivery, and demonstrates the importance of a thorough evaluation of these tools as a part of system implementation.
CDS systems incorporated into clinical pathways have been associated with increased guideline adherence,32–37 demonstrating the benefit they can provide to clinicians, and consistent with findings of previous systematic reviews also showing a positive impact on guideline adherence.10,44 Three studies included in our analysis evaluated the use of CDS systems for PROs, and all demonstrated benefit for ≥1 outcome.40–42 These findings differ somewhat from those of a systematic review of 15 studies that assessed the effect of CDS systems on PROs, which showed a positive effect on symptoms in 3 studies (20%).45 A CDS system used with prescriber alerts demonstrated a positive impact, which is consistent with findings of a previous study.43,46
The findings are also consistent with those of a meta-analysis assessing the impact of health information technology (HIT) on cancer care from 2000 to June 2014.47 CDS systems were the most common (66%) HIT intervention identified and were implemented across several cancer types, including breast, colorectal, and prostate, for detection, diagnosis, and treatment but not for survivorship or end-of-life care. The primary findings show that the beneficial impact differed across the cancer continuum; HIT for diagnosis and treatment was less likely to be associated with benefit compared with prevention. Likewise, HIT targeting behavioral change is less likely to be beneficial compared with HIT targeting improved decision-making. The complexity of diagnosis and treatment, the volume of information needed, and the factors associated with behavior change were given as potential reasons for these findings. Within the CDS systems, key factors that appear to contribute to improved outcomes include the use of real-time provider alerts and point-of-care action on prescription orders and provision of information to clinicians that CDS systems can provide.20,27,32,36,37,40–42 There appear to be resulting factors that create new challenges, such as the need to access separate systems and otherwise increase work time of prescribers or other downstream clinicians, that may reduce outcomes.21,26,34,35
Our systematic review is reported >10 years after a call to action by the AMIA regarding the use of CDS.4 The call included directives for achieving desirable levels of patient safety, care quality, patient-centeredness, and cost-effectiveness. Health systems were called on to optimize CDS to improve the quality of healthcare services and health in the United States. However, not all studies evaluating CDS systems have shown clinical practice improvements.48 Thus, it is imperative that systems and tools, both commercially and locally developed, be assessed for their effectiveness and impact on patient outcomes. Furthermore, we believe it is essential that these assessments also involve outcomes, including those associated with clinical care and costs of system implementation, when possible.
Kowamoto et al8 identified features of CDS systems critical for improving clinical practice in any clinical setting, including (1) automatic provision of decision support as part of clinician workflow; (2) provision of recommendations compared with assessments; (3) provision of decision support at the time and location of decision-making; and (4) computer-based decision support. CDS systems with all 4 features were associated with significant improvements in clinical practice.
A major gap in CDS system use exists across the spectrum of clinical oncology care, and further development of CDS tools is warranted. Findings from the limited studies included in this review are largely positive and consistent with most studies conducted in the general healthcare population. However, many of the included studies used suboptimal study designs, and only 3 were randomized controlled trials. Robust quasi-experimental study designs, such as interrupted time series designs with a comparator group, were rare. Currently, <15 studies assessing the impact of CDS systems are underway or accruing, according to ClinicalTrials.gov.49 A recent study of CDS in patients with lung cancer showed a reduction in inappropriate granulocyte colony-stimulating factor use without an increase in febrile neutropenia rates, illustrating the positive impact these powerful tools can have on clinical oncology care.50 Further analyses of the use of these tools with appropriate study designs and analytic methods are necessary to build the case for wider implementation of CDS systems.
Directions for Future Research
A number of gaps in the current literature were identified during this analysis, including a lack of information related to the specific barriers, facilitators, and implementation strategies associated with CDS system implementation. The studies did not specify the potential challenges facilities may encounter when implementing similar systems or describe identified best practices to support successful implementation or outcomes. In addition, the impact of CDS systems on patient mortality, healthcare costs, or costs associated with implementation and management was also not assessed. Furthermore, outcome assessments for each study did not include overall impact on patient mortality or the impact of system implementation on healthcare costs to the institution, payer, or patient. The costs of system maintenance, upgrades, or enhancements were also not described. These are all crucial elements that must be assessed and understood for optimal implementation to occur.
Study Strengths and Limitations
Strengths of this systematic review include the study team, comprising a research librarian, 2 practicing oncologists, and an oncology clinical pharmacist. The review was conducted according to the PRISMA statement to ensure appropriate methods were used. The potential exists for bias, including searching, exclusion criteria, assembling, and publication, although all efforts were made to minimize this where possible. The total number of CDS systems that have been studied, had no outcome improvement, and were subject to publication bias is unknown.
To date, few studies have evaluated CDS systems in oncology practice. Comparative studies that report outcomes of care include those of CDS systems with CPOE, clinical pathways, CPGs, PROs, and provider alerts. Published studies of CDS systems are largely associated with positive outcomes, and their findings consistent with those of studies conducted in the noncancer population. Key features that appear to support positive outcomes include real-time information and point-of-care action. CPOE showed a small impact on provider behavior, with general alerts and increased pharmacist order review times. Potential drawbacks may be the need to access separate systems and increased workflow for prescribers or other clinicians. None of the studies assessed the impact of CDS systems on patient survival. There is a critical need for CDS systems development and well-designed studies to demonstrate improvement in patient outcomes, including impact on survival and efficiency of cancer care delivery.
The authors wish to acknowledge the guidance of Terry Field and Diana Buist, members of the Health Care Systems (HCS) Cancer Research Network (CRN) and CRN Scholar Program Mentors, who contributed to the development of this project. The overall goal of the CRN is to conduct collaborative research to determine the effectiveness of preventive, curative, and supportive interventions for major cancers that span the natural history of those cancers among diverse populations and health systems. HealthPartners Institute is an Affiliate Member of the HCS Research Network. The authors would also like to acknowledge Mandy Fraser, Amanda Lacy, and Lauryn Davin, who provided technical assistance during the literature review process and with manuscript preparation and submission.
Author contributions: Study concept and design: Pawloski, Brooks, Nielsen. Provision of study materials: Olson-Bullis. Acquisition and assembly of data: Pawloski, Olson-Bullis. Data analysis and interpretation: Pawloski, Brooks, Nielsen. Manuscript preparation and final approval: All authors.
Disclosures: The authors have not received any financial consideration from any person or organization to support the preparation, analysis, results, or discussion of this article.
Funding: This work was supported by a grant from the NCI (U24 CA171524; PI: L. Kushi), with no additional funding provided specifically for this work.
Clinical Decision Support. HealthIT.gov website. Available at: https://www.healthit.gov/policy-researchers-implementers/clinical-decision-support-cds. Accessed March 13 2018.
eHealth for a Healthier Europe! Opportunities for a better use of healthcare resources. Available at: https://joinup.ec.europa.eu/sites/default/files/document/2014-12/eHealth%20for%20a%20Healthier%20Europe%20-%20Opportunities%20for%20a%20better%20use%20of%20healthcare%20resources.pdf. Accessed March 13 2018.
Berner ES. Clinical Decision Support Systems: State of the Art. AHRQ Publication No. 09-0069-EF. Rockville MD: Agency for Healthcare Research and Quality; 2009.
Seidman AD. Computer-assisted decision support in medical oncology: we need it now. The ASCO Post April 10 2016.
Beriwal SRajagopalan MSFlickinger JC. How effective are clinical pathways with and without online peer-review? An analysis of bone metastases pathway in a large, integrated National Cancer Institute-Designated Comprehensive Cancer Center Network. Int J Radiat Oncol Biol Phys 2012;83:1246–1251.
Van Erps JAapro MMacDonald K. Promoting evidence-based management of anemia in cancer patients: concurrent and discriminant validity of RESPOND, a web-based clinical guidance system based on the EORTC guidelines for supportive care in cancer. Support Care Cancer 2010;18:847–858.
U.S. National Library of Medicine. ClinicalTrials.gov. Available at: https://clinicaltrials.gov/ct2/home. Accessed September 13 2018.