Background
Oral chemotherapy has been a mainstay of anticancer treatments for decades. From the early years of chlorambucil, cyclophosphamide, and methotrexate, to capecitabine’s approval in 1998, and through imatinib’s inclusion in formularies in 2001, numerous oral anticancer agents have become standard of care for many cancer types.1 As the pace of drug development has recently accelerated, the development of oral agents has also increased. In the last 3 years alone (2019–2021), the FDA approved 44 new cancer therapies, of which 27 (61%) were oral anticancer agents.2–4 Recognizing that patient self-administration of oral chemotherapy presents unique challenges, in 2013 ASCO and the Oncology Nursing Society (ONS) updated standards for safe chemotherapy administration to include standards on oral chemotherapy management.5 Although the broad definition of oral anticancer agents includes chemotherapy, targeted agents, and endocrine therapies, the ASCO/ONS standards (and hence Quality Oncology Practice Initiative [QOPI] measures) focus on oral chemotherapy, which includes all oral anticancer agents with the exception of endocrine therapy.
To evaluate quality of care across medical oncology practice, in 2013 ASCO developed oral chemotherapy quality measures to use for pilot/testing in the QOPI. QOPI is the largest specialty oncology quality measurement registry and reporting infrastructure and contains >150 evidence-based quality measures.6 These measures include core measures (eg, pathologic confirmation of diagnosis) and disease- and domain-specific modules (eg, marker testing in breast cancer). QOPI involves outpatient oncology practices performing retrospective medical record abstraction. Since its inception in 2006, it has served as a tool to foster a culture of self-examination and improvement within one’s practice.7
To date, performance on these oral chemotherapy test measures in QOPI, particularly the rates of clinician documentation of oral chemotherapy plans and administration, have been lower than expected.8,9 Of practices who reported performance on 17 test/pilot measures of oral chemotherapy administration and management in 2012 through 2013, mean practice scores ranged from 66% to 68% for treatment plan documentation, 51% to 57% for patient education, and 75% to 81% for adherence/toxicity monitoring.9 Compared with practices’ performance on other quality measures, such results illustrate a clear opportunity for improvement in documentation of safe oral chemotherapy administration, education, and monitoring. Recognizing the importance of oral chemotherapy measures to highlight an important and ongoing gap, these measures have come out of testing/piloting to become standard measures within QOPI.
The QOPI Certification Program (QCP), launched in 2010, is available to high-achieving QOPI-participating practices that also fulfill additional certification standards related to the safe delivery of chemotherapy. QOPI-certified practices participate in a rigorous on-site survey process, where an oncology professional assesses compliance with the QOPI certification standards through interviews, observation, and medical record review. In June 2013, guidance on oral chemotherapy was incorporated into the certification standards, which requires that QOPI-certified practices have a policy that outlines the procedure to assess patients’ ability to adhere to chemotherapy administered outside the healthcare setting (QCP Standard 4.2), as well as a policy that requires assessment of each patient’s chemotherapy adherence at defined clinically meaningful intervals (QCP Standard 4.3).10
Although oncology practices that participate in QOPI seem to consistently demonstrate longitudinal improvement in measured performance, this has not been formally studied with a focus on implementation of a set of clinically significant metrics. This is a key step in evaluating effectiveness and outcomes of metrics as well as the impact of a certification program. As QOPI-certified practices must have demonstrated compliance with all ASCO/ONS standards for safe chemotherapy administration (including oral anticancer agents), we hypothesize that oncology practices that are QOPI-certified will perform better than non–QOPI-certified oncology practices on oral chemotherapy measures.10–12
Methods
Overall
We performed a cross-sectional analysis of practice-level utilization of 9 oral chemotherapy quality measures in 2017 or 2018, examining the effect of QOPI certification status on quality measure performance rates.
Data Collection
We examined data in the QOPI registry. QOPI is ASCO’s flagship quality measurement and benchmarking registry, with twice-annual data collection periods (spring and fall). Participants receive individual performance scores by practice, site, and provider, as well as benchmarked scores aggregated from all participating practices. Practices retrospectively abstract patient data from a number of medical charts; the number of charts is derived from the practice size, with practices with more oncologists required to abstract more charts.
Disease-specific panels of ASCO volunteers developed the quality measures under the oversight of the ASCO Measures Steering Group, which reports to the Quality of Care Council. QOPI measures are categorized by modules (a grouping of similar measures) and tracks (a grouping of measures from multiple modules), which when combined with core measures (collected on all patients) provides a comprehensive view of the experience of patients with cancer and helps practices meet their reporting needs. To apply for QOPI certification, a practice must participate in QOPI, achieve or exceed a benchmark score on relevant measures, submit an application and fees (between $3,500 and $24,000, depending the on number of oncologist full-time equivalents and number of sites), and perform well in an on-site audit of certification standards focused on the provision of cancer care in the outpatient setting.
Study Description
The study sample comprised all practices that reported QOPI oral chemotherapy measures in 2017 and 2018 and were eligible as QOPI participants. We analyzed results from oncology practices that were voluntarily participating in the QOPI program on 9 measures of oral chemotherapy administration and management in at least 1 of 3 collection periods: spring or fall of 2017, or spring of 2018.
To perform the site-level analysis for this project, QOPI staff analysts reviewed and validated site-level information. Duplicate sites, incorrect country and state fields, and residual sites that did not successfully submit patient charts for the specific round were removed. Practice-site categorical information was then merged with chart-level information, including specific measure scores and location-specific variables.
The oral chemotherapy measures that were included in the analysis are part of the core module and were available for abstraction to all QOPI participants. The 9 measures cover 3 domains: treatment plan documentation, patient education, and adherence/toxicity monitoring (Table 1). Each domain has a composite variable, consisting of 2 to 4 measures. The composite measure for documented plan, as well as one of its components, were still being tested in early 2018, so these were not included in the analyses. The measure definitions were consistent across the 3 collection rounds. Documentation of body surface area (BSA) was included for comparison purposes. As a long-established criterion for treatment dosing, documentation of BSA was expected to demonstrate limited variability across practices and to not differ based on certification status.
QOPI Quality Measures
Statistical Methods
Data were organized by QOPI measure and by practice, so that each row in the dataset was indexed by QOPI measure and practice with number of charts examined and number of charts adhering to the QOPI measure. Utilization of each QOPI measure for each practice was defined as the percent of charts examined that adhered to the QOPI measure. For practices with >1 site, practice information was combined across sites to calculate a representative measure for the practice (eg, median number of oncologists). Practice variables were treated as categorical and summarized using proportions. For each QOPI measure, at least 3 charts had to be submitted by the practice to be included in the analysis dataset. Otherwise, the measure for that practice was removed from the dataset. The distribution of the number of charts examined per measure across practices, and the utilization per measure across practices, are demonstrated using boxplots. Sensitivity analyses were performed to determine whether the results were sensitive to including practices with limited numbers of charts. Specifically, the analysis was repeated under 2 additional constraints: limiting the dataset to practices that had at least 7 charts and limiting the dataset to practices that had at least 10 charts. The results were similar to the results based on the full dataset and are thus not presented.
The relationship between certification and measure utilization was assessed using a mixed-effects logistic regression model. Unlike a standard logistic model, random effects were included to incorporate clustering of utilization within practices. In addition to QOPI certification status, covariates included in the model were data collection round (data were included from spring 2017, fall 2017, and spring 2018), academic status, region (4 US regions and international), and the number of sites (included as a continuous measure, and log-transformed). The odds ratio for certification and its 95% confidence interval are reported and summarized using graphical display of estimates.
All analyses were performed in R/RStudio (R Project for Statistical Computing) and the lme4 library was used for estimation of the mixed-effects logistic regression model.
Results
Practice Characteristics
A total of 192 practices contributed data, representing 2,305 patients with oral chemotherapy measures. As shown in Table 2, 26% of practices were QOPI-certified and 74% were not. Most practices comprised a single site (55%), had a median of ≤5 oncologists (66%), and were not academically affiliated (82%). A total of 96% of the practices were located in the United States and 4% were foreign (Brazil [n=3], Spain [n=3], and Greece [n=1]). The number of charts per practice for each oral chemotherapy quality measure ranged from 3 to 80 (median, 9).
Practice Characteristics
Performance on Quality Measures
Performance of each practice on each quality measure is shown in Figure 1. The variability across practices (as demonstrated by the size of the interquartile ranges) was substantial for many of the oral chemotherapy measures compared with documentation of BSA. Median performance across practices on the individual oral chemotherapy quality measures also varied considerably, from a low of 44% to a high of 100%. Median performance was >80% for education before the start of oral chemotherapy addressing clinic contact instructions (86%); education before the start of oral chemotherapy addressing toxicities (100%); oral chemotherapy plan documenting the administration schedule (83%); oral chemotherapy plan documenting the dose (100%); oral chemotherapy plan documenting the indications (100%); and medication adherence assessed on a visit/contact following oral chemotherapy initiation (87%).
Distribution of performance on quality measures (see Table 1 for quality measures).
Note: Some practices that reported on at least one of the oral chemotherapy measures included in this analysis reported on <3 charts for ≥1 of the measures. As a result, several of the measures are based on reporting from fewer than 192 practices. Additionally, the composite measure for documented plan, as well as one of its components, “documented plan (c): plan provided to patient/caregiver prior to start of therapy and practitioner(s) providing continuing care (PCP) within 3 months of starting therapy,” were still being tested in early 2018, so these were not included in the analyses.
Abbreviations: admin, administration; BSA, body surface area; chemo, chemotherapy; ed, education; Ht, height; sched, schedule; Tx, treatment; Wt, weight.
Citation: Journal of the National Comprehensive Cancer Network 20, 10; 10.6004/jnccn.2022.7024
Distribution of performance on quality measures (see Table 1 for quality measures).
Note: Some practices that reported on at least one of the oral chemotherapy measures included in this analysis reported on <3 charts for ≥1 of the measures. As a result, several of the measures are based on reporting from fewer than 192 practices. Additionally, the composite measure for documented plan, as well as one of its components, “documented plan (c): plan provided to patient/caregiver prior to start of therapy and practitioner(s) providing continuing care (PCP) within 3 months of starting therapy,” were still being tested in early 2018, so these were not included in the analyses.
Abbreviations: admin, administration; BSA, body surface area; chemo, chemotherapy; ed, education; Ht, height; sched, schedule; Tx, treatment; Wt, weight.
Citation: Journal of the National Comprehensive Cancer Network 20, 10; 10.6004/jnccn.2022.7024
Distribution of performance on quality measures (see Table 1 for quality measures).
Note: Some practices that reported on at least one of the oral chemotherapy measures included in this analysis reported on <3 charts for ≥1 of the measures. As a result, several of the measures are based on reporting from fewer than 192 practices. Additionally, the composite measure for documented plan, as well as one of its components, “documented plan (c): plan provided to patient/caregiver prior to start of therapy and practitioner(s) providing continuing care (PCP) within 3 months of starting therapy,” were still being tested in early 2018, so these were not included in the analyses.
Abbreviations: admin, administration; BSA, body surface area; chemo, chemotherapy; ed, education; Ht, height; sched, schedule; Tx, treatment; Wt, weight.
Citation: Journal of the National Comprehensive Cancer Network 20, 10; 10.6004/jnccn.2022.7024
Median performance was <80% for education before the start of oral chemotherapy addressing missed doses (55%); and education before the start of chemotherapy addressing missed doses, toxicities, and clinic contact instructions (composite measure, 44%).
Of note, when assessment of medication adherence indicated nonadherence, the QOPI data entry system was triggered to ask if medication adherence was addressed. Performance on this measure (medication adherence addressed) is inherently skewed because nonadherence to chemotherapy was low. These results are summarized in supplemental eTable 1 and supplemental eFigure 1 (available with this article at JNCCN.org), which also show performance on the related composite measure (medication adherence assessed AND addressed).
Association of QOPI Certification Status With Performance on Quality Measures
Results examining the association of QOPI certification status with performance on the individual quality measures are shown in Table 3. A significant relationship (as demonstrated by an adjusted odds ratio that exceeded the 95% confidence interval) was observed for the measure assessing oral chemotherapy education addressing clinic contact instructions. Consistent with predictions, certified practices were approximately 5 times more likely to provide education about clinic contact instructions than noncertified QOPI practices. Findings on performance for all the other oral chemotherapy measures were in the predicted direction of favoring certified practices (Figures 2 and 3) but were not statistically significant, likely because of the wide distribution of performance across practices and small sample size. There were no differences seen that correlated with practice size, affiliation, geography, or other practice characteristics.
Tests of Associations Between Certification Status and Performance on Quality Measures
Adjusted odds ratio estimates and 95% confidence intervals for comparisons of certified and uncertified practices (see Table 1 for quality measures).
Abbreviations: admin, administration; BSA, body surface area; chemo, chemotherapy; ed, education; Ht, height; sched, schedule; Tx, treatment; Wt, weight.
Citation: Journal of the National Comprehensive Cancer Network 20, 10; 10.6004/jnccn.2022.7024
Adjusted odds ratio estimates and 95% confidence intervals for comparisons of certified and uncertified practices (see Table 1 for quality measures).
Abbreviations: admin, administration; BSA, body surface area; chemo, chemotherapy; ed, education; Ht, height; sched, schedule; Tx, treatment; Wt, weight.
Citation: Journal of the National Comprehensive Cancer Network 20, 10; 10.6004/jnccn.2022.7024
Adjusted odds ratio estimates and 95% confidence intervals for comparisons of certified and uncertified practices (see Table 1 for quality measures).
Abbreviations: admin, administration; BSA, body surface area; chemo, chemotherapy; ed, education; Ht, height; sched, schedule; Tx, treatment; Wt, weight.
Citation: Journal of the National Comprehensive Cancer Network 20, 10; 10.6004/jnccn.2022.7024
Unadjusted odds ratios (see Table 1 for quality measures).
Abbreviations: admin, administration; BSA, body surface area; chemo, chemotherapy; ed, education; Ht, height; sched, schedule; Tx, treatment; Wt, weight.
Citation: Journal of the National Comprehensive Cancer Network 20, 10; 10.6004/jnccn.2022.7024
Unadjusted odds ratios (see Table 1 for quality measures).
Abbreviations: admin, administration; BSA, body surface area; chemo, chemotherapy; ed, education; Ht, height; sched, schedule; Tx, treatment; Wt, weight.
Citation: Journal of the National Comprehensive Cancer Network 20, 10; 10.6004/jnccn.2022.7024
Unadjusted odds ratios (see Table 1 for quality measures).
Abbreviations: admin, administration; BSA, body surface area; chemo, chemotherapy; ed, education; Ht, height; sched, schedule; Tx, treatment; Wt, weight.
Citation: Journal of the National Comprehensive Cancer Network 20, 10; 10.6004/jnccn.2022.7024
Discussion
In this study of practices participating in ASCO’s QOPI program, we found wide variability in performance related to oral chemotherapy measures across practices; performance for some measures spanned the full range of 0% to 100% (eg, oral chemotherapy education about missed doses and clinic contact instructions). We also saw wide variability in performance across the various measures, with the median performance on individual measures ranging from 44% to 100%. The wide ranges in performance across practices and across measures shows that the implementation of standard procedures for administration of oral chemotherapy is not an easy task. Of note, abstraction of measures is dependent on the availability of documentation in the electronic medical record (EMR).
Using real-world data, we further sought to determine whether QOPI certification was associated with better quality of care, evaluated based on performance on relevant QOPI measures. We found that QOPI-certified practices had higher odds than those not certified of performing well on the measure regarding documentation of clinic contact instructions for patients starting oral chemotherapy. However, certification status was not significantly associated with performance on any of the other measures assessed in this study. As expected, documentation of BSA prior to chemotherapy—our reference measure—was not associated with certification status and demonstrated limited variability across practices, consistent with the fact that these data have long been established as critical components of oncologic care for treatment dosing.
As oncologists, we assume the responsibility of ensuring that our patients receive their treatment in our offices as scheduled. However, based on our results, it is clear that we fail to teach patients how to take on this responsibility when they are self-administering their treatment at home. Our findings highlight the need for the development and implementation of appropriate standards that apply to oral chemotherapy and address the complexities that set it apart from parenteral treatment. In the meantime, there are existing tools and resources clinicians can use to track adherence to oral chemotherapy, including some that are available through EMR software.
Recognizing this issue, the standardization and improved documentation of oral chemotherapy practice processes has been the focus of quality improvement projects implemented by several individual practices and practice groups nationwide.13–15 In a small single-institution study, the Plan-Do-Study-Act (PDSA) methodology was used to develop and implement a workflow for oral chemotherapy initiation, which included a multidisciplinary team and checklist as well as patient education materials. The study showed modest improvements in documentation of oral chemotherapy consent and education after implementation of the new workflow compared with before.14 Although quality improvement does not necessarily have to be evaluated through QOPI, the process of preparing for QOPI certification has been used by some practices as an opportunity to develop, implement, and assess the effect of quality improvement interventions to address perceived quality gaps, including those pertaining to oral chemotherapy prescribing. For example, a group in Colorado documented how their use of strategies implemented through the EMR, including flowsheets and best practice alerts, improved documentation of oral chemotherapy adherence and toxicity.15 In this context, QOPIcan be a valuable tool to document quality improvement within a practice using standardized metrics and outcomes.
Oral chemotherapy is increasingly incorporated into the standard oncology regimens used with both curative and palliative intent.16,17 Yet most oncologists in practice today were trained during a time when cancer treatment was almost exclusively parenteral, necessitating frequent clinic visits and interaction with a variety of clinical team members, including physician, practice nurse, and chemotherapy infusion nurse. The rapid pace with which oral chemotherapies have permeated clinical practice has left a void in the cancer treatment paradigm, and the standard operating procedures that have been in place for parenteral chemotherapy administration have been rapidly adapted and applied for use in the setting of oral chemotherapy. Our study included 3 measures pertaining to patient education prior to beginning oral chemotherapy: education about anticipated toxicities, clinic contact instructions, and missed doses. All 3 components of patient education are included in the 2013 updated ASCO/ONS standards, which included standards for oral chemotherapy administration.5 Notably, the first 2 fit within the intravenous treatment paradigm, but the third measure does not. A priori patient education about missed doses is not needed if the patient needs to be in clinic to receive treatment, as is the case with parenteral administration. Therefore, it appears that we have not adequately adapted our approach to patient education to fit this aspect of the oral chemotherapy paradigm. Furthermore, the use of oral chemotherapy misses an additional opportunity for education and toxicity assessment, given that the chemotherapy infusion nurse is not involved in the patient’s care. Instead, the oncology pharmacist is a member of the care team who can collaborate to enhance clinical care for patients treated with oral chemotherapy by helping improve communication and patient education as well as assisting with adherence monitoring, as suggested by the 2018 Hematology/Oncology Pharmacist Association standards.18 Bolstering this recommendation, 2 recent reviews of interventions to improve adherence to and safety of oral chemotherapy suggest that personal contact (eg, via telephone) between a pharmacist or nurse and a patient, particularly early in the course of treatment with a newly prescribed oral chemotherapy, may both decrease toxicity and improve medication adherence.19,20
A limitation of our analysis is that we were unable to evaluate the relationship between measure performance and longitudinal involvement in the QOPI, which might have allowed us to better understand the role that greater experience reporting QOPI measures may or may not play in quality improvement. Furthermore, the generalizability of our findings may be limited by the structure of many of the practices included. Data on the structure of study practices indicate that a relatively high number were not academically affiliated (Figure 1), which is in contrast to the increasing trend for so-called private practices to join academic medical centers. Additionally, most practices that were eligible for inclusion in our study were small (66% had ≤5 oncologists within the practice). Both of these factors may reflect the perceived value of QOPI certification. Small, nonacademic practices may use QOPI as a roadmap for quality in the absence of a larger institution to provide guidance and support for quality monitoring and improvement efforts. On the other hand, there may be practices that would have enrolled in the certification process but chose not to do so due to other concerns, such as the associated fees, work hours spent on documentation, and staffing needs. These practices may perform well on the measures while choosing not to enroll in QOPI certification. Additional limitations of our study include the lack of measure validation (with some measures excluded from the analyses because they were still being tested), the relatively small sample size, and the inherent heterogeneity of the 45% of practices included in this study that spanned different practice sites.
Conclusions
With clear room for improvement, regardless of QOPI certification status as demonstrated by our findings, future work should include the establishment of quality improvement initiatives to improve oral chemotherapy procedures. The aforementioned ASCO/ONS chemotherapy standards provide a comprehensive list that practices can use to formulate updated operating procedures that better align with the specific needs surrounding oral chemotherapy prescribing and administration.
References
- 1.↑
Weingart SN, Brown E, Bach PB, et al. NCCN task force report: oral chemotherapy. J Natl Compr Canc Netw 2008;6(Suppl 3):S1–14.
- 2.↑
US Food and Drug Administration. Advancing health through innovation: new drug therapy approvals 2021. Accessed March 1, 2022. Available at: https://www.fda.gov/media/155227/download
- 3.↑
US Food and Drug Administration. Novel drug approvals for 2020. Accessed March 1, 2022. Available at: https://www.fda.gov/drugs/new-drugs-fda-cders-new-molecular-entities-and-new-therapeutic-biological-products/novel-drug-approvals-2020
- 4.↑
US Food and Drug Administration. Advancing health through innovation: new drug therapy approvals 2019. Accessed March 1, 2022. Available at: https://www.fda.gov/media/134493/download
- 5.↑
Neuss MN, Polovich M, McNiff K, et al. 2013 updated American Society of Clinical Oncology/Oncology Nursing Society chemotherapy administration safety standards including standards for the safe administration and management of oral chemotherapy. J Oncol Pract 2013;9(Suppl):5s–13s.
- 6.↑
American Society of Clinical Oncology. QOPI-related measures. Accessed October 2, 2019. Available at: https://practice.asco.org/quality-improvement/quality-programs/quality-oncology-practice-initiative/qopi-related-measures
- 7.↑
McNiff KK, Bonelli KR, Jacobson JO. Quality oncology practice initiative certification program: overview, measure scoring methodology, and site assessment standards. J Oncol Pract 2009;5:270–276.
- 8.↑
Greer JA, Lennes IT, Gallagher ER, et al. Documentation of oral versus intravenous chemotherapy plans in patients with metastatic non-small-cell lung cancer. J Oncol Pract 2014;10:e103–106.
- 9.↑
Zerillo JA, Pham TH, Kadlubek P, et al. Administration of oral chemotherapy: results from three rounds of the quality oncology practice initiative. J Oncol Pract 2015;11:e255–262.
- 10.↑
American Society of Clinical Oncology. QOPI certification program standards. Accessed February 22, 2019. Available at: https://practice.asco.org/sites/default/files/drupalfiles/2018-06/QOPI%20Certification%20 Standards%20June%202018.pdf
- 11.↑
Neuss MN, Malin JL, Chan S, et al. Measuring the improving quality of outpatient care in medical oncology practices in the United States. J Clin Oncol 2013;31:1471–1477.
- 12.↑
Siegel RD, Castro KM, Eisenstein J, et al. Quality improvement in the national cancer institute community cancer centers program: the quality oncology practice initiative experience. J Oncol Pract 2015;11:e247–254.
- 13.↑
Simmons C, Power S. Using ASCO QOPI and the QTP as an improvement model in a small safety net cancer center [abstract]. J Clin Oncol 2018;36(Suppl):Abstract 78.
- 14.↑
Brown JS, Hershman DL, O'Neil D, et al. A standardized workflow to improve the consent process among patients initiating an oral anticancer drug [abstract]. J Clin Oncol 2019;37(Suppl):Abstract 259.
- 15.↑
Jeffers K, Walde A. Harnessing the electronic health record to optimize oral anticancer therapy monitoring and follow-up [abstract]. J Clin Oncol 2018;36(Suppl):Abstract 305.
- 16.↑
Conti RM, Fein AJ, Bhatta SS. National trends in spending on and use of oral oncologics, first quarter 2006 through third quarter 2011. Health Aff (Millwood) 2014;33:1721–1727.
- 17.↑
Gustafson E, Kettle JK. Analyzing trends in oral anticancer agents in an academic medical facility. J Hematol Oncol Pharm 2015;5:34–37.
- 18.↑
Mackler E, Segal EM, Muluneh B, et al. 2018 Hematology/Oncology Pharmacist Association best practices for the management of oral oncolytic therapy: pharmacy practice standard. J Oncol Pract 2019;15:e346–355.
- 19.↑
Zerillo JA, Goldenberg BA, Kotecha RR, et al. Interventions to improve oral chemotherapy safety and quality: a systematic review. JAMA Oncol 2018;4:105–117.
- 20.↑
Rosenberg SM, Petrie KJ, Stanton AL, et al. Interventions to enhance adherence to oral antineoplastic agents: a scoping review. J Natl Cancer Inst 2020;112:443–465.
