Background
The FDA defines clinical benefit as an improvement in either the duration or the quality of life (QoL).1 FDA criteria allow cancer drugs to be approved based on surrogate measures that will reasonably predict definitive outcomes, such as overall survival (OS) or QoL. Most intermediate endpoints have not been validated as surrogates for definitive outcomes yet are used as primary endpoints in trials supporting drug approval.2,3 This practice can limit the understanding of the clinical benefit of new drugs.4,5
Recent systematic reviews have indicated that many surrogate endpoints used for drug approvals do not translate into improvements in OS6 or QoL.7,8 In addition, recent findings have revealed that fewer than half of cancer drug indications approved by the FDA2,3 and the European Medicines Agency9 have shown improved OS or QoL in the postmarketing period. Similarly, only a proportion of studies supporting drug approval have shown substantial clinical benefit using the ESMO-Magnitude of Clinical Benefit Scale (ESMO-MCBS) or the ASCO Value Framework (ASCO-VF) at the time of drug approval.10–12 There are limited data on whether the magnitude of clinical benefit changes between the time of regulatory approval and the postmarketing period. In this article, we quantify the proportion of studies meeting thresholds for substantial clinical benefit both at the time of approval and in the postmarketing period and explore associations with improved OS, QoL, and clinical benefit over time.
Methods
Data Sources
We examined data from pivotal trials supporting FDA approval in adult solid tumors between January 1, 2006, and December 31, 2015. These dates were selected to allow at least 3 years of postmarketing follow-up for all drugs. Details of data sources have been described previously.13
Data Extraction, Search Strategies, and Data Analysis
Two authors (A. Bujosa, A. Tibau) extracted data using predesigned electronic forms. For each indication, the review14–17 and approval (accelerated vs regular)18 pathways and the number of trials supporting the application were identified. When >1 pivotal study supported a single indication, a hierarchical selection strategy preferring more definitive endpoints (typically OS) to QoL was used. Then, the ClinicalTrials.gov identification number was collected from the reports of registration trials. To determine changes in the postmarketing period, PubMed was searched first. Second, search results were cross-referenced with ClinicalTrials.gov, and if no follow-up trials were identified, then Google Scholar was searched.
In the postmarketing period, for regular approvals, updates of pivotal trials were analyzed as follow-up studies. For accelerated approvals, we assessed the status of conversion to full approval.18 If conversion to regular approval was granted, then only the confirmatory trial was analyzed as a follow-up study.
For each included trial, we extracted information on trial characteristics, primary efficacy endpoints (OS vs intermediate endpoints), toxicity, and QoL outcomes. We also collected data on whether a companion diagnostic test was available as defined by the FDA.19 For trials performed in the palliative setting, we collected data on the line of therapy. For postmarketing trials, we collected the median time of follow-up, the most rigorous endpoint that was statistically significant, and updated information about efficacy, QoL, and toxicity when available. For accelerated approvals converted to regular approvals, we considered the period between accelerated approval and regular approval as the median time between accelerated approval and conversion to full approval.
A drug was considered to have shown an OS benefit if a statistically significant benefit was observed with experimental therapy.3 A drug was considered to have shown a QoL benefit if a statistically significant difference was reported between the drug and the experimental arm among randomized controlled trials and between baseline and after treatment of single-arm trials. QoL data needed to be based on a global score, a subscale, or a specific item from a validated patient-reported outcome (PRO) instrument.3
Data Scoring
We used the ASCO-VF Net Health Benefit Score version 2.020 and the ESMO-MCBS version 1.121 to quantify the magnitude of clinical benefit. Despite differences in how efficacy, toxicity, and QoL are weighed (the ASCO-VF is linear and the ESMO-MCBS is categorical), both frameworks aim to measure the magnitude of the benefit of novel therapies. ESMO-MCBS and ASCO-VF grades were applied independently by 2 authors (either A. Bujosa and J.C. Tapia, or A. Bujosa and C. Moltó, respectively). The discrepancies were resolved by a fourth author (A. Tibau). In the postmarketing period, for drugs with updated efficacy and QoL data, grades and/or scores were recalculated because updated efficacy or QoL data can change scores. For trials in which updated data reported only 1 domain of efficacy or QoL, grades and/or scores were adjusted, retaining initial data that were not updated. Substantial clinical benefit was defined as follows: for the ASCO-VF, a threshold score of ≥45,22 and for the ESMO-MCBS, a grade of A or B for trials of curative intent and 4 or 5 for those of palliative intent.21
Statistical Analysis
Data were reported descriptively as proportions, medians, and ranges. In the noncurative setting, we explored whether characteristics of trials and applications were associated with OS, QoL, and clinically significant benefit thresholds. Associations were evaluated using univariable logistic regression at 3 timepoints: (1) at the time of approval, (2) after maximum postmarketing time, and (3) at any point in the life cycle of the drug. Regression results were reported as odds ratios and their respective 95% confidence intervals. Multivariable models included variables that were statistically significant in univariable analysis, with variable selection ensuring adequate fitting of the model. To reduce the chance of overfitting, multivariable analysis was performed exclusively using data at any point in the life cycle of the drug. Such analyses were not planned in the curative setting because of the small number of studies supporting such indications. Trends over time were assessed using the chi-square test for trends. All analyses were conducted using SPSS Statistics, version 21 (IBM Corp). Statistical tests were 2-sided, and statistical significance was defined as a 2-tailed P<.05.
Results
Between January 1, 2006, and December 31, 2015, 58 drugs were approved for 96 solid tumor indications based on 96 trials. Characteristics of included applications and trials supporting drug approval at the time of market authorization are shown in Table 1. Of these indications, 74 (77%) were granted regular approval and 22 (23%) were granted accelerated approval. Fourteen approvals (15%) were supported by single-arm studies. For 88 indications (92%), treatment intent was noncurative.
Characteristics of Applications and Trials Supporting Drug Approval
In the postmarketing period, most accelerated approval indications were converted to regular approval. Conversion from accelerated to regular approval was supported by 20 trials. One indication was pending regular approval (nivolumab and ipilimumab for advanced melanoma), and 1 indication was withdrawn (bevacizumab for advanced breast cancer). Consequently, the analysis comprised 96 trials: 74 trials supporting regular approval, 20 postmarketing trials supporting conversion to regular approval, 1 trial supporting the FDA withdrawal of the breast cancer indication for bevacizumab, and 1 trial supporting accelerated approval and pending conversion to regular approval.
OS and QoL Data
Table 2 summarizes the results for OS, QoL, and intermediate endpoints for all included studies. With a median postmarketing period of 3.3 years (minimum 9.2 months, maximum 8.8 years), 70 trials (73%) reported updated data on efficacy (n=59; 61%) and/or QoL (n=48; 50%). For 26 trials (27%), no updated data on efficacy or QoL were available.
OS, QoL, and Clinical Benefit for Trials Included in Original FDA Report and in Postmarketing Period
At the time of marketing, approval was based on OS in 39 trials (41%). Among the 59 trials providing updated efficacy data in the postmarketing setting, 28 (47%) showed improvement in OS, including 11 (19%) that showed OS improvement for the first time in the postmarketing period and 17 (29%) in which OS benefit reported previously was confirmed. In total, significantly improved OS was observed in 50 of 96 indications (52%), no improvement was observed in 31 indications (32%), and for 15 indications (16%), the effect on OS remains unknown.
Only 45 of 96 (47%) initial indications reported QoL data. Of these, 16 indications (36%) showed a statistically significant improvement in at least 1 PRO at the time of market approval. In the postmarketing period, among the 48 trials (50%) reporting updated QoL data, an improvement in QoL was observed for 22 (46%): 11 drugs (23%) showed improvement in QoL for the first time, and in 11 trials (23%) a QoL benefit reported previously was confirmed. Overall, a significant improvement in QoL was observed for 27 of 96 indications (28%), and there was no improvement for 36 indications (36%). For 33 indications (34%), QoL was not assessed or reported formally. Notably, 16 indications (17%) showed an improvement in both OS and QoL.
Measures of Substantial Clinical Benefit
Table 2 shows the ASCO-VF and the ESMO-MCBS scores at the time of approval and in the postmarketing period. ASCO-VF scores were applied to 80 of 96 initial trials (83%). Of the trials for which ASCO-VF could not be applied, 14 were single-arm or the primary endpoint was not suitable for assessment. Only 26 trials (33%) evaluable by the ASCO-VF met the threshold for substantial clinical benefit at the time of marketing approval. In the postmarketing period, 65 trials (68%) provided updated information and were evaluable using the ASCO-VF. Of these, 35 trials (54%) met the ASCO-VF scores for substantial clinical benefit including 17 (26%) that did not initially meet the threshold for substantial benefit but were upgraded to show substantial clinical benefit based on postmarketing data. Four trials (6%) that showed substantial benefit at the time of approval were downgraded because of a lack of substantial benefit based on postmarketing data.
The ESMO-MCBS scores could be applied to 94 of 96 initial trials (98%). Only 25 trials (27%) met the ESMO-MCBS threshold for substantial clinical benefit at the time of marketing approval. In the postmarketing period, 69 trials provided updated information and were scorable for the ESMO-MCBS. Of these, 37 (54%) met the threshold for substantial clinical benefit. Updated results led to a change in the ESMO-MCBS substantial clinical benefit threshold in 18 trials (26%). Of these, 17 trials (25%) that did not initially meet the threshold for substantial benefit were upgraded to show substantial clinical benefit based on postmarketing data. One trial (1%) that showed substantial benefit at the time of approval was downgraded because of a lack of substantial benefit based on postmarketing data.
Overall, based on both initial and updated data, among the 96 indications approved by the FDA, 39 trials (41%) met the criteria for substantial benefit using the ASCO-VF and 42 (44%) met the threshold using the ESMO-MCBS.
Over time, there has been an increase in the number of trials meeting the ASCO-VF threshold at any time (11% for drugs approved in 2006 vs 56% for drugs approved in 2015; Ptrend=.038). There were no significant changes over time in OS or QoL or in the ESMO-MCBS scores. An OS benefit was observed among 60% of drugs approved in 2006 and 55% approved in 2015 (Ptrend=.78). A QoL benefit was observed in 17% of drugs approved in 2006 and 54% approved in 2015 (Ptrend=.17), and for the ESMO-MCBS, 40% of drugs met the substantial clinical benefit threshold in 2006 and 47% in 2015 (Ptrend=.65).
Associations With OS, QoL, and Clinical Benefit
Associations with OS, QoL, and magnitude of clinical benefit as measured using the ASCO-VF and ESMO-MCBS at the time of market authorization are shown in Table 3 and those in the postmarketing period are shown in Table 4.
Predictors of Benefit in Noncurative Setting at Time of Marketing Approval
Predictors of Benefit in the Noncurative Setting in Postmarketing Period
At the time of market approval, there were statistically significant associations between improved OS and regular approval, nonorphan drug designation, sample size, more common tumors compared with others, and absence of crossover interference. Factors associated with OS in the postmarketing period differed, with immunotherapy, companion diagnostic drugs, and approvals based on subgroup analysis showing a survival advantage.
For QoL, there were no significant associations at initial approval. Improved QoL in the postmarketing period was associated with sample size, immunotherapy, and open-label studies.
At the time of market authorization, initial indications and those reporting QoL benefit were more likely to show substantial clinical benefit as determined by the ASCO-VF. In the postmarketing period, substantial clinical benefit as measured by the ASCO-VF was associated with accelerated approvals, drugs granted breakthrough therapy designation, indications for later lines of therapy, indications supported by phase I–II or single-arm trials, and indications with a lower sample size. Drugs initially approved with evidence of improved QoL were associated with substantial clinical benefit both at the time of approval and in the postmarketing period.
For the ESMO-MCBS, at the time of market authorization, drugs with a companion diagnostic test and those with OS benefit or QoL improvement were associated with substantial clinical benefit. Postapproval, only trials of drugs with companion diagnostics were associated with substantial clinical benefit.
Associations with OS, QoL, or substantial clinical benefit at any time are shown in supplemental eTable 1 (available with this article at JNCCN.org). In multivariable analyses, drugs with companion diagnostic tests and immunotherapy were associated with improved OS, QoL, and substantial clinical benefit.
Discussion
In the current analysis, we found that at the time of market approval, among 96 cancer drug indications approved by the FDA between 2006 and 2015, two-thirds of trials supporting FDA-approved anticancer drugs failed to meet the thresholds of substantial clinical benefit established by the ASCO-VF and ESMO-MCBS. With 3.3 years of postmarketing experience, approximately half of the trials showed substantial clinical benefit, thereby increasing the number of trials meeting ASCO-VF and ESMO-MCBS thresholds. Our analysis also showed an increase in the number of trials meeting the ASCO-VF threshold for clinical benefit over time.
There are limited data regarding the impact of postmarketing experience on the magnitude of clinical benefit for FDA-approved anticancer drugs as measured by validated frameworks. Previous data have focused on the availability of data on OS and QoL at the time of drug approval by the FDA from 2011 to 20173 and by the European Medicines Agency from 2009 to 2013.9 Taken together, these studies found that less than half of the anticancer drugs showed an OS benefit or a QoL improvement. These results are consistent with our present work, which covers a longer period. Between 2006 and 2015, the FDA approved most cancer drugs without evidence of OS (60%) or improved QoL (83%). Even after a median of 3.3 years, only 52% of indications were supported by an improvement in OS and 28% showed a statistically significant improvement in QoL.
When we evaluated associations with clinical benefit, we found that factors associated with OS and QoL benefit or substantial clinical benefit differed at the time of approval and in the postmarketing period. Unsurprisingly, trials with an OS benefit at marketing authorization were more likely to be granted regular approval, more commonly evaluated drugs in common cancers, and had a larger magnitude of effect. In contrast, clinical benefit in the postmarketing period was associated with immunotherapy, a treatment for which responses and durability of response can appear later in follow-up.23 In addition, some variables such as drugs with companion diagnostics seemed to show a benefit in certain scenarios. Of concern was our finding that open-label studies were more likely to show improved QoL. This raises concern about ascertainment bias.24
A possible explanation for the variability in associations of study and approval characteristics with OS, QoL, and magnitude of clinical benefit is that the ASCO-VF and ESMO-MCBS evaluate benefit as composite outcomes weighting efficacy, toxicity, and QoL rather than assessing these outcomes in isolation. Another explanation is that in our study a drug was considered to have shown an OS benefit if a statistically significant benefit was observed with experimental therapy. Statistical significance can be observed with small effect sizes, which may not meet the thresholds for substantial clinical benefit, a concept better addressed with the ASCO-VF and ESMO-MCBS, in which efficacy outcomes are measured in terms of relative and absolute benefits.
The increased use of surrogate endpoints as the basis for approval is associated with uncertainty about the definitive benefit of drugs. This ambiguity makes postmarketing data very valuable.25 Although the accelerated approval pathway requires the completion of postmarketing clinical trials to confirm that a drug provides clinical benefit as predicted by the surrogate endpoint, regular approvals do not require further assessment to confirm effectiveness and safety after marketing authorization. Unfortunately, in the postapproval period, only two-thirds of pivotal trials provided updated data on efficacy, which may reflect a lack of positive studies (publication bias) or studies that were underpowered to show statistically significant improvement in OS.
Our data support possible improvement in the approval process for anticancer drugs. First, when surrogate endpoints are used, and especially for accelerated drug approvals, postmarketing studies with both OS and QoL outcomes should be mandated.26 Second, greater efforts need to be made to develop and validate meaningful surrogate endpoints for OS and QoL. Third, PROs and other patient-centered measures should be prioritized at early stages of drug development. In pivotal trials, missing data27 should be minimized and journal editors should encourage the publication of such data.28 Finally, trials should be designed with sufficient statistical power for an analysis of changes in OS and/or QoL.
This study has limitations. First, the duration of postmarketing approval was relatively short. A longer period of postmarketing time may have resulted in a higher proportion of indications showing improved OS and QoL, especially with immune checkpoint inhibitors. Second, PRO data are often missing from FDA cancer-drug labels and are frequently not included in primary publications of pivotal trials. Third, PROs are often exploratory, which makes their interpretation difficult. Fourth, results obtained from analysis with frameworks that rely largely on randomized trials should be treated with caution because the score applied to experimental therapy is highly dependent on the quality of the control arm. Finally, we excluded drugs for hematologic malignancies, thereby limiting the generalizability to solid tumor drug approval.
Conclusions
In patients with solid tumors, an increasing number of approved drugs showed improved OS and QoL and met the ASCO-VF or ESMO-MCBS threshold for substantial benefit over the course of postmarketing time compared with the time of approval. However, fewer than half of the trials supporting FDA approval showed an OS benefit or a QoL improvement, and just more than half of the trials showed substantial clinical benefit. With an increasing number of drug approvals being based on single-arm trials and surrogate endpoints,29 robust postmarketing trials are becoming increasingly important.30 Regulators and professional societies should prioritize the collection of outcome data throughout the postmarketing life cycle of drugs and be prepared to update indications or treatment recommendations on the basis of all available data.
References
- 2.↑
Kim C, Prasad V. Cancer drugs approved on the basis of a surrogate end point and subsequent overall survival: an analysis of 5 tears of US Food and Drug Administration approvals. JAMA Intern Med 2015;175:1992–1994.
- 3.↑
Zettler M, Basch E, Nabhan C. Surrogate end points and patient-reported outcomes for novel oncology drugs approved between 2011 and 2017. JAMA Oncol 2019;5:1358–1359.
- 4.↑
Tibau A, Molto C, Borrell M, et al. Magnitude of clinical benefit of cancer drugs approved by the US Food and Drug Administration based on single-arm trials. JAMA Oncol 2018;4:1610–1611.
- 5.↑
Shepshelovich D, Tibau A, Goldvaser H, et al. Postmarketing modifications of drug labels for cancer drugs approved by the US Food and Drug Administration between 2006 and 2016 with and without supporting randomized controlled trials. J Clin Oncol 2018;36:1798–1804.
- 6.↑
Prasad V, Kim C, Burotto M, et al. The strength of association between surrogate end points and survival in oncology: a systematic review of trial-level meta-analyses. JAMA Intern Med 2015;175:1389–1398.
- 7.↑
Kovic B, Jin X, Kennedy SA, et al. Evaluating progression-free survival as a surrogate outcome for health-related quality of life in oncology: a systematic review and quantitative analysis. JAMA Intern Med 2018;178:1586–1596.
- 8.↑
Hwang TJ, Gyawali B. Association between progression-free survival and patients’ quality of life in cancer clinical trials. Int J Cancer 2019;144:1746–1751.
- 9.↑
Davis C, Naci H, Gurpinar E, et al. Availability of evidence of benefits on overall survival and quality of life of cancer drugs approved by European Medicines Agency: retrospective cohort study of drug approvals 2009-13. BMJ 2017;359:j4530.
- 10.↑
Del Paggio JC, Sullivan R, Schrag D, et al. Delivery of meaningful cancer care: a retrospective cohort study assessing cost and benefit with the ASCO and ESMO frameworks. Lancet Oncol 2017;18:887–894.
- 11.↑
Vokinger KN, Hwang TJ, Grischott T, et al. Prices and clinical benefit of cancer drugs in the USA and Europe: a cost-benefit analysis. Lancet Oncol 2020;21:664–670.
- 12.↑
Booth CM, Del Paggio JC. Approvals in 2016: questioning the clinical benefit of anticancer therapies. Nat Rev Clin Oncol 2017;14:135–136.
- 13.↑
Tibau A, Molto C, Ocana A, et al. Magnitude of clinical benefit of cancer drugs approved by the US Food and Drug Administration. J Natl Cancer Inst 2018;110:486–492.
- 14.↑
U.S. Food & Drug Administration. New molecular entity (NME) drug and new biologic approvals. Accessed July 20, 2021. Available at: https://www.fda.gov/drugs/nda-and-bla-approvals/new-molecular-entity-nme-drug-and-new-biologic-approvals
- 15.↑
U.S. Food & Drug Administration. Efficacy supplement approvals. Accessed July 20, 2021. Available at: https://www.fda.gov/drugs/nda-and-bla-approvals/efficacy-supplement-approvals
- 16.↑
U.S. Food & Drug Administration. CDER breakthrough therapy designation approvals. Accessed July 20, 2021. Available at: https://www.fda.gov/media/95302/download
- 17.↑
U.S. Food & Drug Administration. Search orphan drug designations and approvals. Accessed July 20, 2021. Available at: http://www.accessdata.fda.gov/scripts/opdlisting/oopd/index.cfm
- 18.↑
U.S. Food & Drug Administration. Accelerated approvals. Accessed July 20, 2021. Available at: https://www.fda.gov/drugs/drug-and-biologic-approval-and-ind-activity-reports/nda-and-bla-approvals
- 19.↑
U.S. Food & Drug Administration. List of cleared or approved companion diagnostic devices (in vitro and imaging tools). Accessed July 21, 2021. Available at: https://www.fda.gov/medical-devices/in-vitro-diagnostics/list-cleared-or-approved-companion-diagnostic-devices-in-vitro-and-imaging-tools
- 20.↑
Schnipper LE, Davidson NE, Wollins DS, et al. Updating the American Society of Clinical Oncology Value Framework: revisions and reflections in response to comments received. J Clin Oncol 2016;34:2925–2934.
- 21.↑
Cherny NI, Dafni U, Bogaerts J, et al. ESMO-Magnitude of Clinical Benefit Scale version 1.1. Ann Oncol 2017;28:2340–2366.
- 22.↑
Cherny NI, de Vries EGE, Dafni U, et al. Comparative assessment of clinical benefit using the ESMO-Magnitude of Clinical Benefit Scale version 1.1 and the ASCO Value Framework net health benefit score. J Clin Oncol 2019;37:336–349.
- 23.↑
Ben-Aharon O, Magnezi R, Leshno M, et al. Association of immunotherapy with durable survival as defined by value frameworks for cancer care. JAMA Oncol 2018;4:326–332.
- 24.↑
Roydhouse JK, King-Kallimanis BL, Howie LJ, et al. Blinding and patient-reported outcome completion rates in US Food and Drug Administration cancer trial submissions, 2007-2017. J Natl Cancer Inst 2019;111:459–464.
- 25.↑
Moscow JA, Fojo T, Schilsky RL. The evidence framework for precision cancer medicine. Nat Rev Clin Oncol 2018;15:183–192.
- 26.↑
Zettler M, Nabhan C. Fulfillment of postmarketing requirements to the FDA for therapies granted oncology indications between 2011 and 2016. JAMA Oncol 2018;4:993–994.
- 28.↑
Allegra CJ, Goodwin PJ, Ganz PA. Can we find the positive in negative clinical trials? J Natl Cancer Inst 2019;111:637–638.
- 29.↑
Nass SJ, Rothenberg ML, Pentz R, et al. Accelerating anticancer drug development—opportunities and trade-offs. Nat Rev Clin Oncol 2018;15:777–786.
- 30.↑
Gyawali B, Hey SP, Kesselheim AS. Assessment of the clinical benefit of cancer drugs receiving accelerated approval. JAMA Intern Med 2019;179:906–913.