Background
Regulatory approval of cancer drugs is often based on evidence from surrogate endpoints. Recent work suggests that most drugs have been approved by the FDA or the European Medicines Agency based on improvements in surrogate endpoints alone without evidence of extending or improving life.1–3 These surrogate outcomes, such as response rates or progression-free survival (PFS), are also reported in primary publications, at which time overall survival (OS) data may be immature.1 Other clinically relevant data, such as quality of life (QoL) and long-term survival outcomes, are often reported in subsequent publications after initial drug approval.2 However, the submission of such subsequent evidence is generally not required for the reassessment of ongoing approval, despite its importance in clinical decision-making. This highly relevant information is thus often unavailable at the time of approval of novel oncology drugs and may not be considered by regulatory agencies for ongoing approval.
Closing the knowledge gap and facilitating shared decision-making between patients and physicians about the clinical benefits and costs of cancer drugs were the main goals of the ASCO Value in Cancer Care Task Force. ASCO developed a value framework (VF) in 2015 in an attempt to quantify the net health benefit (NHB) of oncology drugs via a single summary score.4,5 This framework relies on key elements provided by the pivotal randomized clinical trial, including efficacy, toxicity, prolonged survival, symptom palliation, and improvement in QoL with drugs that improve OS and increase QoL obtaining higher scores. In 2018, ASCO defined threshold scores of ≤40, ≥45, and between 40 and 45 as low, substantial, and intermediate NHB, respectively.6 Research has shown that recent FDA-approved oncology drugs show a median NHB of 37, defined by ASCO as a low NHB.7 Further studies have suggested that despite the increasing cost of oncology drugs, there is not a proportional increase in the clinical benefit.8,9
As new evidence arises from trials or as data mature, considerable components of their NHB score may change. Thus, the initial ASCO-VF NHB score and the corresponding classification of benefit calculated at approval may not accurately reflect the overall benefit of a drug given all publicly available evidence. To our knowledge, it is presently unclear how the benefit of FDA-approved drugs may evolve with the publication of subsequent evidence. Therefore, we aimed to investigate the ASCO-VF NHB of oncology drugs at the time of FDA regulatory approval and reassess the benefit over time given newly available evidence.
Methods
Drug Identification and Trial Extraction
Oncology drug indications approved by the FDA10 in January 2006 through December 2016 with clinical trials scorable using the ASCO-VF version 2 (v2) were identified from the FDA database (hematology/oncology approvals and safety notification page and archives).10 Key identifying information (eg, NCT number, study name, number of randomized patients, and highlighted results) was extracted from trials cited within the FDA submission as evidence for clinical efficacy and were used to locate the publication of primary clinical trial results and subsequent publications on the Web of Science.11 Primary publications, supplementary appendices, and subsequent publications were identified with a follow-up time of 3 years from FDA approval. Clinical trials for oncology drugs treating advanced disease with at least one follow-up publication containing adequate data to allow for scoring using at least one element of the ASCO-VF were included.
Data Extraction and Scoring
The ASCO-VF assigns an NHB score for each regimen based on 3 categories: clinical benefit, toxicity, and a bonus category. Clinical benefit is determined by evaluating survival endpoints (or surrogate endpoints for survival) according to a set hierarchy: hazard ratio (HR) for death, median OS, HR for disease progression, median PFS, and response rate. If the OS determination is obscured by trial design (ie, crossover), then PFS is used to determine the clinical benefit.4 Subsequently, toxicity is evaluated by assessing whether an improvement in toxicity over the comparator is represented by calculating the percentage difference in total toxicity points. Finally, bonus points are awarded based on the tail of the curve, palliation, QoL, and treatment-free interval criteria. Thus, to determine the NHB, the following data were extracted, when available, from each trial: OS/PFS HR or median, response rate, toxicity, OS or PFS Kaplan-Meier curves, reporting of cancer-related symptoms, QoL measures, and treatment-free interval results. Extracted data were then used to calculate each component of the NHB and the overall NHB at the time of regulatory approval.
ASCO-VF NHB scores were subsequently recalculated using the most updated publicly available evidence until 3 years postapproval. In concordance with framework guidelines, if only PFS data were initially reported at the time of FDA approval and OS data later became publicly available, if not confounded by crossover, OS would then be used to capture clinical benefit at reassessment.
Clinical trials were scored using the ASCO-VF v2 by 2 independent reviewers. Whenever possible, scores endorsed by the ASCO-VF based on the joint publication by ASCO and the ESMO in 20186 were used to verify scores. Discrepancies were resolved and scores were finalized after discussion between reviewers. In concordance with threshold scores previously outlined by the ASCO-VF, scores were classified as low benefit if the NHB was ≤40, as substantial benefit for NHB ≥45, and as intermediate benefit for NHB was between 40 and 45.6 Classifications of benefit were assessed at the time of primary publication and at 3 years post-FDA approval using all publicly available data in eligible subsequent publications.
Results
Characteristics of Included Studies
Of 113 indications that met the eligibility criteria, there were no updated results within 3 years of approval, necessitating a change in value scores for 58 indications (51%). Thus, our analysis cohort included 55 FDA-approved indications with relevant subsequent publications in the 3 years after FDA approval (Figure 1). Characteristics of all 55 trials, within which 35,422 patients received treatment with 38 unique oncology drugs, are presented in Table 1. Five indications (9.1%) were approved under the accelerated approval pathway, 23 (41.8%) had OS as the primary endpoint, and 32 (58.2%) had PFS as the primary endpoint.

Flow diagram of included trials.
Abbreviation: ASCO-VF, ASCO Value Framework.
Citation: Journal of the National Comprehensive Cancer Network J Natl Compr Canc Netw 2021; 10.6004/jnccn.2020.7677
Characteristics of Included Trials (N=55)

Ninety relevant subsequent publications published within 3 years of FDA approval were identified. There was a median of 1 (interquartile range [IQR], 1–2) subsequent publication with an ASCO-scorable element, published a median of 1.5 years (IQR, 1.1–2.2) after FDA approval. Most subsequent publications provided updates on OS, PFS, and QoL (33.3%, 21.1%, and 47.8%, respectively), and many provided updated toxicity results (31.1%). Characteristics of subsequent publications are presented in Table 2.
Characteristics of Subsequent Publications (n=90)

Initial ASCO-VF NHB
At the time of approval, the mean NHB was 38.1 (SD, 17.6) and median NHB was 40.6 (IQR, 24.2–52.4). The clinical benefit components of scores were all based on HRs reported in primary publications (33 [60.0%] for OS and 21 [38.2%] for PFS), except for 1 indication based on response rate. Twenty-eight indications (50.9%) qualified for tail-of-the-curve bonuses (8 [14.8%] for OS and 20 [36.4%] for PFS). Six indications (10.9%) were awarded palliation bonuses, 10 (18.2%) were awarded QoL bonuses, and 1 (1.8%) was awarded a treatment-free interval bonus. Among 55 indications, 22 (40.0%) showed substantial benefit, 6 (10.9%) showed intermediate benefit, and 27 (49.1%) showed low benefit at the time of FDA approval.
Reassessment of ASCO-VF NHB
With a follow-up time of 3 years post-FDA approval, the mean updated NHB was 38.8 (SD, 20.8) and median updated NHB was 38.9 (IQR, 21.6–55.4). Twenty-one indications (38.2%) had an increase in NHB, 20 (36.4%) had a decrease in NHB, and 14 (25.5%) had no change in NHB (supplemental eFigure 1, available with this article at JNCCN.org). Twenty indications (36.4%) had an updated clinical benefit score based on HR. Among them, 13 (23.6%) were based on updated OS HR and 7 (12.7%) were based on updated PFS HR. A change in the scoring endpoint occurred in 5 (9.1%) indications, all but 1 of which were from PFS HR to OS HR.
Overall, of 24 indications (43.6%) that had a change in clinical benefit score, 21 (38.2%) showed an increase and 3 (5.5%) showed a decrease. Other changes from initial scoring were the result of toxicity updates (n=31; 56.4%), new tail-of-the-curve bonus points (n=5; 9.1%), new palliation bonuses (n=8; 14.5%), and new QoL bonuses (n=10; 18.2%) (Figure 2). Although the incidence of these bonuses increased at 3 years postapproval, most indications that were awarded new bonuses were already classified as substantial, or not sufficient to change classification.

Change in components of NHB for (A) clinical benefit and (B) toxicity.
Abbreviations: NHB, net health benefit; OS, overall survival; PFS, progression-free survival; QoL, quality of life.
Citation: Journal of the National Comprehensive Cancer Network J Natl Compr Canc Netw 2021; 10.6004/jnccn.2020.7677
Reassessment of the NHB in subsequent publications showed a change from the initial classification of benefit in 15 trials (27.3%): 7 (12.7%) changed to a higher category compared with their initial classification, whereas 8 (14.5%) changed to a lower category. Of these, 6 indications (10.9%) changed to a substantial benefit from a lower category and 6 (10.9%) changed to a lower category of benefit from being a substantial benefit (Figure 3). Trials initially classified as substantial generally stayed in the same category (16/22; 72.7%). Similarly, 23 of 27 trials (85.2%) initially classified as low remained in the same category after reassessment.

Changes in classification of net health benefit at reassessment 3 years after FDA approval: (A) initial substantial benefit, (B) initial low benefit, (C) initial intermediate benefit, and (D) all trials.
Citation: Journal of the National Comprehensive Cancer Network J Natl Compr Canc Netw 2021; 10.6004/jnccn.2020.7677
Overall, at reassessment with all subsequent publications within 3 years post-FDA approval, 22 trials (40.0%) showed substantial NHB, 5 (9.1%) showed intermediate NHB, and 28 (50.9%) showed low NHB.
Discussion
We found that the value of cancer drugs approved by the FDA, as measured using the ASCO-VF, is a dynamic entity that changes with the arrival of newer or mature data. However, for most indications, these updates in value scores based on updated data were not sufficient to change the benefit of a drug from one category to another. These findings have several policy implications.
Prior research has revealed that many oncology drugs approved by the FDA do not show meaningful clinical benefit according to the ASCO-VF, consistent with our finding that almost half of eligible cancer drugs at the time of FDA approval had low benefit.7 Our findings also show that in general, the value of most cancer drugs remains stable over time postapproval, despite the accumulation of more data, as only 27% of indications had a change in their classification of benefit, with an almost equal number of indications moving into better and worse benefit categories.
First, these results suggest that the clinical benefit of a cancer drug may not necessarily improve with time and accumulation of more data, and may equally degrade. This implies that the clinical benefit and thereby the value assessment, cost-effectiveness assessments (in jurisdictions where applicable), and even approval status (especially for accelerated approvals conditional upon clinical benefit) must be reevaluated at certain time intervals after regulatory approval. Indeed, studies have shown that even for accelerated approvals in which a confirmatory trial with clinical benefit is mandated, only one-fifth of cancer drugs show a benefit in OS at a later timepoint.12 Thus, a dynamic and live update of the clinical benefit of cancer drugs is important from a policy perspective.
Second, because of their high prices, cancer drugs constitute one of the most important causes of financial toxicity for patients and society. Studies have consistently shown that the clinical benefit of cancer drugs does not increase in parallel with escalating drug costs.7–9 Uniquely, cancer drug prices not only are substantial at launch but also increase with time at a rate that is substantially higher than the rate of inflation.13 Part of the justification for this steep rate of increase in cancer drug prices is the presumed belief that the drug evidence for clinical benefit is more mature and robust with time. Indeed, the overall clinical benefit of such drugs may not be entirely evident at the time of initial approval. However, our study shows that the benefit usually stays the same despite updated information from trials, and when it does change, it may equally change toward more benefit or more harm. This information suggests that cancer drug prices may need to be continually negotiated depending on the availability of new evidence.
An important metric of clinical benefit for patients, but one that often gets relegated to a secondary endpoint not worthy of being published with the primary publication, is QoL. In our study, we found that QoL information was available a median of 1.2 years after FDA approval. QoL follow-up studies represented 47.8% of our collected relevant subsequent publications. Whenever QoL information was published subsequently, QoL was not usually improved (55.8% did not result in a bonus), although there were 10 additional indications awarded a QoL bonus (an 18.2% increase from initial approval). This finding suggests that there is a significant bias against publishing detrimental QoL results from cancer drugs. VFs may need to penalize lack of QoL information by adjusting their scores downwards in order to provide incentives to improve this trend, including the delay in publication of QoL results, because QoL is often a key criterion in shared decision-making.14
Of the 5 indications granted accelerated approval in our study, 4 (80.0%) were classified as low benefit by the ASCO-VF at the time of approval and remained low benefit after 3 years. This result is consistent with a previous study showing only one-fifth of cancer drugs that received accelerated approval were subsequently associated with improved survival.12 However, the median time from accelerated approval to confirmation of clinical benefit in a confirmatory trial is usually several years and thus that benefit is not captured in our study.
One of the key limitations of our study is that the subsequent data may have been updated at a timepoint beyond 3 years postapproval. We chose 3 years as the landmark to allow sufficient time for drugs approved as late as December 2016 to be assessed. In addition, because the ASCO-VF applies to only head-to-head comparisons of regimens, there is a low number of indications approved in the accelerated program in our study, because single-arm trial designs provide data for most accelerated approval indications. In addition, the inherent interrater variability is a potential limitation within ASCO-VF scoring, as with any measurement tool. Nevertheless, after factoring previous literature into account,6,15,16 we found that only a small number of cases differed, and our final findings remained the same after sensitivity analysis (results not shown). Finally, because only a small number of drugs approved under the accelerated approval pathway were included, the present study could not draw strong conclusions analyzing this subgroup. Future studies could focus on the ASCO-VF NHB of confirmatory trials since the creation of the accelerated approval pathway.
Conclusions
On average, among FDA-approved drugs, no overall improvement in the ASCO-VF NHB occurred with longer follow-up and evolution of evidence. As further evidence was published, more than 25% of indications changed in classification, which could be either an increase or a decrease in NHB. QoL information was updated postapproval in >40% of indications. Thus, reassessing the ASCO-VF NHB when more evidence becomes available may be beneficial to inform clinical shared decision-making and evidence reappraisal for regulatory and reimbursement decisions.
References
- 1.↑
Kim C, Prasad V. Cancer drugs approved on the basis of a surrogate end point and subsequent overall survival: an analysis of 5 years of US Food and Drug Administration approvals. JAMA Intern Med 2015;175:1992–1994.
- 2.↑
Davis C, Naci H, Gurpinar E, . Availability of evidence of benefits on overall survival and quality of life of cancer drugs approved by European Medicines Agency: retrospective cohort study of drug approvals 2009-13. BMJ 2017;359:j4530.
- 3.↑
Zettler M, Basch E, Nabhan C. Surrogate end points and patient-reported outcomes for novel oncology drugs approved between 2011 and 2017. JAMA Oncol 2019;5:1358–1359.
- 4.↑
Schnipper LE, Davidson NE, Wollins DS, . Updating the American Society of Clinical Oncology Value Framework: revisions and reflections in response to comments received. J Clin Oncol 2016;34:2925–2934.
- 5.↑
Schnipper LE, Davidson NE, Wollins DS, . American Society of Clinical Oncology statement: a conceptual framework to assess the value of cancer treatment options. J Clin Oncol 2015;33:2563–2577.
- 6.↑
Cherny NI, de Vries EGE, Dafni U, . Comparative assessment of clinical benefit using the ESMO-Magnitude of Clinical Benefit Scale Version 1.1 and the ASCO Value Framework Net Health Benefit Score. J Clin Oncol 2019;37:336–349.
- 7.↑
Vivot A, Jacot J, Zeitoun JD, . Clinical benefit, price and approval characteristics of FDA-approved new drugs for treating advanced solid cancer, 2000-2015. Ann Oncol 2017;28:1111–1116.
- 8.↑
Huey R, Anand S, Rogers JE, . Value appraisal of FDA approved cancer drugs over the past decade. J Clin Oncol 2019;37(Suppl):e18585.
- 9.↑
Saluja R, Arciero VS, Cheng S, . Examining trends in cost and clinical benefit of novel anticancer drugs over time. J Oncol Pract 2018;14:e280–294.
- 10.↑
U.S. Food & Drug Administration. Hematology/oncology (cancer) approvals & safety notifications. Accessed November 6, 2020. Available at: https://www.fda.gov/drugs/resources-information-approved-drugs/hematologyoncology-cancer-approvals-safety-notifications
- 11.↑
Clarivate Analytics. Web of Science Core Collection. Accessed January 21, 2020. Available at: http://www.webofknowledge.com/WOS
- 12.↑
Gyawali B, Hey SP, Kesselheim AS. Assessment of the clinical benefit of cancer drugs receiving accelerated approval. JAMA Intern Med 2019;179:906–913.
- 13.↑
Gordon N, Stemmer SM, Greenberg D, . Trajectories of injectable cancer drug costs after launch in the United States. J Clin Oncol 2018;36:319–325.
- 14.↑
Meropol NJ, Egleston BL, Buzaglo JS, . Cancer patient preferences for quality and length of life. Cancer 2008;113:3459–3466.
- 15.↑
Everest L, Shah M, Chan KKW. Comparison of long-term survival benefits in trials of immune checkpoint inhibitor vs non-immune checkpoint inhibitor anticancer agents using ASCO Value Framework and ESMO Magnitude of Clinical Benefit Scale. JAMA Netw Open 2019;2:e196803.
- 16.↑
Ben-Aharon O, Magnezi R, Leshno M, . Mature versus registration studies of immuno-oncology agents: does value improve with time? JCO Oncol Pract 2020;16:e779–790.