Are Surrogate Endpoints Unbiased Metrics in Clinical Benefit Scores of the ASCO Value Framework?

View More View Less
  • a Odette Cancer Centre, Sunnybrook Health Sciences Centre, and
  • b Department of Medicine, University of Toronto;
  • c Canadian Agency for Drugs and Technologies in Health; and
  • d Canadian Centre for Applied Research in Cancer Control, Toronto, Ontario, Canada.
Restricted access

Background: Clinical benefit scores (CBS) are key elements of the ASCO Value Framework (ASCO-VF) and are weighted based on a hierarchy of efficacy endpoints: hazard ratio for death (HR OS), median overall survival (mOS), HR for disease progression (HR PFS), median progression-free survival (mPFS), and response rate (RR). When HR OS is unavailable, the other endpoints serve as “surrogates” to calculate CBS. CBS are computed from PFS or RR in 39.6% of randomized controlled trials. This study examined whether surrogate-derived CBS offer unbiased scoring compared with HR OS–derived CBS. Methods: Using the ASCO-VF, CBS for advanced disease settings were computed for randomized controlled trials of oncology drug approvals by the FDA, European Medicines Agency, and Health Canada in January 2006 through December 2017. Mean differences of surrogate-derived CBS minus HR OS–derived CBS assessed the tendency of surrogate-derived CBS to overestimate or underestimate clinical benefit. Spearman’s correlation evaluated the association between surrogate- and HR OS–derived CBS. Mean absolute error assessed the average difference between surrogate-derived CBS relative to HR OS–derived CBS. Results: CBS derived from mOS, HR PFS, mPFS, and RR overestimated HR OS–derived CBS in 58%, 68%, 77%, and 55% of pairs and overall by an average of 5.62 (n=90), 6.86 (n=110), 29.81 (n=101), and 3.58 (n=108), respectively. Correlation coefficients were 0.80 (95% CI, 0.70–0.86), 0.38 (0.20–0.53), 0.20 (0.00–0.38), and 0.01 (–0.18 to 0.19) for mOS-, HR PFS–, mPFS-, and RR-derived CBS, respectively, and mean absolute errors were 11.32, 12.34, 40.40, and 18.63, respectively. Conclusions: Based on the ASCO-VF algorithm, HR PFS–, mPFS-, and RR-derived CBS are suboptimal surrogates, because they were shown to be biased and poorly correlated to HR OS–derived CBS. Despite lower weighting than OS in the ASCO-VF algorithm, PFS still overestimated CBS. Simple rescaling of surrogate endpoints may not improve their validity within the ASCO-VF given their poor correlations with HR OS–derived CBS.

Submitted October 27, 2018; accepted for publication June 18, 2019.

Previous presentation: An abstract was presented at the 2017 Canadian Centre for Applied Research in Cancer Control Conference, May 25–26, 2017, Toronto, Ontario, Canada, and at the 2017 ASCO Annual Meeting, June 2–6, 2017, Chicago, Illinois.

Author contributions: Principal investigator: Chan. Search strategy: Chan. Study concept and design: Chan. Data acquisition: Cheng, Jiang, McDonald, Arciero, Ezeife, Rahmadian. Data analysis and interpretation: Cheng, Chan. Manuscript preparation: Cheng, Cheung, Chan. Critical revision: All authors. Guarantor of work and final approval: All authors.

Disclosures: The authors have disclosed that they have not received any financial considerations from any person or organization to support the preparation, analysis, results, or discussion of this article.

Correspondence: Kelvin K.W. Chan, MD, MSc, PhD, Division of Medical Oncology, Odette Cancer Centre, Sunnybrook Health Sciences Centre, 2075 Bayview Avenue, T2-058, Toronto, Ontario M4N 3M5, Canada. Email: kelvin.chan@sunnybrook.ca

View associated content

Supplementary Materials

    • Supplemental Materials (PDF 607.55 KB)
  • 1.

    Schnipper LE, Davidson NE, Wollins DS, . Updating the American Society of Clinical Oncology value framework: revisions and reflections in response to comments received. J Clin Oncol 2016;34:29252934.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 2.

    Schnipper LE, Davidson NE, Wollins DS, . American Society of Clinical Oncology statement: a conceptual framework to assess the value of cancer treatment options. J Clin Oncol 2015;33:25632577.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 3.

    Booth CM, Cescon DW, Wang L, . Evolution of the randomized controlled trial in oncology over three decades. J Clin Oncol 2008;26:54585464.

  • 4.

    Cressman S, Browman GP, Hoch JS, . A time-trend economic analysis of cancer drug trials. Oncologist 2015;20:729736.

  • 5.

    Saluja R, Arciero VS, Cheng S, . Examining trends in cost and clinical benefit of novel anticancer drugs over time. J Oncol Pract 2018;14:e280294.

  • 6.

    Cheng S, McDonald EJ, Cheung MC, . Do the American Society of Clinical Oncology Value Framework and the European Society of Medical Oncology Magnitude of Clinical Benefit Scale measure the same construct of clinical benefit? J Clin Oncol 2017;35:27642771.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 7.

    Hyndman RJ, Athanasopoulos G. Forecasting: Principles and Practice, 2nd ed. Lexington, KY: OTexts; 2018.

  • 8.

    Zaki R, Bulgiba A, Ismail R, . Statistical methods used to test for agreement of medical instruments measuring continuous variables in method comparison studies: a systematic review. PLoS One 2012;7:e37908.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 9.

    Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;327:307310.

  • 10.

    U.S. Food & Drug Administration. Hematology/Oncology (cancer) approvals and safety notifications. Available at: https://www.fda.gov/drugs/informationondrugs/approveddrugs/ucm279174.htm. Accessed May 30, 2018.

    • Export Citation
  • 11.

    European Medicines Agency. European public assessment reports. Available at: http://www.ema.europa.eu/ema/index.jsp?curl=pages/medicines/landing/epar_search.jsp&mid=WC0b01ac058001d125. Accessed May 30, 2018.

    • Export Citation
  • 12.

    Health Canada. The drug and health product register. Available at: https://hpr-rps.hres.ca/reg-content/summary-basis-decision-result.php?lang=en&term=. Accessed May 30, 2018.

    • Export Citation
  • 13.

    Cherny NI, Sullivan R, Dafni U, . A standardised, generic, validated approach to stratify the magnitude of clinical benefit that can be anticipated from anti-cancer therapies: the European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS). Ann Oncol 2015;26:15471573.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 14.

    Cherny NI, Dafni U, Bogaerts J, . ESMO-Magnitude of Clinical Benefit Scale version 1.1. Ann Oncol 2017;28:23402366.

  • 15.

    Saad ED, Buyse M. Statistical controversies in clinical research: end points other than overall survival are vital for regulatory approval of anticancer agents. Ann Oncol 2016;27:373378.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 16.

    Miller KD, Chap LI, Holmes FA, . Randomized phase III trial of capecitabine compared with bevacizumab plus capecitabine in patients with previously treated metastatic breast cancer. J Clin Oncol 2005;23:792799.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 17.

    Hellmann MD, Kris MG, Rudin CM. Medians and milestones in describing the path to cancer cures: telling “tails.” JAMA Oncol 2016;2:167168.

  • 18.

    Rahman R, Fell G, Trippa L, . Violations of the proportional hazards assumption in randomized phase III oncology clinical trials [abstract]. J Clin Oncol 2018;36(Suppl):Abstract 2543.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 19.

    Saad ED, Katz A, Hoff PM, . Progression-free survival as surrogate and as true end point: insights from the breast and colorectal cancer literature. Ann Oncol 2010;21:712.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 20.

    Zhao F. Surrogate end points and their validation in oncology clinical trials. J Clin Oncol 2016;34:14361437.

  • 21.

    Ellis LM, Bernstein DS, Voest EE, . American Society of Clinical Oncology perspective: raising the bar for clinical trials by defining clinically meaningful outcomes. J Clin Oncol 2014;32:12771280.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 22.

    Robinson AG, Booth CM, Eisenhauer EA. Progression-free survival as an end-point in solid tumours—perspectives from clinical trials and clinical practice. Eur J Cancer 2014;50:23032308.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 23.

    Burzykowski T, Molenberghs G, Buyse M, . Validation of surrogate end points in multiple randomized trials with failure time end points. J R Stat Soc Ser C Appl Stat 2001;50:405422.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 24.

    Booth CM, Eisenhauer EA. Progression-free survival: meaningful or simply measurable? J Clin Oncol 2012;30:10301033.

  • 25.

    Sherrill B, Amonkar M, Wu Y, . Relationship between effects on time-to-disease progression and overall survival in studies of metastatic breast cancer. Br J Cancer 2008;99:15721578.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 26.

    Hackshaw A, Knight A, Barrett-Lee P, . Surrogate markers and survival in women receiving first-line combination anthracycline chemotherapy for advanced breast cancer. Br J Cancer 2005;93:12151221.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 27.

    Miksad RA, Zietemann V, Gothe R, . Progression-free survival as a surrogate endpoint in advanced breast cancer. Int J Technol Assess Health Care 2008;24:371383.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 28.

    Soria JC, Massard C, Le Chevalier T. Should progression-free survival be the primary measure of efficacy for advanced NSCLC therapy? Ann Oncol 2010;21:23242332.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 29.

    Ciani O, Davis S, Tappenden P, . Validation of surrogate endpoints in advanced solid tumors: systematic review of statistical methods, results, and implications for policy makers. Int J Technol Assess Health Care 2014;30:312324.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 30.

    Schnipper LE, Schilsky RL. Converging on the value of value frameworks. J Clin Oncol 2017;35:27322734.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 995 995 16
PDF Downloads 294 294 15
EPUB Downloads 0 0 0