Quantitative Imaging Assessment for Clinical Trials in Oncology

Authors:
Katherine E. Hersberger Department of Internal Medicine, University of Michigan Medical School;
University of Michigan Rogel Cancer Center; and

Search for other papers by Katherine E. Hersberger in
Current site
Google Scholar
PubMed
Close
 PhD
,
Mishal Mendiratta-Lala Department of Radiology, University of Michigan Medical School, Ann Arbor, Michigan.

Search for other papers by Mishal Mendiratta-Lala in
Current site
Google Scholar
PubMed
Close
 MD
,
Rocky Fischer University of Michigan Rogel Cancer Center; and

Search for other papers by Rocky Fischer in
Current site
Google Scholar
PubMed
Close
 MS
,
Ravi K. Kaza Department of Radiology, University of Michigan Medical School, Ann Arbor, Michigan.

Search for other papers by Ravi K. Kaza in
Current site
Google Scholar
PubMed
Close
 MD
,
Isaac R. Francis University of Michigan Rogel Cancer Center; and
Department of Radiology, University of Michigan Medical School, Ann Arbor, Michigan.

Search for other papers by Isaac R. Francis in
Current site
Google Scholar
PubMed
Close
 MD
,
Mirabella S. Olszewski University of Michigan Rogel Cancer Center; and

Search for other papers by Mirabella S. Olszewski in
Current site
Google Scholar
PubMed
Close
 MS
,
John F. Harju University of Michigan Rogel Cancer Center; and

Search for other papers by John F. Harju in
Current site
Google Scholar
PubMed
Close
 MBA, PMP
,
Wei Shi University of Michigan Rogel Cancer Center; and

Search for other papers by Wei Shi in
Current site
Google Scholar
PubMed
Close
 MBA, MS
,
Frank J. Manion University of Michigan Rogel Cancer Center; and

Search for other papers by Frank J. Manion in
Current site
Google Scholar
PubMed
Close
 PhD
,
Mahmoud M. Al-Hawary University of Michigan Rogel Cancer Center; and
Department of Radiology, University of Michigan Medical School, Ann Arbor, Michigan.

Search for other papers by Mahmoud M. Al-Hawary in
Current site
Google Scholar
PubMed
Close
 MD
, and
Vaibhav Sahai Department of Internal Medicine, University of Michigan Medical School;
Department of Internal Medicine, University of Michigan Medical School;

Search for other papers by Vaibhav Sahai in
Current site
Google Scholar
PubMed
Close
 MBBS, MS
Restricted access

Background: Objective radiographic assessment is crucial for accurately evaluating therapeutic efficacy and patient outcomes in oncology clinical trials. Imaging assessment workflow can be complex; can vary with institution; may burden medical oncologists, who are often inadequately trained in radiology and response criteria; and can lead to high interobserver variability and investigator bias. This article reviews the development of a tumor response assessment core (TRAC) at a comprehensive cancer center with the goal of providing standardized, objective, unbiased tumor imaging assessments, and highlights the web-based platform and overall workflow. In addition, quantitative response assessments by the medical oncologists, radiologist, and TRAC are compared in a retrospective cohort of patients to determine concordance. Patients and Methods: The TRAC workflow includes an image analyst who pre-reviews scans before review with a board-certified radiologist and then manually uploads annotated data on the proprietary TRAC web portal. Patients previously enrolled in 10 lung cancer clinical trials between January 2005 and December 2015 were identified, and the prospectively collected quantitative response assessments by the medical oncologists were compared with retrospective analysis of the same dataset by a radiologist and TRAC. Results: This study enlisted 49 consecutive patients (53% female) with a median age of 60 years (range, 29–78 years); 2 patients did not meet study criteria and were excluded. A linearly weighted kappa test for concordance for TRAC versus radiologist was substantial at 0.65 (95% CI, 0.46–0.85; standard error [SE], 0.10). The kappa value was moderate at 0.42 (95% CI, 0.20–0.64; SE, 0.11) for TRAC versus oncologists and only fair at 0.34 (95% CI, 0.12–0.55; SE, 0.11) for oncologists versus radiologist. Conclusions: Medical oncologists burdened with the task of tumor measurements in patients on clinical trials may introduce significant variability and investigator bias, with the potential to affect therapeutic response and clinical trial outcomes. Institutional imaging cores may help bridge the gap by providing unbiased and reproducible measurements and enable a leaner workflow.

Submitted June 12, 2018; accepted for publication June 18, 2019.

Author contributions: Study design: Hersberger, Sahai. Core design: Hersberger, Fischer, Francis, Olszewski, Harju, Shi, Manion, Sahai. Data acquisition: Hersberger, Mendiratta-Lala, Kaza, Francis, Al-Hawary, Sahai. Statistics: Sahai. Manuscript preparation: Hersberger, Fischer, Harju, Sahai. Manuscript editing: All authors.

Disclosures: The authors have disclosed that they have not received any financial considerations from any person or organization to support the preparation, analysis, results, or discussion of this article.

Funding: Research reported in this publication was supported by the NCI of the NIH under award number P30CA046592. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

Correspondence: Vaibhav Sahai, MBBS, MS, Department of Internal Medicine, University of Michigan Medical School, 1500 East Medical Center Drive, C412 Med Inn Building, Ann Arbor, MI 48109. Email: vsahai@med.umich.edu

Supplementary Materials

    • Supplemental Materials (PDF 474.65 KB)
  • Collapse
  • Expand
  • 1.

    Rosenkrantz AB, Mendiratta-Lala M, Bartholmai BJ, et al.. Clinical utility of quantitative imaging. Acad Radiol 2015;22:3349.

  • 2.

    Miller AB, Hoogstraten B, Staquet M, et al.. Reporting results of cancer treatment. Cancer 1981;47:207214.

  • 3.

    Therasse P, Arbuck SG, Eisenhauer EA, et al.. New guidelines to evaluate the response to treatment in solid tumors. J Natl Cancer Inst 2000;92:205216.

  • 4.

    Eisenhauer EA, Therasse P, Bogaerts J, et al.. New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1). Eur J Cancer 2009;45:228247.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 5.

    Choi H, Charnsangavej C, Faria SC, et al.. Correlation of computed tomography and positron emission tomography in patients with metastatic gastrointestinal stromal tumor treated at a single institution with imatinib mesylate: proposal of new computed tomography response criteria. J Clin Oncol 2007;25:17531759.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 6.

    Cheson BD, Horning SJ, Coiffier B, et al.. Report of an international workshop to standardize response criteria for non-Hodgkin’s lymphomas. J Clin Oncol 1999;17:1244.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 7.

    Cheson BD, Pfistner B, Juweid ME, et al.. Revised response criteria for malignant lymphoma. J Clin Oncol 2007;25:579586.

  • 8.

    Cheson BD, Fisher RI, Barrington SF, et al.. Recommendations for initial evaluation, staging, and response assessment of Hodgkin and non-Hodgkin lymphoma: the Lugano classification. J Clin Oncol 2014;32:30593067.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 9.

    Wahl RL, Jacene H, Kasamon Y, et al.. From RECIST to PERCIST: evolving considerations for PET response criteria in solid tumors. J Nucl Med 2009;50(Suppl 1):122S150S.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 10.

    Wen PY, Macdonald DR, Reardon DA, et al.. Updated response assessment criteria for high-grade gliomas: Response Assessment in Neuro-Oncology Working Group. J Clin Oncol 2010;28:19631972.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 11.

    Lin NU, Lee EQ, Aoyama H, et al.. Response assessment criteria for brain metastases: proposal from the RANO group. Lancet Oncol 2015;16:e270278.

  • 12.

    Scher HI, Halabi S, Tannock I, et al.. Design and end points of clinical trials for patients with progressive prostate cancer and castrate levels of testosterone: recommendations of the Prostate Cancer Clinical Trials Working Group. J Clin Oncol 2008;26:11481159.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 13.

    Scher HI, Morris MJ, Stadler WM, et al.. Trial design and objectives for castration-resistant prostate cancer: updated recommendations from the Prostate Cancer Clinical Trials Working Group 3. J Clin Oncol 2016;34:14021418.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 14.

    Wolchok JD, Hoos A, O’Day S, et al.. Guidelines for the evaluation of immune therapy activity in solid tumors: immune-related response criteria. Clin Cancer Res 2009;15:74127420.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 15.

    Nishino M, Giobbie-Hurder A, Gargano M, et al.. Developing a common language for tumor response to immunotherapy: immune-related response criteria using unidimensional measurements. Clin Cancer Res 2013;19:39363943.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 16.

    Seymour L, Bogaerts J, Perrone A, et al.. iRECIST: guidelines for response criteria for use in trials testing immunotherapeutics. Lancet Oncol 2017;18:e143152.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 17.

    Okada H, Weller M, Huang R, et al.. Immunotherapy response assessment in neuro-oncology: a report of the RANO working group. Lancet Oncol 2015;16:e534542.

  • 18.

    On and offsite image reads: is basing drug efficacy on the site read risky business? Available at: http://www.appliedclinicaltrialsonline.com/and-offsite-image-reads. Accessed October 14, 2019.

    • PubMed
    • Export Citation
  • 19.

    U.S. Food and Drug Administration. Guidance Document. Developing medical imaging drug and biological products part 3: design, analysis, and interpretation of clinical studies. Available at: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/developing-medical-imaging-drug-and-biological-products-part-3-design-analysis-and-interpretation. Accessed October 14, 2019.

    • PubMed
    • Export Citation
  • 20.

    U.S. Food and Drug Administration. Clinical trial imaging endpoint process standards: guidance for industry. Available at: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/clinical-trial-imaging-endpoint-process-standards-guidance-industry. Accessed October 14, 2019.

    • PubMed
    • Export Citation
  • 21.

    Graham MM, Badawi RD, Wahl RL. Variations in PET/CT methodology for oncologic imaging at U.S. academic medical centers: an imaging response assessment team survey. J Nucl Med 2011;52:311317.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 22.

    Ford R, Schwartz L, Dancey J, et al.. Lessons learned from independent central review. Eur J Cancer 2009;45:268274.

  • 23.

    Koshkin VS, Bolejack V, Schwartz LH, et al.. Assessment of imaging modalities and response metrics in Ewing sarcoma: correlation with survival. J Clin Oncol 2016;34:36803685.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 24.

    Nygren P, Blomqvist L, Bergh J, et al.. Radiological assessment of tumour response to anti-cancer drugs: time to reappraise. Acta Oncol 2008;47:316318.

  • 25.

    Dodd LE, Korn EL, Freidlin B, et al.. Blinded independent central review of progression-free survival in phase III clinical trials: important design element or unnecessary expense? J Clin Oncol 2008;26:37913796.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 26.

    Von Hoff DD, Ervin T, Arena FP, et al.. Increased survival in pancreatic cancer with nab-paclitaxel plus gemcitabine. N Engl J Med 2013;369:16911703.

  • 27.

    Sullivan DC. Imaging as a quantitative science. Radiology 2008;248:328332.

  • 28.

    Yankeelov TE, Mankoff DA, Schwartz LH, et al.. Quantitative imaging in cancer clinical trials. Clin Cancer Res 2016;22:284290.

  • 29.

    Folio LR, Nelson CJ, Benjamin M, et al.. Quantitative radiology reporting in oncology: survey of oncologists and radiologists. AJR Am J Roentgenol 2015;205:W233243.

  • 30.

    Sevenster M, Chang P, Bozeman J, et al.. Radiologic measurement dictation and transcription error rates in RECIST (Response Evaluation Criteria in Solid Tumors) clinical trials: a limitation of the radiology narrative report to accurately communicate quantitative data. Presented at the Radiological Society of North America 2013 Scientific Assembly and Annual Meeting; December 1–6, 2013; Chicago, IL.

    • PubMed
    • Export Citation
  • 31.

    Shao T, Wang L, Templeton AJ, et al.. Use and misuse of waterfall plots. J Natl Cancer Inst 2014;106:dju331.

  • 32.

    Urban T, Zondervan RL, Hanlon WB, et al.. Imaging analysts: how bringing onboard multimodality trained personnel can impact oncology trials. Available at: http://www.appliedclinicaltrialsonline.com/imaging-analysts?id=&sk=&date=&pageID=5. Accessed October 14, 2019.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 33.

    Tang PA, Pond GR, Chen EX. Influence of an independent review committee on assessment of response rate and progression-free survival in phase III clinical trials. Ann Oncol 2010;21:1926.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 34.

    Hersberger KE, Fischer R, Bebee PA, et al.. On TRAC at the Rogel Cancer Center: centralized trial imaging metrics system. Presented at the Association of American Cancer Institutes Clinical Research Innovation Meeting; July 11–12, 2018; Chicago, Illinois. Available at: https://www.aaci-cancer.org/Files/Admin/CRI/2018-Abstracts-and-Posters.pdf.

    • PubMed
    • Export Citation
  • 35.

    Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159174.

  • 36.

    Jaffe TA, Wickersham NW, Sullivan DC. Quantitative imaging in oncology patients: part 2, oncologists’ opinions and expectations at major U.S. cancer centers. AJR Am J Roentgenol 2010;195:W1930.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 37.

    Rubin DL, Willrett D, O’Connor MJ, et al.. Automated tracking of quantitative assessments of tumor burden in clinical trials. Transl Oncol 2014;7:2335.

  • 38.

    Keil S, Barabasch A, Dirrichs T, et al.. Target lesion selection: an important factor causing variability of response classification in the Response Evaluation Criteria for Solid Tumors 1.1. Invest Radiol 2014;49:509517.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 39.

    Yoon SH, Kim KW, Goo JM, et al.. Observer variability in RECIST-based tumour burden measurements: a meta-analysis. Eur J Cancer 2016;53:515.

  • 40.

    Gierada DS, Pilgram TK, Ford M, et al.. Lung cancer: interobserver agreement on interpretation of pulmonary findings at low-dose CT screening. Radiology 2008;246:265272.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 41.

    Erasmus JJ, Gladish GW, Broemeling L, et al.. Interobserver and intraobserver variability in measurement of non-small-cell carcinoma lung lesions: implications for assessment of tumor response. J Clin Oncol 2003;21:25742582.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 3012 575 21
PDF Downloads 880 172 13
EPUB Downloads 0 0 0