Validating a Patient-Reported Outcomes–Derived Algorithm for Classifying Symptom Complexity Levels Among Patients With Cancer

View More View Less
  • 1 Alberta Health Services, and
  • 2 University of Calgary, Calgary, Alberta, Canada.
Restricted access

Background: The patient-reported outcomes (PROs) symptom complexity algorithm, derived from self-reported symptom scores using the Edmonton Symptom Assessment System and concerns indicated on the Canadian Problem Checklist, has not been validated extensively. Methods: This is a retrospective chart review study using data from the Alberta Cancer Registry and electronic medical records from Alberta Health Services. The sample includes patients with cancer who visited a cancer facility in Alberta, Canada, from February 2016 through November 2017 (n=1,466). Results: The effect size (d=1.2) indicates that the magnitude of difference in health status between the severe- and low-complexity groups is large. The symptom complexity algorithm effectively classified subgroups of patients with cancer with distinct health status. Using Karnofsky performance status, the algorithm shows a sensitivity of 70.3%, specificity of 84.1%, positive predictive value of 79.1%, negative predictive value of 76.7%, and accuracy of 77.7%. An area under the receiver operating characteristic of 0.824 was found for the complexity algorithm, which is generally regarded as good, This same finding was also regarded as superior to the alternative algorithm generated by 2-step cluster analysis (area under the curve, 0.721). Conclusions: The validity of the PRO-derived symptom complexity algorithm is established in this study. The algorithm demonstrated satisfactory accuracy against a clinician-driven complexity assessment and a strong correlation with the known group analysis. Furthermore, the algorithm showed a higher screening capacity compared with the algorithm generated from 2-step cluster analysis, reinforcing the importance of contextualization when classifying patients’ symptoms, rather than purely relying on statistical outcomes. The algorithm carries importance in clinical settings, acting as a symptom complexity flag, helping healthcare teams identify which patients may need more timely, targeted, and individualized patient symptom management.

Submitted August 13, 2019; accepted for publication May 7, 2020.

Author contributions: Study concept and design: Watson. Sample preparation: Photitai, Chmielewski, Smith. Critical feedback, research, and analysis: All authors. Manuscript preparation: Watson, Qi, DeIure. Manuscript editing: DeIure, Photitai.

Disclosures: The authors have disclosed that they have not received any financial consideration from any person or organization to support the preparation, analysis, results, or discussion of this article.

Correspondence: Linda Watson, RN, PhD, Alberta Health Services, Holy Cross Site, Box ACB, 2210-2nd Street SW, Calgary, Alberta, T2S 3C3 Canada. Email: Linda.Watson@albertahealthservices.ca
  • 1.

    Shippee ND, Shah ND, May CR, . Cumulative complexity: a functional, patient-centered model of patient complexity can improve research and practice. J Clin Epidemiol 2012;65:10411051.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 2.

    Pask S, Pinto C, Bristowe K, . A framework for complexity in palliative care: a qualitative study with patients, family carers and professionals. Palliat Med 2018;32:10781090.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 3.

    Anderson GF. Medicare and chronic conditions. N Engl J Med 2005;353:305309.

  • 4.

    Karabulut N, Erci B, Ozer N, . Symptom clusters and experiences of patients with cancer. J Adv Nurs 2010;66:10111021.

  • 5.

    Barsevick AM, Whitmer K, Nail LM, . Symptom cluster research: conceptual, design, measurement, and analysis issues. J Pain Symptom Manage 2006;31:8595.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 6.

    Barsevick AM. The elusive concept of the symptom cluster. Oncol Nurs Forum 2007;34:971980.

  • 7.

    Snyder C, Brundage M, Rivera YM, . PRO-cision Medicine Methods Toolkit to address the challenges of personalizing cancer care using patient-reported outcomes: introduction to the supplement. Med Care 2019;57(Suppl 1):S17.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 8.

    Watanabe S, Nekolaichuk C, Beaumont C, . The Edmonton Symptom Assessment System—what do patients think? Support Care Cancer 2009;17:675683.

  • 9.

    Watanabe SM, Nekolaichuk C, Beaumont C, . A multicenter study comparing two numerical versions of the Edmonton Symptom Assessment System in palliative care patients. J Pain Symptom Manage 2011;41:456468.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 10.

    Bultz BD, Groff SL, Fitch M, . Implementing screening for distress, the 6th vital sign: a Canadian strategy for changing practice. Psychooncology 2011;20:463469.

  • 11.

    Canadian Partnership Against Cancer. The 2012 cancer system performance report. Accessed August 18, 2019. Available at: https://www.partnershipagainstcancer.ca/topics/2012-cancer-system-performance-report/

  • 12.

    Shi Q, Smith TG, Michonski JD, . Symptom burden in cancer survivors 1 year after diagnosis: a report from the American Cancer Society’s studies of cancer survivors. Cancer 2011;117:27792790.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 13.

    Statistics Solutions. Conduct and interpret a cluster analysis. Accessed June 18, 2019. Available at: https://www.statisticssolutions.com/cluster-analysis-2/

  • 14.

    Rabin R, de Charro F. EQ-5D: a measure of health status from the EuroQol Group. Ann Med 2001;33:337343.

  • 15.

    Grandy S, Fox KM. EQ-5D visual analog scale and utility index values in individuals with diabetes and at risk for diabetes: findings from the Study to Help Improve Early evaluation and management of risk factors Leading to Diabetes (SHIELD). Health Qual Life Outcomes 2008;6:18.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 16.

    Augustovski F, Rey-Ares L, Irazola V, . An EQ-5D-5L value set based on Uruguayan population preferences. Qual Life Res 2016;25:323333.

  • 17.

    Schag CC, Heinrich RL, Ganz PA. Karnofsky performance status revisited: reliability, validity, and guidelines. J Clin Oncol 1984;2:187193.

  • 18.

    Yates JW, Chalmer B, McKegney FP. Evaluation of patients with advanced cancer using the Karnofsky performance status. Cancer 1980;45:22202224.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 19.

    Crooks V, Waller S, Smith T, . The use of the Karnofsky performance scale in determining outcomes and risk in geriatric outpatients. J Gerontol 1991;46:M139144.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 20.

    Hollen PJ, Gralla RJ, Kris MG, . Measurement of quality of life in patients with lung cancer in multicenter trials of new therapies. Psychometric assessment of the Lung Cancer Symptom Scale. Cancer 1994;73:20872098.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 21.

    Chang VT, Hwang SS, Feuerman M. Validation of the Edmonton Symptom Assessment Scale. Cancer 2000;88:21642171.

  • 22.

    Moro C, Brunelli C, Miccinesi G, . Edmonton symptom assessment scale: Italian validation in two palliative care settings. Support Care Cancer 2006;14:3037.

  • 23.

    Carvajal A, Centeno C, Watson R, . A comprehensive study of psychometric properties of the Edmonton Symptom Assessment System (ESAS) in Spanish advanced cancer patients. Eur J Cancer 2011;47:18631872.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 24.

    Paiva CE, Manfredini LL, Paiva BS, . The Brazilian version of the Edmonton Symptom Assessment System (ESAS) is a feasible, valid and reliable instrument for the measurement of symptoms in advanced cancer patients. PLoS One 2015;10:e0132073.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 25.

    Portz JD, Kutner JS, Blatchford PJ, . High symptom burden and low functional status in the setting of multimorbidity. J Am Geriatr Soc 2017;65:22852289.

  • 26.

    Cohen J. The statistical power of abnormal-social psychological research: a review. J Abnorm Soc Psychol 1962;65:145153.

  • 27.

    Péus D, Newcomb N, Hofer S. Appraisal of the Karnofsky performance status and proposal of a simple algorithmic system for its evaluation. BMC Med Inform Decis Mak 2013;13:7279.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 28.

    Tandon P, Reddy KR, O’Leary JG, . A Karnofsky performance status-based score predicts death after hospital discharge in patients with cirrhosis. Hepatology 2017;65:217224.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 29.

    Thuluvath PJ, Thuluvath AJ, Savva Y. Karnofsky performance status before and after liver transplantation predicts graft and patient survival. J Hepatol 2018;69:818825.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 30.

    Sandanger I, Moum T, Ingebrigtsen G, . Concordance between symptom screening and diagnostic procedure: the Hopkins Symptom Checklist-25 and the Composite International Diagnostic Interview I. Soc Psychiatry Psychiatr Epidemiol 1998;33:345354.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 31.

    Basch E, Deal AM, Kris MG, . Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. J Clin Oncol 2016;34:557565.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 32.

    Broderick JE, Schneider S, Junghaenel DU, . Validity and reliability of patient-reported outcomes measurement information system instruments in osteoarthritis. Arthritis Care Res (Hoboken) 2013;65:16251633.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 33.

    Goldstein H, Spiegelhalter DJ. League tables and their limitations: statistical issues in comparisons of institutional performance. J R Stat Soc Ser A 1996;159:385443.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 34.

    Selby P, Velikova G. Taking patient reported outcomes centre stage in cancer research – why has it taken so long? Res Involv Engagem 2018;4:2529.

  • 35.

    Hutchings HA, Alrubaiy L. Patient-reported outcome measures in routine clinical care: the PROMise is a better future? Dig Dis Sci 2017;62:18411843.

  • 36.

    Gilbert A, Sebag-Montefiore D, Davidson S, . Use of patient-reported outcomes to measure symptoms and health related quality of life in the clinic. Gynecol Oncol 2015;136:429439.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 427 427 427
PDF Downloads 120 120 120
EPUB Downloads 0 0 0