Background: Current toxicity evaluation is primarily focused on high-grade adverse events (AEs) reported by clinicians. However, the cumulative effect of multiple lower-grade AEs may also impact patients' quality of life (QoL). Further, patient-reported toxicity may be more representative of patients' treatment experiences. This study aimed to determine whether cumulative toxicity comprising all-grade AEs is more associated with QoL than cumulative toxicity comprising high-grade AEs only, and whether patient-reported cumulative toxicity is more associated with QoL than clinician-reported cumulative toxicity. Methods: Patients with metastatic castration-naïve prostate cancer participating in the phase III GETUG-AFU 15 trial completed questionnaires on AEs (at 3 and 6 months) and QoL (at baseline and 3 and 6 months). Clinicians reported AEs during clinical visits. Cumulative toxicity scores were calculated for clinicians and patients in 3 ways: total number of high-grade AEs, total number of all-grade AEs, and total number of all AEs multiplied by their grade (severity score). Relationships between cumulative toxicity scores and QoL were studied using longitudinal regression analyses; unstandardized (B) and standardized regression coefficients (β) are reported. Results: Of 385 patients, 184 with complete QoL and toxicity data were included. Clinician-reported all-grade AEs (B, −2.2; 95% CI, −3.3 to −1.1; P<.01) and severity score (B, −1.4; 95% CI, −2.2 to −0.7; P<.01) were associated with deteriorated physical QoL, whereas the total number of high-grade AEs was not. All patient-reported scores were significantly (P<.01 for all) associated with deteriorated physical and global QoL. Standardized regression coefficients indicated that patient-reported toxicity scores were more associated with QoL outcomes than clinician-reported scores, with the strongest association found for the all-grade AEs and severity cumulative toxicity scores. Conclusions: Patient- and clinician-based cumulative toxicity scores comprising all-grade AEs better reflect impact on patient QoL than toxicity scores comprising high-grade AEs only. To assess the effect of toxicity on QoL, patient-reported cumulative toxicity scores are preferred.