Background: Many new targeted cancer drugs have received FDA approval based on durable responses in nonrandomized controlled trials (non-RCTs). The goal of this study was to evaluate whether the response rates (RRs) and durations of response (DoRs) of targeted cancer drugs observed in non-RCTs are consistent when these drugs are tested in RCTs. Methods: We used the FDA’s Table of Pharmacogenomic Biomarkers in Drug Labeling to identify cancer drugs that were approved based on changes in biomarker endpoints through December 2017. We then identified the non-RCTs and RCTs for these drugs for the given indications and extracted the RRs and DoRs. We compared the RRs and median DoR in non-RCTs versus RCTs using the ratio of RRs and the ratio of DoRs, defined as the RRs (or DoRs) in non-RCTs divided by the RRs (or DoRs) in RCTs. The ratio of RRs or DoRs was pooled across the trial pairs using random-effects meta-analysis. Results: Of the 21 drug–indication pairs selected, both non-RCTs and RCTs were available for 19. The RRs and DoRs in non-RCTs were greater than those in RCTs in 63% and 87% of cases, respectively. The pooled ratio of RRs was 1.06 (95% CI, 0.95–1.20), and the pooled ratio of DoRs was 1.17 (95% CI, 1.03–1.33). RRs and DoRs derived from non-RCTs were also poor surrogates for overall survival derived from RCTs. Conclusions: The RRs were not different between non-RCTs and RCTs of cancer drugs approved based on changes to a biomarker, but the DoRs in non-RCTs were significantly higher than in RCTs. Caution must be exercised when approving or prescribing targeted drugs based on data on durable responses derived from non-RCTs, because the responses could be overestimates and poor predictors of survival benefit.