Randomized Controlled Trials and 21st Century Epistemology

View More View Less

In the 20th century, empiricism, in the form of personal, hands-on experimentation in science and medicine, was paramount. As an example, Barry reports of Oswald Avery, the Rockefeller Institute scientist who established DNA as the genetic material, “For he always did everything, down to washing the glassware, with precision and discipline.”1

In the twenty-first century we are witnessing what I would suggest is a step backward from the principles of empiricism. Direct, personal, clinical, and research experience in medicine is increasingly considered an impediment rather than a virtue in the search for scientific truth. The methods espoused by the McMaster University evidence-based medicine (EBM) group and, most recently, in the pages of JAMA by Otis Brawley, MD, and the American Cancer Society (ACS), relegate clinical experts to a comment-only role, and delegate collection, grading, and interpretation of data, as well as writing of clinical practice guideline manuscripts, to EBM practitioners.2 A key assumption in this process is that EBM experts are the correct group to harvest pertinent evidence, cull the strongest evidence, consolidate and analyze the data contained in this evidence, and write clinical practice guidelines. In selection of the strongest evidence, EBM principles place a special cachet on evidence collected in prospective randomized clinical trials (RCTs) despite the fact that RCTs sometimes, perhaps even often, contain flaws that weaken the strength of that evidence.

An example of this can be seen in the CALGB 9334 trial discussed in this issue of JNCCN. Multiple problems can be seen in study design, management, statistical analysis, and interpretation for this RCT, and the article's authors have been responsible in candidly discussing these weaknesses.

The concern, however, is that those gathering evidence and writing guidelines on the subject of malignant pleural effusions at some later date, who will typically not have substantial clinical experience in the treatment of malignant pleural effusions—and may not even be physicians—could incorrectly consider the data in this publication to be stronger evidence than it truly represents.

For example, EBM experts without clinical experience may not recognize that the study design was faulty, in that it randomized individuals with trapped lung to a treatment, talc pleurodesis, that is often ineffective in these patients. Additionally, it used percent reexpansion as a criterion for success, when such reexpansion cannot be expected in patients with a trapped lung. Guideline authors without clinical experience might also miss the implications of skewed (> 60%) accrual of patients with lung cancer into the study—that survival in these patients is typically much shorter than in patients with breast cancer, for example. They might also fail to recognize that most patients were treated at institutions with very low volume (20 participating centers enrolled between 1 and 7 cases) when many practitioners were just learning to use indwelling pleural catheters effectively.

Furthermore, very low accrual (12%) of projected numbers randomized to bedside talc pleurodesis versus tunneled catheter drainage, and a long unexplained delay in publication of the study further weaken the strength of conclusions drawn from study data. An important variance in statistical analysis is also seen. The original primary end point, lung reexpansion of 90%, was changed to expansion of 70% or greater because only 36% of patients achieved greater than 90% reexpansion. Such post hoc redefinition of a primary study end point is problematic.

The authors conclude that the combined success rate was higher with tunneled catheter drainage than with pleurodesis (62% vs. 46%; P = .064). This appears to be true when considering a mixture of cases containing both trapped and untrapped lungs, but evidence did not show that indwelling catheter drainage is superior in patients with full lung reexpansion. Furthermore, because the rates of success reported in this study are inferior to those published in prior RCTs and large single-institution retrospective series, selection of the “strongest” RCT data values in mathematical modeling studies might yield misleadingly poor results in terms of effectiveness, cost-effectiveness, and value.36

One observation that deserves emphasis is that, although no differences were seen between baseline dyspnea score and lung expansion, patients with better expansion had lower dyspnea scores and quality-of-life measures. This suggests that avoidance of trapped lung through earlier intervention may be important in patients with malignant pleural effusions.

In summary, data collected from multiple, low-volume centers with clinicians who are new to the studied technique may be less valid than similar data produced by experienced clinicians in high-volume centers. And yet, because the low-volume evidence is collected in a RCT, EBM will rank the results as superior. The CALGB study discussed illustrates this point and the potential deficiencies of randomized control studies.

References

  • 1

    Barry JM. The Great Influenza: The Epic Story of the Deadliest Plague in History. New York, NY: Penguin Group.

  • 2

    Brawley O, Byers T, Chen A. New American Cancer Society process for creating trustworthy cancer screening guidelines. JAMA 2011;306:24952499.

    • Search Google Scholar
    • Export Citation
  • 3

    Dresler CM, Olak J, Herndon JE II.; Cooperative Groups Cancer and Leukemia Group B; Eastern Cooperative Oncology Group; North Central Cooperative Oncology Group; Radiation Therapy Oncology Group. Phase III intergroup study of talc poudrage vs talc slurry sclerosis for malignant pleural effusion. Chest 2005;127:909915.

    • Search Google Scholar
    • Export Citation
  • 4

    Tremblay A, Michaud G. Single-center experience with 250 tunnelled pleural catheter insertions for malignant pleural effusion. Chest 2006;129:362368.

    • Search Google Scholar
    • Export Citation
  • 5

    Steger V, Mika U, Toomes H. Who gains most? A 10-year experience with 611 thoracoscopic talc pleurodeses. Ann Thorac Surg 2007;83:19401945.

  • 6

    Warren WH, Kalimi R, Khodadadian LM, Kim AW. Management of malignant pleural effusions using the Pleur(x) catheter. Ann Thorac Surg 2008;85:10491055.

    • Search Google Scholar
    • Export Citation

If the inline PDF is not rendering correctly, you can download the PDF file here.

Frederic W. Grannis Jr, MD, is Clinical Professor of Thoracic Surgery at City of Hope National Medical Center in Duarte, CA. He has a long-term clinical and research interest in management of pleural and pericardial effusions in cancer patients and has authored the chapter on “Fluid Complications” in 13 editions of Cancer Management: A Multidisciplinary Approach: Medical, Surgical and Radiation Oncology.

The ideas and viewpoints expressed in this editorial are those of the author and do not necessarily represent any policy, position, or program of NCCN.

  • 1

    Barry JM. The Great Influenza: The Epic Story of the Deadliest Plague in History. New York, NY: Penguin Group.

  • 2

    Brawley O, Byers T, Chen A. New American Cancer Society process for creating trustworthy cancer screening guidelines. JAMA 2011;306:24952499.

    • Search Google Scholar
    • Export Citation
  • 3

    Dresler CM, Olak J, Herndon JE II.; Cooperative Groups Cancer and Leukemia Group B; Eastern Cooperative Oncology Group; North Central Cooperative Oncology Group; Radiation Therapy Oncology Group. Phase III intergroup study of talc poudrage vs talc slurry sclerosis for malignant pleural effusion. Chest 2005;127:909915.

    • Search Google Scholar
    • Export Citation
  • 4

    Tremblay A, Michaud G. Single-center experience with 250 tunnelled pleural catheter insertions for malignant pleural effusion. Chest 2006;129:362368.

    • Search Google Scholar
    • Export Citation
  • 5

    Steger V, Mika U, Toomes H. Who gains most? A 10-year experience with 611 thoracoscopic talc pleurodeses. Ann Thorac Surg 2007;83:19401945.

  • 6

    Warren WH, Kalimi R, Khodadadian LM, Kim AW. Management of malignant pleural effusions using the Pleur(x) catheter. Ann Thorac Surg 2008;85:10491055.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 61 16 2
PDF Downloads 34 11 2
EPUB Downloads 0 0 0