JAMA feature: Most clinical trials fail to add meaningful evidence
test tubes - 80.40 Kb
If randomized controlled clinical trials are the gold standard for advancing knowledge in medical care, then the research community has been panning more particles than nuggets. That is the finding of a comprehensive analysis that focused on cardiovascular, mental health and oncology interventional trials. “Most of the trials are quite small,” author Robert M. Califf, MD, said in an interview.      

The study, published May 2 in the Journal of the American Medical Association, was designed to identify the characteristics of clinical trials registered in the ClinicalTrials.gov database as a first step for assessing the capacity of the clinical trial enterprise for providing robust evidence to guide clinical decision making and policy.

The study was conducted by the Clinical Trials Transformation Initiative (CTTI), a 60-member public-private partnership whose executive committee is co-chaired by Califf, director of the Duke Translational Medicine Institute and vice chancellor of clinical research at Duke University Medical Center in Durham, N.C.

“There is concern that clinical trials, particularly in the U.S., are increasingly inefficient and expensive,” said Califf. For their analysis, Califf and colleagues downloaded data on 96,346 clinical trials on Sept. 27, 2010, into a relational database. They selected the three therapeutic areas of cardiovascular, mental health and oncology because the three together account for the largest number of disability-adjusted life-years in the U.S.

The analysis was restricted to studies registered between October 2007 and September 2010; the window reflected enactment of legislation that required sponsors or their designees to register clinical trials in ClinicalTrials.gov and provide key data elements. The researchers used those reported data elements to determine characteristics of the trials. Other outcome measures included changes in characteristics over time, differences by specialty and factors associated with randomization, blinding and data monitoring committees.

Many of the trials were small, single-center and funded by sources other than industry or the National Institutes of Health (NIH). Ninety-six percent had anticipated enrollment of 1,000 participants or fewer, and 62 percent aimed for 100 or fewer. The median number of participants was 58 in completed trials and 70 in trials not yet completed. Two-thirds of the trials were single-center studies. Almost half received funding from entities other than industry or the NIH. The researchers also noted a significant heterogeneity among trials in the use of randomization, blinding and data monitoring committees.

“If you look at the number of trials that are large enough to give us definitive evidence about medical decisions, it is a much smaller number than I would have thought,” Califf said. He added that small studies have value, such as adding insight on mechanism of disease or serving as phase one studies in drug development. But he said there remained a huge evidence gap that such studies cannot fill.

Funding for many studies came from academic medical centers to provide internal research support for medical students, he said.   

While the study was designed to determine fundamental characteristics and not highlight any one of the specialties, the analysis did touch on differences. For instance, they reported that randomization and blinding were less frequently reported in earlier-phase, oncology and device trials.

Cardiovascular-specific findings included:
  • Cardiovascular trials were less numerous than oncology but more numerous than mental health in trials listed as still enrolling participants, at 10 percent vs. 31.5 percent and 9.3 percent, respectively.
  • Cardiovascular made up the largest proportion of trials focused on prevention, at 10.4 percent vs. 8.1 percent for oncology and 5.9 percent for mental health.
  • Cardiovascular trials also topped the list for medical device trials, at 20.2 percent for cardiovascular vs. 7 percent for oncology and 3.8 percent for mental health.
  • Enrollment goals for cardiovascular trials generally were twice that of oncology trials, with a median of 100 vs. 54.
  • Both cardiovascular and mental health trials were more oriented toward later-phase trials than oncology. 
  • Cardiovascular trials displayed the smallest proportion of studies with at least one research site in North America (47.9 percent vs. 65.1 percent for oncology and 69.1 percent for mental health).
  • A smaller proportion of cardiovascular trials were single-centered with no randomization compared with oncology (26.2 percent vs. 64.7 percent) but not mental health (20.8 percent).
  • And a larger proportion of mental health trials were blinded (60 percent for mental health vs. 49 percent for cardiovascular and 12.4 percent for oncology).

The team plans to publish at least 20 specialty-specific studies in the future, Califf said, including analyses on pediatric cardiovascular trials, adult cardiovascular trials and peripheral artery disease. Those papers will drill down on the details and shed light on what works well and what does not in the clinical trial enterprise.

“Our goal is to improve how clinical trials are done,” he said. To facilitate in that effort, CTTI has made an analysis-ready copy of is database available for others to use for analyses that may inform practice and policy.

The authors concluded: “We anticipate that the ‘sunshine’ on the national clinical trials portfolio brought about by ClinicalTrials.gov, coupled with the greater ease of obtaining an analysis data set from the database for AACT [Aggregate Analysis of ClinicalTrials.gov] will engender much-needed debate about clinical trial methodologies and funding allocation.”