Solving the small crappy trial problem

Published by


Randomised controlled trials (RCTs) yield the strongest evidence to inform medical practice. Each year, more than 40,000 new RCTs are registered, and some have estimated that they cost more than 80 billion US dollars. Yet RCTs are remarkably inefficient.

The issues plaguing RCTs’ efficiency are well-known: it takes years from trial initiation to completion (and timelines only increase), they are costly (and costs increase), many patient populations are underrepresented, and the questions addressed in RCTs do not necessarily answer the questions that patients and clinicians have. On top of all that, many RCTs fall in the category of “the small crappy (or shitty) trial problem”: underpowered, not providing solid evidence and wasting resources that could have been used in a better way. The symptoms of a diseased/broken system.

Angus et al: “In both inpatient and outpatient settings, clinical teams make approximately 100 decisions per day, yet as little as 3% are supported by any scientific evidence.”

This clinical trial ecosystem is broken and needs treatment! At least that was concluded by an international group of trialists that met at the first JAMA summit meeting in October last year. An excellent summary of that meeting (which I attended) by Derek Angus has now been published.

The cause of the broken trial enterprise is – at least in part – that in most resource-rich countries, clinical trials and health care delivery function as separate entities, with siloed goals, infrastructure, and incentives. The COVID-19 pandemic made that very clear: the clinical research response was uncoordinated, with many underpowered studies not contributing to guidance of patient treatment. The adaptive platform trials RECOVERY and REMAP-CAP are the frequently mentioned exceptions, as these studies rapidly enrolled many patients and integrated randomisation into daily practice. Yet their success (massive and fast enrolment) was also based on the top-down decision of the Chief Medical Officers in the UK to prioritise both studies in the National Health Service.

The proposed treatment for the broken trial ecosystem contains four elements: (1) greater clarity to ensure appropriate regulation and oversight of implementation science, quality improvement, embedded clinical trials, and learning health systems; (2) greater adoption of study designs that improve statistical and logistical efficiency and lower the burden on participants and clinicians, allowing trials to be smarter, safer, and faster; (3) better integration of RCTs with electronic health records (which requires greater adoption of standards and processes designed to ensure health data are adequately reliable and accurate and capable of being transferred responsibly and efficiently across platforms and organisations; (4) alignment of stakeholders in the clinical trials and health care delivery enterprises through financial and nonfinancial incentives.

“Solutions exist for each of these problems, and there are examples of success for each, but there is a failure to implement at adequate scale.”

Ecraid pursues to implement many of the recommendations listed in this manuscript: to establish a warm-base clinical trial network in which multiple trials – integrated in daily patient care – for relevant infections constantly run.