The clinical trial bottlenecks game
Can you speed up a vaccine trial by changing its design?
Once a drug or vaccine is developed, it takes roughly a decade for it to be tested in all stages of clinical trials, reviewed, approved, and finally available for people to take – and that’s assuming it’s successful at all!
Clinical trials are therefore often described as a bottleneck to medical progress. But others believe that rapid progress in AI will allow better drugs to be developed such that all, or most, diseases will be curable within ten years. Who’s right?
Here, I want to illustrate how bottlenecks work. You’ll learn that the same bottleneck can be solved in different ways, but some levers are more impactful than others (and some are more expensive to change than others).
We'll focus on a phase three trial, the last stage of clinical testing before a drug is approved. Your goal is to cut the modeled timeline of a trial from about 2 years to one year. Will you succeed? Let’s find out.
The setup
We’ll use the example of human papillomavirus (HPV) vaccines. HPV spreads through sexual contact and can cause several cancers, including cervical, penile, anal, oral, head and neck cancers. Thankfully, HPV vaccines are highly effective against the types they target. But clinical trials for the vaccines still take years to run.
In this simplified interactive model, adapted from real HPV vaccine trials, you’ll get a sense of how each lever affects the timeline, or whether they’re already near their limit.
Reference trial
The phase three Gardasil FUTURE II trial
Our starting point is a simplified example adapted from FUTURE II.
- Sample size
- About 12,000 participants
- Recruitment
- About 8% recruited each month
- True vaccine efficacy
- 98%
- Outcome
- Serious cervical cell changes
- Modeled trial duration
- About 2 years
Roughly 12,000 women participated in the FUTURE II trial for HPV vaccines. By randomizing participants to vaccine or placebo, it looked at whether vaccines reduced the risk of serious changes in cervical cells linked to HPV-16 or HPV-18, two of the most harmful types of the virus. It was successful, and found that the vaccine cut the risk by 98%!
Further reading
If you liked this, follow the Clinical Trials Abundance blog for more! You can also learn more about policy ideas from the blogposts, articles and reports below.
Ideas for clinical trial reform
- The case for clinical trial abundance — Ruxandra Teslo and Willy Chertman, IFP.
- Clinical trials were not always this complicated — Adam Kroetsch.
- Every disease is a policy failure – Saloni Dattani, Works in Progress.
- Why biotech companies go to Australia to test new drugs — Ruxandra Teslo and Adam Kroetsch, IFP.
- To fix trials, we need to pay attention to the boring stuff — Adam Kroetsch.
- AI won’t automatically accelerate clinical trials — Ruxandra Teslo.
- Proxy Praxis: How surrogate endpoints can speed drug development — Ruxandra Teslo, IFP.
- Why were Covid vaccine trials so fast? — Saloni Dattani.
- Manufacturing requirements are killing cell and gene therapy — Ruxandra Teslo and Amol Punjabi.
- The case for sharing clinical trial data — Saloni Dattani.
Methodology
What the model does
In this model, participants are recruited into the trial gradually and are assigned equally to vaccine and control groups. Disease events accumulate over time while some usable follow-up data is lost. The timeline ends when enough events have accumulated that the trial's statistical rule is met.
Statistical rule
The model tests whether the vaccine's true efficacy (selected by the user) is greater than 50%, using 90% power and a two-sided 5% significance level. At the 98% efficacy, as in the reference value, this requires about 13 expected endpoint events.
Fixed assumptions
For simplicity, this model assumes that the rate of disease is constant over time, that recruitment occurs linearly over time (with a constant share of participants being recruited per month), and that participants who are lost to follow up are not replaced by new participants. It also assumes that the follow-up begins from month seven for each participant.
Coins
The budget game uses 100 coins for illustrative purposes, rather than an estimate of the true financial cost, which there is little data for. Rather, every lever displays the same cost curve, with diminishing returns, which are scaled to each lever. For the disease incidence, the curve begins at the selected outcome’s starting rate to an illustrative 1.5 events per 1,000 participants per month. Spending on disease incidence represents the research and trial infrastructure needed to identify and recruit from higher-incidence places or populations. The cost curves make the trade-offs playable but don't imply that the interventions have known or comparable prices.
Reference values
Data for most reference values comes from the ClinicalTrials.gov record for NCT00092534, the FUTURE II trial testing Gardasil, the HPV vaccine. Its primary publication reports that the recruitment window spanned from June 2002 to May 2003, that participants had an average follow-up of three years, a disease incidence of 0.3 per 100 people per year in the control group, and that the vaccine was found to have 98% efficacy against serious cervical cell changes linked to HPV-16 or HPV-18. The site uses 98% as the true vaccine efficacy. Unlike the original trial, and for simplicity, it applies a threshold of testing whether a vaccine's efficacy is higher than 50%, with 90% power, and two-sided 5% significance level to every outcome.
Although the reference trial used serious cervical cell changes (CIN2 or worse) linked to HPV-16 or HPV-18 as its primary outcome, the model allows other endpoints to be selected. These have illustrative incidence rates and efficacy values.
Download all reference values and sources (CSV)
Important limitations
This model is not a forecast for a proposed clinical trial. It does not model changes in disease incidence over time, geographic variation, adherence, regulatory review, manufacturing, costs, safety outcomes, or ethical constraints on trial design. The results depend on the selected assumptions and should be interpreted as comparisons within the model.
Data
Successful combinations are stored in a shared dataset so they can be displayed to later users after those users have reached the one-year target themselves. The selected nickname, coins spent, model parameters, and calculated duration are stored for the chart and leaderboards.