Last week I engaged in a discussion on X about the validity of this recent publication on a randomized controlled trial (RCT) on neonatal mortality in newborns.

As a background: BCG vaccination is routinely administered in low-and-middle income countries to protect children from tuberculosis. Yet, these vaccines may offer additional non-specific protection against other pathogens – a phenomenon known as “trained immunity”. Several different BCG vaccine types exist, such as the Danish and the Russian strains, which may differ in their non-specific effects.

The World Health Organization (WHO) recommends BCG vaccination at birth in countries where tuberculosis is endemic. Yet, vaccination is often delayed among low-birth weights newborns (<2,000 grams), who have a considerably high risk of dying within their first 28 days of life.

Study Design

This multicenter open-label RCT, conducted in India, enrolled 5,420 newborns (with an average age of 0.9 days). The infants were randomized to receive either:

1. Early vaccination: BCG Danish strain administered intradermally together with oral polio vaccine (OPV) within 48 hours after birth; or,
2. Usual Care (Control): the same vaccines administered upon the infant’s hospital discharge.

The primary endpoint was survival at Day 28 of life.

Findings

Both groups were comparable for patient characteristics: Administration of vaccines occurred after a median of 0.03 and 11.8 days in both the early vaccination and control groups and all subjects enrolled were included in an intention to treat analysis. Day 28 mortality was 8.8% and 10.1% for the early vaccination and control group, yielding an adjusted hazard ratio of 0.83 (95% CI 0.69 to 0.98; P=0.03).  Infection-related neonatal mortality per person year was 0.40 and 0.73 in the intervention and control group, yielding an adjusted hazard ratio of 0.53, 95% CI 0.40 to 0.70).

No deaths from tuberculosis occurred, and no serious adverse effects were associated with vaccination. Data collection was complete, with nine patients alive when lost to follow-up before Day 28.

How to Assess Internal and External Validity and Precision of this Study?

The trial’s design and reported execution provide strong evidence for the reported protective effect of early vaccination – an almost 20% reduction in mortality (or 1.3% absolute reduction; number needed to treat = 21), achieved simply by administering the vaccines 11 days earlier.

While blinding a BCG vaccination is unfeasible due to the characteristic skin mark that follows, but with a primary outcome – mortality – that can be determined objectively, the absence of blinding does not reduce validity, as long as data collection is complete.

Whether the effect size is sufficiently large and precise enough for policy action is a matter for public health authorities. However, given the intervention involves only administering vaccines earlier, with no extra costs or health risks, the findings are highly compelling.

External Validity

External validity is provided by a meta-analysis of the same intervention (also with BCG Danish) in five RCTs (Figure 1). The pooled effect size is consistently favorable in all five studies, with low clinical heterogeneity (the same outcome, with the same intervention in similar patient populations) and low statistical heterogeneity (I²=19,2%).

Figure 1

Context and Implications

Interestingly, in a previous study with a similar design, conducted by the same investigators on a similar patient population but using the BCG Russia strain, showed no effect on patient survival.

During the COVID-19 pandemic, I was involved in four RCTs evaluating the effects of BCG Danish as a protective measure for healthcare workers and the elderly against COVID-19 and its sequalae. Those studies yielded high-quality evidence on the absence of such effects in those populations.   To me, these studies on neonates have provided high-quality evidence that early administration of BCG Danish improves Day 28 survival rates. Given these findings, I am doubtful about whether future trials comparing BCG Danish with other BCG types, control or placebo in this patient population would be considered ethical.