Study Design
COVID-19 Vaccine Effectiveness: An Evaluation of the Test-Negative Design Using Randomized Placebo-Controlled Clinical Trials Leah I. B. Andrews* Leah Andrews M. Elizabeth Halloran Kathleen M. Neuzil Lars van der Laan Yunda Huang Jessica Andriesen Mayur Patel Heather Angier Leigh H. Fisher Holly Janes Kelsey Sumner Brendan Flannery Dean Follmann Peter B. Gilbert
During the COVID-19 pandemic, the United States government established the COVID-19 Prevention Network (CoVPN) and coordinated multiple phase 3 randomized, placebo-controlled vaccine efficacy trials. After regulatory authorization of the COVID-19 vaccines, the test-negative design (TND) was widely employed to estimate vaccine effectiveness in diverse populations and settings under real-world conditions. The TND is an observational study design that enrolls individuals with COVID-like symptoms who seek SARS-CoV-2 testing and compares vaccination status between cases who test positive vs. noncases who test negative. The TND reduces confounding by healthcare-seeking behavior, though other sources of confounding may remain. We assessed the conditions in which the TND reliably evaluates vaccine performance against virologically confirmed, symptomatic COVID-19 using CoVPN vaccine trial data from Moderna COVE, AstraZeneca/Oxford AZD1222, Janssen/Johnson & Johnson ENSEMBLE, Novavax PREVENT-19, and Sanofi/GSK VAT00008. We investigated the core TND assumption that vaccinated and unvaccinated healthcare-seeking populations have the same incidence of COVID-like symptoms caused by other pathogens besides SARS-CoV-2. We also constructed TND samples from the trial data according to four TND approaches and compared vaccine effectiveness estimators based on standard logistic regression vs. targeted maximum likelihood estimation under a semiparametric logistic regression model. In this randomized and blinded setting that is free of confounding, the core TND assumption was supported and TND vaccine effectiveness estimates were nearly identical to randomized, placebo-controlled vaccine efficacy estimates (three trials analyzed to date). Our study supports use of the resource-efficient TND for inferences on real-world COVID-19 vaccine effectiveness and presents a statistical method with robust confounding control, which is important when analyzing observational TND data.
