Counterfactual groups to assess treatment efficacy in HIV prevention trials in high risk populations in Uganda

Abstract Background: Assessment of efficacy in HIV prevention trials remains a challenge in the era of widespread use of active controls. We investigated use of countefactual groups to assess treatment efficacy.Methods: We used data from placebo arms of two previous HIV prevention efficacy trials (Pro2000 vaginal microbicide trial, 2005-2009 and dapivirine vaginal ring trial, 2013-2016) and four observational cohorts (two in each of the periods; (a) during the conduct of a simulated HIV vaccine efficacy trial (SiVET), 2012-2017 and (b) prior to SiVET (2005-2011), and compared HIV prevention efficacy trial targeted outcomes with SiVETs. SiVET participants were administered a licensed Hepatitis B vaccine at 0,1 and 6 months mimicking an HIV vaccine efficacy trial schedule. Participants were tested for HIV quarterly for one year. The probability of SiVET assignment conditioned on measured participants baseline characteristics were estimated using propensity scores (PS) and matched between SiVET and placebo arm of trials. Similar calculations were repeated for observational cohorts in the pre and during SiVET periods. We compared HIV incidence rate ratio (IRR) between SiVET and the trials or observational data before and after PS matching. Results: This analysis involved data from 3,387 participants; observational cohorts before SiVET 1495 (44.2%), placebo arms of previous trials 367 (10.8%), observational cohorts during SiVET conduct 953 (28.1%) and SiVETs 572 (16.9%). Before propensity score matching (PSM), there were significant imbalances in participants baseline characteristics between SiVET and all the other studies and HIV incidence was lower in SiVET. After PSM, the participants characteristics were comparable. The HIV incidence in SiVET was similar to that in the previous trial, IRR=1.01 95%CI:0.16-4.70), p=0.968, and observational data during SiVET, IRR=0.74, 95%CI 0.34-1.54), p=0.195 but much lower compared to the observational data pre SiVET, IRR=0.48, 95%CI:0.20-1.04) p=0.023.Conclusion: PSM can be used to create countefactual groups from other data sources. The best counterfactual group for assessing treatment effect is provided by data collected in the placebo arm of previous trials followed by that from observational data collected concurrently to the current trial (SiVET). Even with PSM, observational data collected prior to the current trial may over estimate treatment effect.