This article introduces and discusses the Quantitative Decision-Making (QDM) statistical framework and its application in vaccine development. The QDM framework encourages researchers to set clear success criteria and enables decision-makers to evaluate the risk associated with endorsing a particular study and investing in the further development of the asset. This is achieved through the computation of the Probability of Success (PoS) of a study, which measures the likelihood to meet certain pre-specified criteria. The article discusses the challenges encountered in vaccine trials and outlines strategies for managing multiple endpoints and surrogate endpoints. The article provides some case studies to illustrate some concepts and concludes with a discussion on the general approach. Supplementary materials for this article are available online.

Abstract Objectives The use of correlates of protection (CoPs) in vaccination trials offers significant advantages as useful clinical endpoint substitutes. Vaccines with very high vaccine efficacy (VE) are documented in the literature (95% or above). Callegaro, A., and F. Tibaldi. 2019. “Assessing Correlates of Protection in Vaccine Trials: Statistical Solutions in the Context of High Vaccine Efficacy.” BMC Medical Research Methodology 19: 47 showed that the rare infections observed in the vaccinated groups of these trials poses challenges when applying conventionally-used statistical methods for CoP assessment such as the Prentice criteria and meta-analysis. The objective of this work is to investigate the impact of this problem on another statistical method for the assessment of CoPs called Principal stratification. Methods We perform simulation experiments to investigate the effect of high vaccine efficacy on the performance of the Principal Stratification approach. Results Similarly to the Prentice framework, simulation results show that the power of the Principal Stratification approach decreases when the VE grows. Conclusions It can be challenging to validate principal surrogates (and statistical surrogates) for vaccines with very high vaccine efficacy.

The assurance of a future clinical trial is a key quantitative tool for decision-making in drug development. It is derived from prior knowledge (Bayesian approach) about the clinical endpoint of interest, typically from previous clinical trials. In this paper, we examine assurance in the specific context of vaccine development, where early development (Phase 2) is often based on immunological endpoints (e.g., antibody levels), while the confirmatory trial (Phase 3) is based on the clinical endpoint (very large sample sizes and long follow-up). Our proposal is to use the Phase 2 vaccine efficacy predicted by the immunological endpoint (using a model estimated from epidemiological studies) as prior information for the calculation of the assurance.

The use of correlates of protection (CoPs) in vaccination trials offers significant advantages as useful clinical endpoint substitutes. Vaccines with very high vaccine efficacy (VE) are documented in the literature (VE ≥95%). The rare events (number of infections) observed in the vaccinated groups of these trials posed challenges when applying conventionally-used statistical methods for CoP assessment. In this paper, we describe the nature of these challenges, and propose easy-to-implement and uniquely-tailored statistical solutions for the assessment of CoPs in the specific context of high VE. The Prentice criteria and meta-analytic frameworks are standard statistical methods for assessing vaccine CoPs, but can be problematic in high VE cases due to the rare events data available. As a result, lack of fit and the problem of infinite estimates may arise, in the former and latter methods respectively. The use of flexible models within the Prentice framework, and penalized-likelihood methods to solve the issue of infinite estimates can improve the performance of both methods in high VE settings. We have 1) devised flexible non-linear models to counteract the Prentice framework lack of fit, providing sufficient statistical power to the method, and 2) proposed the use of penalised likelihood approaches to make the meta-analytic framework applicable on randomized subgroups, such as regions. The performance of the proposed methods for high VE cases was evaluated by running simulations. As vaccines with high efficacy are documented in the literature, there is a need to identify effective statistical solutions to assess CoPs. Our proposed adaptations are straight-forward and improve the performance of conventional statistical methods for high VE data, leading to more reliable CoP assessments in the context of high VE settings.

Many people in business and medicine regard statistics as at best a nuisance, or as a challenging (frequently incomprehensible) and therefore best ignored subject (whenever possible), or at worst as an hinderance to science. Even Einstein liked to say that “God doesn’t gamble.” .

In recent years, many vaccines have been developed for the prevention of a variety of diseases. Many of these vaccines, like the one for herpes zoster, are supposed to act in a multilevel way. Ideally, they completely prevent expression of the virus, but failing that they help to reduce the severity of the disease. A simple approach to analyze these data is the so-called burden-of-illness test. The method assigns a score, say W, equal to 0 for the uninfected and a post-infection outcome X > 0 for the infected individuals. One of the limitations of this test is the potential low power when the vaccine efficacy is close to 0. To overcome this limitation, we propose a Fisher adjusted test where we combine a statistic for infection with a statistic for post-infection outcome adjusted for selection bias. The advantages and disadvantages of different methods proposed in the literature are discussed. We compared the methods via simulations in herpes zoster, HIV, and malaria vaccine trial settings. In addition, we applied these methods to published data on HIV vaccine. The paper ends with some recommendations and conclusions.

Background.  To investigate the relationship between hemagglutinin-inhibition (HI) antibody levels to the risk of influenza disease, we conducted a correlate of protection analysis using pooled data from previously published randomized trials. Methods.  Data on the occurrence of laboratory-confirmed influenza and HI levels pre- and postvaccination were analyzed from 4 datasets: 3 datasets included subjects aged <65 years who received inactivated trivalent influenza vaccine (TIV) or placebo, and 1 dataset included subjects aged ≥65 years who received AS03-adjuvanted TIV (AS03-TIV) or TIV. A logistic model was used to evaluate the relationship between the postvaccination titer of A/H3N2 HI antibodies and occurrence of A/H3N2 disease. We then built a receiver-operating characteristic curve to identify a potential cutoff titer between protection and no protection. Results.  The baseline odds ratio of A/H3N2 disease was higher for subjects aged ≥65 years than <65 years and higher in seasons of strong epidemic intensity than moderate or low intensity. Including age and epidemic intensity as covariates, a 4-fold increase in titer was associated with a 2-fold decrease in the risk of A/H3N2 disease. Conclusions.  The modeling exercise confirmed a relationship between A/H3N2 disease and HI responses, but it did not allow an evaluation of the predictive power of the HI response.

A prophylactic human papillomavirus (HPV) vaccine targeting oncogenic HPV types in addition to HPV-16 and -18 may broaden protection against cervical cancer. Two Phase I/II, randomized, controlled studies were conducted to compare the immunogenicity and safety of investigational tetravalent HPV L1 virus-like particle (VLP) vaccines, containing VLPs from two additional oncogenic genotypes, with the licensed HPV-16/18 AS04-adjuvanted vaccine (control) in healthy 18-25 year-old women.In one trial (NCT00231413), subjects received control or one of 6 tetravalent HPV-16/18/31/45 AS04 vaccine formulations at months (M) 0,1,6. In a second trial (NCT00478621), subjects received control or one of 5 tetravalent HPV-16/18/33/58 vaccines formulated with different adjuvant systems (AS04, AS01 or AS02), administered on different schedules (M0,1,6 or M0,3 or M0,6).One month after the third injection (Month 7), there was a consistent trend for lower anti-HPV-16 and -18 geometric mean antibody titers (GMTs) for tetravalent AS04-adjuvanted vaccines compared with control. GMTs were statistically significantly lower for an HPV-16/18/31/45 AS04 vaccine containing 20/20/10/10 μg VLPs for both anti-HPV-16 and anti-HPV-18 antibodies, and for an HPV-16/18/33/58 AS04 vaccine containing 20/20/20/20 μg VLPs for anti-HPV-16 antibodies. There was also a trend for lower HPV-16 and -18-specific memory B-cell responses for tetravalent AS04 vaccines versus control. No such trends were observed for CD4(+) T-cell responses. Immune interference could not always be overcome by increasing the dose of HPV-16/18 L1 VLPs or by using a different adjuvant system. All formulations had acceptable reactogenicity and safety profiles. Reactogenicity in the 7-day post-vaccination period tended to increase with the introduction of additional VLPs, especially for formulations containing AS01.HPV-16 and -18 antibody responses were lower when additional HPV L1 VLPs were added to the HPV-16/18 AS04-adjuvanted vaccine. Immune interference is a complex phenomenon that cannot always be overcome by changing the antigen dose or adjuvant system.

When the true relationship between a covariate and an outcome is nonlinear, one should use a nonlinear mean structure that can take this pattern into account. In this article, the fractional polynomial modeling framework, which assumes a prespecified set of powers, is extended to a nonlinear fractional polynomial framework (NLFP). Inferences are drawn in a Bayesian fashion. The proposed modeling paradigm is applied to predict the long-term persistence of vaccine-induced anti-HPV antibodies. In addition, the subject-specific posterior probability to be above a threshold value at a given time is calculated. The model is compared with a power-law model using the deviance information criterion (DIC). The newly proposed model is found to fit better than the power-law model. A sensitivity analysis was conducted, from which a relative independence of the results from the prior distribution of the power was observed. Supplementary materials for this article are available online.

In infectious diseases, it is important to predict the long-term persistence of vaccine-induced antibodies and to estimate the time points where the individual titers are below the threshold value for protection. This article focuses on HPV-16/18, and uses a so-called fractional-polynomial model to this effect, derived in a data-driven fashion. Initially, model selection was done from among the second- and first-order fractional polynomials on the one hand and from the linear mixed model on the other. According to a functional selection procedure, the first-order fractional polynomial was selected. Apart from the fractional polynomial model, we also fitted a power-law model, which is a special case of the fractional polynomial model. Both models were compared using Akaike's information criterion. Over the observation period, the fractional polynomials fitted the data better than the power-law model; this, of course, does not imply that it fits best over the long run, and hence, caution ought to be used when prediction is of interest. Therefore, we point out that the persistence of the anti-HPV responses induced by these vaccines can only be ascertained empirically by long-term follow-up analysis.

Objectives. Strong and sustained HPV-16 and -18 antibody responses have been observed in previously unexposed women aged 15-25 years vaccinated with the AS04-adjuvanted HPV-16/18 L1 virus-like particle vaccine. While awaiting the extended results of ongoing trials, our objective was to predict the long-term persistence of anti-HPV-16/18 antibodies in vaccinees by applying three statistical models using immunogenicity data from vaccinated women with serum samples collected LIP to 6.4 years after first vaccination. Two different data lock-points (up to 5.5 years and up to 6.4 years) were used to assess the robustness of the models.Methods. Three statistical models were applied to estimate the long-term persistence of anti-HPV-16/18 antibodies in 393 women vaccinated with the AS04-adjuvanted HPV-16/18 vaccine. Individual antibody levels for each study participant at each timepoint up to 6.4 years were input to previously published power-law and modified power-law models. The power-law model estimates antibody decay over time. The modified power-law model takes into account both antibody persistence over time and immune memory. A third model, the piece-wise model, fits the data based on three different non-overlapping intervals (between Months 7 and 12, Months 12 and 21, and over 21 months), corresponding to the observed decay of vaccine-induced antibodies.Results. HPV-16 and -18 antibodies peaked at Month 7 and gradually plateaued at Months 18-24 and remained stable through 6.4 years. Mean antibody levels at the last timepoint were several fold higher than those associated with natural infection. All three models predict that HPV-16 and -18 mean antibody levels will remain well above those associated with natural infection for at least 20 years, when using data from 5.5 as well as 6.4 years' follow-up. Predictions are similar for the modified power-law model and improve with longer follow-up for both the power-law and the piece-wise models.Conclusions. Vaccination with the AS04-adjuvanted HPV-16/18 vaccine is predicted to provide long-term persistence for both HPV-16 and -18 antibodies, independent of the statistical model applied. Model predictions are based on conservative mathematical assumptions. Since the input of longer term data of up to 6.4 years showed an improved profile compared with that for data LIP to 5.5 years, the predictions of antibody persistence based on population means are conservative when predicting that antibody levels will remain well above levels induced by natural infection for 20 years. (C) 2009 Elsevier Inc. All rights reserved.

Meta-analytical approaches have been extensively used to analyze medical data. In most cases, the data come from different studies or independent trials with similar characteristics. However, these methods can be applied in a broader sense. In this paper, we show how existing meta-analytic techniques can also be used as well when dealing with parameters estimated from individual hierarchical data. Specifically, we propose to apply statistical methods that account for the variances (and possibly covariances) of such measures. The estimated parameters together with their estimated variances can be incorporated into a general linear mixed model framework. We illustrate the methodology by using data from a first-in-man study and a simulated data set. The analysis was implemented with the SAS procedure MIXED and example code is offered.

BiometricsVolume 63, Issue 3 p. 960-962 The authors replied as follows: A. Alonso, A. Alonso Center for Statistics, Hasself University, BelgiumSearch for more papers by this authorG. Molenberghs, G. Molenberghs Center for Statistics, Hasself University, BelgiumSearch for more papers by this authorT. Burzykowski, T. Burzykowski Center for Statistics, Hasself University, BelgiumSearch for more papers by this authorD. Renard, D. Renard Novartis, Basel, SwitzerlandSearch for more papers by this authorH. Geys, H. Geys Center for Statistics, Hasself University, BelgiumSearch for more papers by this authorZ. Shkedy, Z. Shkedy Center for Statistics, Hasself University, BelgiumSearch for more papers by this authorF. Tibaldi, F. Tibaldi Glaxo Smith Kline, Brussels, BelgiumSearch for more papers by this authorJ. Cortinas Abrahantes, J. Cortinas Abrahantes Center for Statistics, Hasself University, BelgiumSearch for more papers by this authorM. Buyse, M. Buyse International Drug Development Institute (IDDI), Brussels, BelgiumSearch for more papers by this author A. Alonso, A. Alonso Center for Statistics, Hasself University, BelgiumSearch for more papers by this authorG. Molenberghs, G. Molenberghs Center for Statistics, Hasself University, BelgiumSearch for more papers by this authorT. Burzykowski, T. Burzykowski Center for Statistics, Hasself University, BelgiumSearch for more papers by this authorD. Renard, D. Renard Novartis, Basel, SwitzerlandSearch for more papers by this authorH. Geys, H. Geys Center for Statistics, Hasself University, BelgiumSearch for more papers by this authorZ. Shkedy, Z. Shkedy Center for Statistics, Hasself University, BelgiumSearch for more papers by this authorF. Tibaldi, F. Tibaldi Glaxo Smith Kline, Brussels, BelgiumSearch for more papers by this authorJ. Cortinas Abrahantes, J. Cortinas Abrahantes Center for Statistics, Hasself University, BelgiumSearch for more papers by this authorM. Buyse, M. Buyse International Drug Development Institute (IDDI), Brussels, BelgiumSearch for more papers by this author First published: 31 August 2007 https://doi.org/10.1111/j.1541-0420.2007.00852_2.xRead the full textAboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onEmailFacebookTwitterLinkedInRedditWechat References Burzykowski, T., Molenberghs, G., and Buyse, M. (2005). The Evaluation of Surrogate Endpoints. New York : Springer. 10.1007/b138566 Google Scholar Buyse, M. andMolenberghs, G. (1998). The validation of surrogate endpoints in randomized experiments. Biometrics 54, 1014–1029. 10.2307/2533853 CASPubMedWeb of Science®Google Scholar Buyse, M., Molenberghs, G., Burzykowski, T., Renard, D., andGeys, H. (2000). The validation of surrogate endpoints in meta-analyses of randomized experiments. Biostatistics 1, 49–67. 10.1093/biostatistics/1.1.49 CASPubMedGoogle Scholar Frangakis C., andRubin, D. (2002). Principal stratification in causal inference. Biometrics 58, 21–29. 10.1111/j.0006-341X.2002.00021.x PubMedWeb of Science®Google Scholar Freedman, L., Graubard, B., andSchatzkin, A. (1992). Statistical validation of intermediate endpoints for chronic diseases. Statistics in Medicine 11, 167–178. 10.1002/sim.4780110204 CASPubMedWeb of Science®Google Scholar Prentice, R. L. (1989). Surrogate endpoints in clinical trials: Definitions and operational criteria. Statistics in Medicine 8, 431–440. 10.1002/sim.4780080407 CASPubMedWeb of Science®Google Scholar Volume63, Issue3September 2007Pages 960-962 ReferencesRelatedInformation

Summary We put a perspective on the strengths and limitations of statistical methods for the evaluation of surrogate endpoints. Whereas using several trials overcomes some of the limitations of a single‐trial framework (Prentice, 1989, Statistics in Medicine8, 431–440), arguably the evaluation of surrogate endpoints can never be done using only statistical evidence but such evidence should be seen as but one component in a decision‐making process that involves, among others, a number of clinical and biological considerations. We briefly present a hierarchical framework that incorporates ideas from Prentice's work and is uniformly applicable to different types of surrogate and true clinical outcomes.

This work was motivated by the need to find surrogate endpoints for survival of patients in oncology studies. The goal of this article is to determine associations between five time-to-event outcomes coming from three clinical trials for non-small cell lung cancer. To this end, we propose to use the multivariate Dale model for time-to-event data introduced by Tibaldi et al. (Stat. Med. 2003). We fit the model to these data, using a pseudo-likelihood approach to estimate the model parameters. We evaluate and compare the performance of different dimensional models and we relate the Dale model association parameter, i.e. the odds ratio, to well-known quantities such as Kendall's tau and Spearman's rho. Finally, the results are discussed with a perspective on surrogate marker validation. Some suggestions are made regarding further studies in this field.

In this paper, we propose a multivariate Plackett–Dale model for survival outcomes. A pseudo‐likelihood method for the estimation of the parameters is proposed and these ideas are applied to two case studies. The modelling approach is similar in spirit but different from Parner's approach. The first study is in AIDS, where the overall survival time and different opportunistic infections in HIV‐infected patients are studied. The second study is on adoption data where the association of the survival times within families is modelled, illustrating the use of the proposed methodology for the context of population genetics. Copyright © 2004 John Wiley & Sons, Ltd.