Randomized experiments are widely used to estimate the causal effects of treatment or intervention across various scientific fields. Nowadays, numerous experiments are conducted in network settings where the outcome of one unit depends not only on its own treatment but also on the treatments of other units within the network. This phenomenon, known as network interference, poses significant challenges for experimental design. Existing randomization methods often neglect interference or rely on strict assumptions, such as homogeneous peers or linearity in peer interference. We introduce two innovative and easy-to-implement randomization designs for estimating direct treatment effects in the presence of heterogeneous or nonlinear network interference: network interference balancing rerandomization (NetRR) and network maximized matching randomization (NetMM). We explore the theoretical properties of two widely used estimators under these proposed randomization designs. Numerical results in the simulated experiment and the real-world social network experiment confirm the desirable performance of the proposed methods. Additionally, we discuss the tradeoff between the risks associated with model assumptions and the feasibility of the randomization design.

Exact recovery in stochastic block models (SBMs) is well understood in undirected settings, but remains considerably less developed for directed and sparse networks, particularly when the number of communities diverges. Spectral methods for directed SBMs often lack stability in asymmetric, low-degree regimes, and existing non-spectral approaches focus primarily on undirected or dense settings. We propose a fully non-spectral, two-stage procedure for community detection in sparse directed SBMs with potentially growing numbers of communities. The method first estimates the directed probability matrix using a neighborhood-smoothing scheme tailored to the asymmetric setting, and then applies K-means clustering to the estimated rows, thereby avoiding the limitations of eigen- or singular value decompositions in sparse, asymmetric networks. Our main theoretical contribution is a uniform row-wise concentration bound for the smoothed estimator, obtained through new arguments that control asymmetric neighborhoods and separate in- and out-degree effects. These results imply the exact recovery of all community labels with probability tending to one, under mild sparsity and separation conditions that allow both γ_n → 0 and K_n →∞. Simulation studies, including highly directed, sparse, and non-symmetric block structures, demonstrate that the proposed procedure performs reliably in regimes where directed spectral and score-based methods deteriorate. To the best of our knowledge, this provides the first exact recovery guarantee for this class of non-spectral, neighborhood-smoothing methods in the sparse, directed setting.

Model misspecification is ubiquitous in data analysis because the data-generating process is often complex and mathematically intractable. Therefore, assessing estimation uncertainty and conducting statistical inference under a possibly misspecified working model is unavoidable. In such a case, classical methods such as bootstrap and asymptotic theory-based inference frequently fail since they rely heavily on the model assumptions. In this article, we provide a new bootstrap procedure, termed local residual bootstrap, to assess estimation uncertainty under model misspecification for generalized linear models. By resampling the residuals from the neighboring observations, we can approximate the sampling distribution of the statistic of interest accurately. Instead of relying on the score equations, the proposed method directly recreates the response variables so that we can easily conduct standard error estimation, confidence interval construction, hypothesis testing, and model evaluation and selection. It performs similarly to classical bootstrap when the model is correctly specified and provides a more accurate assessment of uncertainty under model misspecification, offering data analysts an easy way to guard against the impact of misspecified models. We establish desirable theoretical properties, such as the bootstrap validity, for the proposed method using the surrogate residuals. Numerical results and real data analysis further demonstrate the superiority of the proposed method.

In network analysis, one frequently needs to conduct inference for network parameters based on a single observed network. Since the sampling distribution of the statistic is often unknown, one has to rely on the bootstrap. However, because of the complex dependence structure among vertices, existing bootstrap methods often yield unsatisfactory performance, especially for small or moderate sample sizes. Here we propose a new network bootstrap procedure, termed the local bootstrap, for estimating the standard errors of network statistics. The method involves resampling the observed vertices along with their neighbour sets, and then reconstructing the edges between the resampled vertices by drawing from the set of edges connecting their neighbour sets. We justify the proposed method theoretically with desirable asymptotic properties for statistics such as motif density, and demonstrate its excellent numerical performance for small and moderate sample sizes. Our approach encompasses several existing methods, such as the empirical graphon bootstrap, as special cases. We investigate the advantages of the proposed method over existing methods in terms of edge randomness, vertex heterogeneity and neighbour set size, which can help to shed light on the complex issue of network bootstrapping.

In network analysis, we often need to conduct statistical inference on the population network model using a single observed network via network statistics such as the largest eigenvalues of the adjacency matrix. However, since the sampling distributions of these network statistics are often complex, we rely on the bootstrap for such inference. In this article, we introduce a new bootstrap method for weighted networks to conduct statistical inference, focusing on the eigenvalues of the adjacency matrix as our statistic of interest. We establish desirable properties of the proposed method, such as bootstrap consistency. We demonstrate its applications in standard error estimation, confidence interval construction, and hypothesis testing, and show its superior performance in simulation and real data analysis.

Customer segmentation has wide applications in business activities, such as personalized marketing and targeted product development. To realize customer segmentation, clustering methods are commonly used. However, modern customer segmentation encounters challenges characterized by highdimensionality and mixed-type variables (i.e., the mixture of continuous variables and categorical variables). It brings great challenges to customer segmentation, because most existing clustering methods are only designed for data with one single type of variables. Furthermore, the existence of noise variables highlights the necessity of simultaneous variable selection and data clustering. Motivated by these issues, we develop a Davies-Bouldin index based sparse clustering (DBI-SC) method for customer segmentation with high-dimensional mixed-type data. In this method we define dissimilarity measures for continuous variables and categorical variables separately. Then an adjusted DBI criterion is designed to measure the contribution of each variable to clustering. For variable selection we apply the sparse clustering framework and introduce different penalty parameters for the mixed-type variables. The screening consistency property of the DBI-SC method is also investigated. Extensive simulation studies demonstrate the satisfactory performance of the DBI-SC method in both clustering and variable selection. Finally, a designated driving service dataset is analyzed for customer segmentation using the proposed method.

Traditional genetic programming (GP) often struggles in stock alpha factor discovery due to its vast search space, overwhelming computational burden, and sporadic effective alphas. We find that GP performs better when focusing on promising regions rather than random searching. This paper proposes a new GP framework with carefully chosen initialization and structural constraints to enhance search performance and improve the interpretability of the alpha factors. This approach is motivated by and mimics the alpha searching practice and aims to boost the efficiency of such a process. Analysis of 2020-2024 Chinese stock market data shows that our method yields superior out-of-sample prediction results and higher portfolio returns than the benchmark.

Recently, causal inference under interference has gained increasing attention in the literature. In this paper, we focus on randomized designs for estimating the total treatment effect (TTE), defined as the average difference in potential outcomes between fully treated and fully controlled groups. We propose a simple design called weighted random isolation (WRI) along with a restricted difference-in-means estimator (RDIM) for TTE estimation. Additionally, we derive a novel mean squared error surrogate for the RDIM estimator, supported by a network-adaptive weight selection algorithm. This can help us determine a fair weight for the WRI design, thereby effectively reducing the bias. Our method accommodates directed networks, extending previous frameworks. Extensive simulations demonstrate that the proposed method outperforms nine established methods across a wide range of scenarios.

In comparative studies, researchers often seek an optimal covariate balance. However, chance imbalance still exists in randomized experiments, and becomes more serious as the number of covariates increases. To address this issue, we introduce a new randomization procedure, called adaptive randomization via the Mahalanobis distance (ARM). The proposed method allocates units sequentially and adaptively, using information on the current level of imbalance and the incoming unit's covariate. Theoretical results and numerical comparison show that with a large number of covariates or a large number of units, the proposed method shows substantial advantages over traditional methods in terms of the covariate balance, estimation accuracy, hypothesis testing power, and computational time. The proposed method attains the optimal covariate balance, in the sense that the estimated treatment effect attains its minimum variance asymptotically, and can be applied in both causal inference and clinical trials. Lastly, numerical studies and a real-data analysis provide further evidence of the advantages of the proposed method. An R package CARM implementing the proposed method is freely accessible in CRAN.

Statistical inference of the high-dimensional regression coefficients is challenging because the uncertainty introduced by the model selection procedure is hard to account for. A critical question remains unsettled; that is, is it possible and how to embed the inference of the model into the simultaneous inference of the coefficients? To this end, we propose a notion of simultaneous confidence intervals called the sparsified simultaneous confidence intervals. Our intervals are sparse in the sense that some of the intervals' upper and lower bounds are shrunken to zero (i.e., $[0,0]$), indicating the unimportance of the corresponding covariates. These covariates should be excluded from the final model. The rest of the intervals, either containing zero (e.g., $[-1,1]$ or $[0,1]$) or not containing zero (e.g., $[2,3]$), indicate the plausible and significant covariates, respectively. The proposed method can be coupled with various selection procedures, making it ideal for comparing their uncertainty. For the proposed method, we establish desirable asymptotic properties, develop intuitive graphical tools for visualization, and justify its superior performance through simulation and real data analysis.

In comparative studies, covariate balance and sequential allocation schemes have attracted growing academic interest. Although many theoretically justified adaptive randomization methods achieve the covariate balance, they often allocate patients in pairs or groups. To better meet the practical requirements where the clinicians cannot wait for other participants to assign the current patient for some economic or ethical reasons, we propose a method that randomizes patients individually and sequentially. The proposed method conceptually separates the covariate imbalance, measured by the newly proposed modified Mahalanobis distance, and the marginal imbalance, that is the sample size difference between the 2 groups, and it minimizes them with an explicit priority order. Compared with the existing sequential randomization methods, the proposed method achieves the best possible covariate balance while maintaining the marginal balance directly, offering us more control of the randomization process. We demonstrate the superior performance of the proposed method through a wide range of simulation studies and real data analysis, and also establish theoretical guarantees for the proposed method in terms of both the convergence of the imbalance measure and the subsequent treatment effect estimation.

Randomization is a distinguishing feature of clinical trials for unbiased assessment of treatment efficacy. With a growing demand for more flexible and efficient randomization schemes and motivated by the idea of adaptive design, in this article we propose the network and covariate adjusted response‐adaptive (NCARA) design that can concurrently manage three challenges: (1) maximizing benefits of a trial by assigning more patients to the superior treatment group randomly; (2) balancing social network ties across treatment arms to eliminate potential network interference; and (3) ensuring balance of important covariates, such as age, gender, and other potential confounders. We conduct simulation with different network structures and a variety of parameter settings. It is observed that the NCARA design outperforms four alternative randomization designs in solving the above‐mentioned problems and has comparable power and type I error for detecting true difference between treatment groups. In addition, we conduct real data analysis to implement the new design in two clinical trials. Compared to equal randomization (the original design utilized in the trials), the NCARA design slightly increases power, largely increases the percentage of patients assigned to the better‐performing group, and significantly improves network and covariate balances. It is also noted that the advantages of the NCARA design are augmented when the sample size is small and the level of network interference is high. In summary, the proposed NCARA design assists researchers in conducting clinical trials with high‐quality and high‐efficiency.

伴随网络时代发展,个体间关系逐渐被纳入研究视角,基于社交网络的经济试验广泛开展.传统的随机化方法面临两大挑战:其一,参与试验的受试者间不再满足个体处理稳定性假设,若忽略网络干扰作用可能会导致估计偏差;其二,出于经济因素考虑,研究者希望在试验过程中分配更多受试者到优效干预组.针对上述问题,本文提出社交网络调整自适应随机化方法,该方法将响应变量自适应随机化方法与网络干扰作用平衡相结合,保证干预效果估计并将更多节点分配到优效干预组.模拟分析与实证研究均表明本文研究方法具有综合优势,是网络时代中进行经济试验的必要工具.

In this study, we advocate a two-stage framework to deal with the issues encountered in surveys with long questionnaires. In Stage I, we propose a split questionnaire design (SQD) developed by minimizing a quadratic cost function while achieving reliability constraints on estimates of means, which effectively reduces the survey cost, alleviates the burden on the respondents, and potentially improves data quality. In Stage II, we develop a block weighted least squares (BWLS) estimator of linear regression coefficients that can be used with data obtained from the SQD obtained in Stage I. Numerical studies comparing existing methods strongly favor the proposed estimator in terms of prediction and estimation accuracy. Using the European Social Survey (ESS) data, we demonstrate that the proposed SQD can substantially reduce the survey cost and the number of questions answered by each respondent, and the proposed estimator is much more interpretable and efficient than present alternatives for the SQD data.

Multi-arm trials are common in medical and health research for comparing the efficacy of competing drugs and interventions, among other applications. While ensuring covariate balance is a critical issue for comparative studies to be successful, classical multi-arm trials often fail to balance covariates among multi-treatments. An adaptive randomization via Mahalanobis distance for multi-arm trials is proposed to improve the covariate balance and thus the quality of the subsequent treatment effect estimation. The investigation scope includes the implementation of the proposed method and also its theoretical properties. Both theoretical and numerical results demonstrate the proposed method can attain desirable covariate balance, and thus improving the subsequent estimation efficiency. Compared with other competing methods, the computational cost of the proposed method is also favorable. An illustrative real case analysis of the efficacy of different doses of Canagliflozin, a treatment for patients with type 2 diabetes, also proves that the proposed method has broad applicability.

Although model selection is ubiquitous in scientific discovery, the stability and uncertainty of the selected model is often hard to evaluate. How to characterize the random behavior of the model selection procedure is the key to understand and quantify the model selection uncertainty. To this goal, initially several graphical tools are proposed. These include the G-plots and H-plots, to visualize the distribution of the selected model. Then the concept of model selection deviation to quantify the model selection uncertainty is introduced. Similar to the standard error of an estimator, model selection deviation measures the stability of the selected model given by a model selection procedure. For such a measure, a bootstrap estimation procedure is discussed and its desirable performance is demonstrated through simulation studies and real data analysis.

In the era of data science, it is common to encounter data with different subsets of variables obtained for different cases. An example is the split questionnaire design (SQD), which is adopted to reduce respondent fatigue and improve response rates by assigning different subsets of the questionnaire to different sampled respondents. A general question then is how to estimate the regression function based on such block-wise observed data. Currently, this is often carried out with the aid of missing data methods, which may unfortunately suffer intensive computational cost, high variability, and possible large modeling biases in real applications. In this article, we develop a novel approach for estimating the regression function for SQD-type data. We first construct a list of candidate models using available data-blocks separately, and then combine the estimates properly to make an efficient use of all the information. We show the resulting averaged model is asymptotically optimal in the sense that the squared loss and risk are asymptotically equivalent to those of the best but infeasible averaged estimator. Both simulated examples and an application to the SQD dataset from the European Social Survey show the promise of the proposed method.