Weighted Linear Programming Discriminant Analysis for High‐dimensional Binary Classification

Linear discriminant analysis (LDA) is widely used for various binary classification problems. In contrast to the LDA that estimates the precision matrix Ω and the mean difference vector δ in the classification rule separately, the linear programming discriminant (LPD) rule estimates the product Ωδ directly through a constrained ℓ1 minimization. The LPD rule has very good classification performance on many high‐dimensional binary classification problems. However, to estimate β* = Ωδ, the LPD rule uses equal weights for all the elements of β* in the constrained ℓ1 minimization. It may not deliver the optimal estimate of β*, and therefore the estimated discriminant direction can be suboptimal. In order to obtain better estimates of β* and the discriminant direction, we can heavily penalize βj in the constrained ℓ1 minimization if we suspect the jth feature is useless for the classification while moderately penalize βj if we suspect the jth feature is useful. In this paper, based on the LPD rule and some popular feature screening methods, we propose a new weighted linear programming discriminant (WLPD) rule for the high‐dimensional binary classification problem. The screening statistics used in the marginal two‐sample t‐test screening, Kolmogorov–Smirnov filter, and the maximum marginal likelihood screening will be used to construct appropriate weights for different elements of β* flexibly. Besides the linear programming algorithm, we develop a new alternating direction method of multipliers algorithm to solve the high‐dimensional constrained ℓ1 minimization problem efficiently. Our numerical studies show that our proposed WLPD rule can outperform LPD and serve as an effective binary classification tool.