Sparse Envelope Model: Efficient Estimation and Response Variable Selection in Multivariate Linear Regression

The envelope model allows efficient estimation in multivariate linear regression. In this paper, we propose the sparse envelope model, which is motivated by applications where some response variables are invariant with respect to changes of the predictors and have zero regression coefficients. The envelope estimator is consistent but not sparse, and in many situations it is important to identify the response variables for which the regression coefficients are zero. The sparse envelope model performs variable selection on the responses and preserves the efficiency gains offered by the envelope model. Response variable selection arises naturally in many applications, but has not been studied as thoroughly as predictor variable selection. In this paper, we discuss response variable selection in both the standard multivariate linear regression and the envelope contexts. In response variable selection, even if a response has zero coefficients, it should still be retained to improve the estimation efficiency of the nonzero coefficients. This is different from the practice in predictor variable selection. We establish consistency and the oracle property and obtain the asymptotic distribution of the sparse envelope estimator.