In this paper, we study online estimation and inference of regression coefficients in the presence of hidden confounders by leveraging over-parameterized models. Unlike existing offline approaches that rely on factor and sparse models, our closed-form estimator simultaneously removes hidden-confounder bias and is directly applicable to streaming data. Using tools from random matrix theory, we analyse phase transition phenomena in the variance of the coefficient estimator that arise as the sample size transitions from being smaller than to larger than the number of predictors. Notably, we show that adding more covariates only slightly affects the estimator's variance, mitigating concerns about variance inflation in over-parameterized settings. We validate the effectiveness of our method for both individual coefficient inference and multiple testing through simulations and applications to two real datasets.
