Variational Bayesian Gaussian mixture model for off‐grid DOA estimation

Abstract Wireless signals are commonly subject to diverse and complex noise interference. The typical assumption of Gaussian white noise often oversimplifies the noise, resulting in reduced accuracy in estimating the direction of arrival (DOA), especially in complex scenarios. To tackle this issue, this paper introduces a new Bayesian model for off‐grid DOA estimation. This model utilizes Gaussian mixture model (GMM)‐based Dirichlet processes (DP) to characterize noise, allowing adaptive adjustments in the number of Gaussian mixture models. Leveraging the factor graph representation of the Bayesian model, a low‐complexity mixed messaging passing algorithm, employing generalized approximate message passing (GAMP) and mean field (MF), is proposed. Simulation results validate the efficacy of the proposed algorithm.