Factor analysis is integrated with the Combination of a Uniform and a Binomial distribution (CUB) model to analyze multivariate ordinal data. By augmenting the CUB model with latent random factors, the proposed Factor Augmented CUB (FACUB) model generalizes the conventional multivariate CUB approach to capture complex correlations among items. This framework functions as a probabilistic principal component analysis tailored for multivariate ordinal data, enabling natural dimensionality reduction. For efficient inference, a maximum variational likelihood method is developed via a fast variational expectation-maximization algorithm. The consistency and asymptotic normality of the resulting estimator are established using profile M-estimation theory, and extensions for specific response styles are discussed. The effectiveness and practical utility of the model are demonstrated through comprehensive simulations and two complementary case studies: a low-dimensional application providing an intuitive illustration of the latent space, and a moderate-dimension application incorporating covariates to showcase the recovery of complex dependence structures.