Hierarchical paired comparison modeling, a cultural consensus theory approach

We introduce a set of models designed to analyze datasets involving responses from multiple subjects on pairwise comparisons from a fixed discrete set of alternatives. These models are part of a greater body of work known as Cultural Consensus Theory (CCT). Like other CCT models, these simultaneously infer each individual’s tendency toward aligning with the group consensus, level of agreement on each item, and also a latent consensus value of each alternative. Two primary models are discussed, referred to as the Strong and Weak Consensus Paired-Comparison Models (SCPCM and WCPCM respectively). The SCPCM works under the assumption that all individuals are answering in accordance to the latent consensus values but with varying degrees of accuracy, while the WCPCM relaxes this assumption and assumes minor deviations from latent consensus values in people’s average valuation of alternatives. The WCPCM also includes inferences on participants’ individual tendencies toward self-consistency (related to their tendencies toward committing violations of transitivity) as well as inferences on the tendency of each item to be evaluated consistently by individuals. The Case III Thurstonian model is used as the backbone for both CPCMs, and inference is conducted under a hierarchical Bayesian framework. Model checks along with applications to both simulated and real data are overviewed.