Optimizing Markov Random Field Inference via Event-driven Gibbs Sampling

Markov Random Field (MRF) is a powerful graphical model for representing a wide range of applications in statistical machine learning. MRF encodes the conditional dependence among random variables (RVs). One approach to solving problems represented by MRF is using probabilistic algorithms such as Gibbs sampling. These methods go through all RVs in MRF and update them iteratively, until converged to the final result. In this work, we build on three observations to skip updating RVs that cannot change their value during the current iteration, hence avoiding unnecessary work: i) after the warmup period, most RVs tend to not change values very often, ii) an RV can only change its value if either it has a non-concentrated probability distribution function (PDF), or at least one of the RVs on which it is conditionally dependent has changed its value, and iii) approximation techniques used for hardware specialization make it increasingly likely that RVs have concentrated PDFs. Therefore, we introduce event-driven Gibbs sampling which only updates RVs when necessary. Our analysis shows significant speedup can be gained for two image analysis applications.