Troubleshooting Bayesian cognitive models: A tutorial with matstanlib

Using Bayesian methods to apply computational models of cognitive processes, or Bayesian cognitive modeling, is an important new trend in psychological research. The rise of Bayesian cognitive modeling has been accelerated by the introduction of software such as Stan and PyMC3 that efficiently automates the Markov chain Monte Carlo (MCMC) sampling used for Bayesian model fitting. Unfortunately, Bayesian cognitive models can struggle to pass the computational checks required of all Bayesian models. If any failures are left undetected, inferences about cognition based on model output may be biased or incorrect. As such, Bayesian cognitive models almost always require troubleshooting before being used for inference. Here, we present a deep treatment of the diagnostic checks and procedures that are critical for effective troubleshooting, but are often left underspecified by tutorial papers. After a conceptual introduction to Bayesian cognitive modeling and MCMC sampling, we outline the diagnostic metrics, procedures, and plots necessary to identify problems in model output with an emphasis on how these requirements have recently been improved. Throughout, we explain how the most commonly encountered problems may be remedied with specific, practical solutions. We also introduce matstanlib, our MATLAB modeling support library, and demonstrate how it facilitates troubleshooting of an example hierarchical Bayesian model of reinforcement learning implemented in Stan. With this comprehensive guide to techniques for detecting, identifying, and overcoming problems in fitting Bayesian cognitive models, psychologists across subfields can more confidently build and use Bayesian cognitive models.All code is freely available from github.com/baribault/matstanlib.