Testing rationality of decision-making and choice by evaluating the mathematical property of transitivity has a long tradition in biology, economics, psychology, and zoology. This paradigm is fraught with conceptual, mathematical, and statistical pitfalls. In this overview, we tackle five major obstacles. One challenge lies in spelling out what transitivity of latent preferences really says and what it actually implies about observable choice behavior. Most notably, this step is fraught with aggregation artifacts, in that aggregated behavior can be profoundly misleading about individual behavior. Another hurdle comes from hard mathematical problems associated with characterizing the properties of heterogeneous transitive populations. A third challenge is the prevalence of straw man hypotheses in this area of research. The fourth difficulty is associated with adopting appropriate statistical inference tools that correctly accommodate the idiosyncratic mathematical properties of order-constrained statistical hypotheses. The fifth hurdle arises with the role of scientific parsimony in rationality research. We walk readers through key concepts, mathematical models, and statistical techniques for testing rationality. Throughout, we provide examples using the methods and data of two prominent published papers on animal choice behavior as our case studies. We explain how these papers tackled the five hurdles to varying degrees of success.

Testing rationality of decision-making and choice by evaluating the mathematical property of transitivity has a long tradition in biology, economics, psychology, and zoology. This paradigm is fraught with conceptual, mathematical, and statistical pitfalls. In this overview, we tackle five major obstacles. One challenge lies in spelling out what transitivity of latent preferences really says and what it actually implies about observable choice behavior. Most notably, this step is fraught with aggregation artifacts, in that aggregated behavior can be profoundly misleading about individual behavior. Another hurdle comes from hard mathematical problems associated with characterizing the properties of heterogeneous transitive populations. A third challenge is the prevalence of straw man hypotheses in this area of research. The fourth difficulty is associated with adopting appropriate statistical inference tools that correctly accommodate the idiosyncratic mathematical properties of order-constrained statistical hypotheses. The fifth hurdle arises with the role of scientific parsimony in rationality research. We walk readers through key concepts, mathematical models, and statistical techniques for testing rationality. Throughout, we provide examples using the methods and data of two prominent published papers on animal choice behavior as our case studies. We explain how these papers tackled the five hurdles to varying degrees of success.

This stand-alone tutorial gives an introduction to the QTESIR 2.1 public domain software package for the specification and statistical analysis of certain order-constrained probabilistic choice models. Like its predecessors, QTEST 2.1 allows a user to specify a variety of probabilistic models of binary responses and to carry out state-of-the-art frequentist order-constrained hypothesis tests within a Graphical User Interface (GUI). QTEST 2.1 automatizes the mathematical characterization of so-called "random preference models", adds some parallel computing capabilities, and, most importantly, adds tools for Bayesian inference and model selection. In this tutorial, we provide an in-depth introduction to the Bayesian features: We review order-constrained Bayesian p-values, DIC and Bayes factors, building on the data, models, and prior QTEST based frequentist data analyses of an earlier (frequentist) tutorial by Regenwetter et al. (2014). (C) 2019 Elsevier Inc. All rights reserved.