Adaptive Judgment in the Cognitive Reflection Test: A Computational Analysis

The Cognitive Reflection Test (CRT) is widely used in reasoning and decision making research, but it lacks a formal cognitive foundation. As a result, debates persist over why CRT scores correlate with mathematical ability, why some individuals solve CRT problems easily while others struggle, and which mental processes drive observed patterns in data. We use an ecological perspective combined with computational cognitive modeling to address these questions, focusing in particular on the bat-and-ball problem. First, we specify the learning environment by assembling a large dataset of grade school verbal math problems. Second, we specify a formal learning mechanism that selects the arithmetic operation most likely to be applied to a novel problem based on the linguistic association of the problem with learned exemplars. Our model generates the intuitive errors elicited by the bat-and-ball problem (as well as other CRT items) and explains why these errors can be seen as byproducts of adaptive cognition. It also makes new predictions about the effect of environmental structure and problem wording on strategy selection and downstream performance, and we validate these predictions in two new preregistered experiments. Overall, our work provides theoretical clarity and quantitative rigor to our understanding of intuitive judgment, and shows how such judgments can be understood as rational adaptations to the learning environment.