Transformer language models can generate strikingly natural text by modeling language as a sequence of tokens. Yet, by relying primarily on surface-level co-occurrence statistics, they fail to form globally consistent latent representations of entities and events, lack of which contributes to brittleness in relational direction (e.g., reversal curse), contextualization errors, and data inefficiency. On the other hand, cognitive science shows that human comprehension involves converting the input linguistic stream into compact, event-like representations that persist in memory while verbatim form is short-lived. Motivated by this view, we introduce Thought Gestalt (TG) model, a recurrent Transformer that models language at two levels of abstraction - tokens and sentence-level "thought" states. TG generates the tokens of one sentence at a time while cross-attending to a memory of prior sentence representations. In TG, token and sentence representations are generated using the same set of model parameters and trained with a single objective, the next-token cross-entropy: by retaining the computation graph of sentence representations written to memory, gradients from future token losses flow backward through cross-attention to optimize the parameters generating earlier sentence vectors. In scaling experiments, TG consistently improves efficiency over matched GPT-2 runs, among other baselines, with scaling fits indicating GPT-2 requires ~5-8% more data and ~33-42% more parameters to match TG's loss. TG also reduces errors on relational direction generalization on a father-son reversal curse probe.

The 2024 Nobel Prize in Physics recognizes breakthroughs contributing to the emergence of a new understanding of the computations that underlie human intelligence, with profound implications for artificially intelligent systems. John Hopfield and Geoffrey Hinton played seminal roles in these breakthroughs. I begin by characterizing the scientific context in which the new understanding of intelligence began to arise. I then consider Hopfield's and Hinton's ideas and their centrality in the sciences of natural and artificial intelligence, while mentioning others who made major contributions. I conclude with a brief consideration of the current state of the fields these ideas have influenced and of the importance of governmental and scientific organizations in the past and future of these sciences.

The 25th anniversary of the Rumelhart Prize in Cognitive Science and a special issue of Topics in Cognitive Science celebrating the achievements of two recent Rumelhart Prize recipients provides an opportunity to reflect on the prize, the scientists that it honors, and the scientific values it seeks to promote. I offer my perspective on these topics as a long-time member of the Cognitive Science Society, a collaborator and friend of David Rumelhart, and as the first chair of the Rumelhart Prize selection committee. I see the prize as celebrating several aspects of what I believe many cognitive scientists aspire to achieve. We seek to make contributions to our understanding of our unique human ability to make sense of the world and of each other. We seek to employ a wide range of tools and methods, as well as insights from a wide range of perspectives. We seek to engage with our colleagues and our students, to create community, and even to have fun while we pursue our scientific goals. The careers of Dave Rumelhart and of the two Rumelhart Prize Winners celebrated in this special issue all richly exemplify these traits.

Mathematics relies on formal systems of rules that can be treated in isolation or grounded in a conceptual system that provides meaning for the relationships the rules express. Here, we show how the conceptual system provided by the unit circle, a visuospatial structure that provides a meaning for formal expressions in the domain of trigonometry, supports a generalizable understanding of trigonometric relationships, allowing for transfer beyond relationships explicitly taught. We examined the utility of the unit circle in our first study, in which we presented trigonometric identity problems to undergraduates (N = 50) who had prior coursework in pre-calculus trigonometry. Students reported using the unit circle to solve these problems more often than other approaches, and those who reported using the circle solved more problems correctly. Using other students from the same population, we then manipulated the systems they used by presenting a refresher lesson, using either formal rules or rules grounded in relationships on the unit circle (N = 35 in each group). Students in both conditions improved on taught problems, but only students in the grounded condition showed improvement on held-out transfer problems. Using findings from a third study further exploring the grounded condition (N = 64 participants), we found evidence that the circle supported transfer in two ways: by providing a procedure that could be used to solve both taught and transfer problems without rules and by allowing students to appreciate rules as capturing relationships between meaningful quantities, facilitating their application and extension. This project served as the starting place for the development of a curriculum that supports reliance on the unit circle and led to robust learning and retention of trigonometric relationships for most students with sufficient relevant prior knowledge, as described in Part II of this article.

Many models used in artificial intelligence and cognitive science rely on multi-element patterns stored in "slots" - dedicated storage locations - in a digital computer. As biological brains likely lack slots, we consider how they might achieve similar functional outcomes without them by building on the neurally-inspired modern Hopfield network (MHN; Krotov Hopfield, 2021), which stores patterns in the connection weights of an individual neuron. We propose extensions of this approach to increase its biological plausibility as a model of memory and to capture an important advantage of slot-based computation in contemporary language models. For memory, neuroscience research suggests that the weights of overlapping sparse ensembles of neurons, rather than a dedicated individual neuron, are used to store a memory. We introduce the K-winner MHN, extending the approach to ensembles, and find that within a continual learning regime, the ensemble-based MHN exhibits greater retention of older memories, as measured by the graded sensitivity measure d', than a standard (one-neuron) MHN. Next, we consider the powerful use of slot-based memory in contemporary language models. These models use slots to store long sequences of past inputs and their learned encodings, supporting later predictions and allowing error signals to be transported backward in time to adjust weights underlying the learned encodings of these past inputs. Inspired by these models' successes, we show how the MHN can be extended to capture both of these important functional outcomes. Collectively, our modeling approaches constitute steps towards understanding how biologically plausible mechanisms can support computations that have enabled AI systems to capture human-like abilities that no prior models have been able to achieve.

When do machine learning systems fail to generalize, and what mechanisms could improve their generalization? Here, we draw inspiration from cognitive science to argue that one weakness of machine learning systems is their failure to exhibit latent learning – learning information that is not relevant to the task at hand, but that might be useful in a future task. We show how this perspective links failures ranging from the reversal curse in language modeling to new findings on agent-based navigation. We then highlight how cognitive science points to episodic memory as a potential part of the solution to these issues. Correspondingly, we show that a system with an oracle retrieval mechanism can use learning experiences more flexibly to generalize better across many of these challenges. We also identify some of the essential components for effectively using retrieval, including the importance of within-example in-context learning for acquiring the ability to use information across retrieved examples. In summary, our results illustrate one possible contributor to the relative data inefficiency of current machine learning systems compared to natural intelligence, and help to understand how retrieval methods can complement parametric learning to improve generalization.

What types of numeric representations emerge in neural systems, and what would a satisfying answer to this question look like? In this work, we interpret Neural Network (NN) solutions to sequence based number tasks using a variety of methods to understand how well we can interpret them through the lens of interpretable Symbolic Algorithms (SAs) – precise programs describable by rules and typed, mutable variables. We use autoregressive GRUs, LSTMs, and Transformers trained on tasks where the correct tokens depend on numeric information only latent in the task structure. We show through multiple causal and theoretical methods that we can interpret raw NN activity through the lens of simplified SAs when we frame the activity in terms of neural subspaces rather than individual neurons. Using Distributed Alignment Search (DAS), we find that, depending on network architecture, dimensionality, and task specifications, alignments with SA's can be very high, or they can be only approximate, or fail altogether. We extend our analytic toolkit to address the failure cases by expanding the DAS framework to a broader class of alignment functions that more flexibly capture NN activity in terms of interpretable variables from SAs, and we provide theoretic and empirical explorations of Linear Alignment Functions (LAFs) in contrast to the preexisting Orthogonal Alignment Functions (OAFs). Through analyses of specific cases we confirm the usefulness of causal interventions on neural subspaces for NN interpretability, and we show that recurrent models can develop graded, symbol-like number variables in their neural activity. We further show that shallow Transformers learn very different solutions than recurrent networks, and we prove that such models must use anti-Markovian solutions – solutions that do not rely on cumulative, Markovian hidden states – in the absence of sufficient attention layers.

Large language models exhibit exciting capabilities, yet can show surprisingly narrow generalization from finetuning. E.g. they can fail to generalize to simple reversals of relations they are trained on, or fail to make simple logical deductions based on trained information. These failures to generalize factual information from fine-tuning can significantly hinder the reasoning capabilities of these models. On the other hand, language models' in-context learning (ICL) shows different inductive biases and deductive reasoning capabilities. Here, we explore these differences in generalization and deductive reasoning between in-context- and fine-tuning-based learning. To do so, we constructed several novel datasets to evaluate and improve models' abilities to make generalizations over factual information from novel data. These datasets are designed to create clean tests of generalization, by isolating the knowledge in the dataset from that in pretraining. We expose pretrained large models to controlled subsets of the information in these datasets – either through ICL or fine-tuning – and evaluate their performance on test sets that require various types of generalization. We find overall that in data-matched settings, ICL can generalize several types of inferences more flexibly than fine-tuning (though we also find some qualifications of prior findings, such as cases when fine-tuning can generalize to reversals embedded in a larger structure of knowledge). We build on these findings to propose a method to enable improved generalization from fine-tuning: adding in-context reasoning traces to finetuning data. We show that this method improves generalization across various splits of our datasets and other benchmarks. Our results have implications for understanding the generalization afforded by different modes of learning in language models, and practically improving their performance.

Abstract Deep learning—the study of learning in artificial neural networks containing many layers of neuron-like elements—captures and even exceeds human abilities in many domains. Because human brains are also deep neural networks that learn, deep networks provide a fertile ground for modeling human memory and learning, and they open up the possibility of joint engagement between the study of biological and artificial intelligence. This chapter introduces the basic constructs employed in deep learning and considers several of the widely used deep-learning paradigms and architectures. It then considers how the constructs of deep neural network models relate to traditional constructs in the psychological literature on learning and memory. Next, the chapter reviews recent developments in the field of reinforcement learning that have broad implications for human learning and memory. The chapter concludes by noting that human intelligence still exceeds current deep learning systems in many ways and describes future directions for research aimed toward bridging the gap.

reasoning is a key ability for an intelligent system. Large language models (LMs) achieve above-chance performance on abstract reasoning tasks but exhibit many imperfections. However, human abstract reasoning is also imperfect. Human reasoning is affected by our real-world knowledge and beliefs, and shows notable "content effects"; humans reason more reliably when the semantic content of a problem supports the correct logical inferences. These content-entangled reasoning patterns are central to debates about the fundamental nature of human intelligence. Here, we investigate whether language models-whose prior expectations capture some aspects of human knowledge-similarly mix content into their answers to logic problems. We explored this question across three logical reasoning tasks: natural language inference, judging the logical validity of syllogisms, and the Wason selection task. We evaluate state of the art LMs, as well as humans, and find that the LMs reflect many of the same qualitative human patterns on these tasks-like humans, models answer more accurately when the semantic content of a task supports the logical inferences. These parallels are reflected in accuracy patterns, and in some lower-level features like the relationship between LM confidence over possible answers and human response times. However, in some cases the humans and models behave differently-particularly on the Wason task, where humans perform much worse than large models, and exhibit a distinct error pattern. Our findings have implications for understanding possible contributors to these human cognitive effects, as well as the factors that influence language model performance.

Neural networks have long been used to model human intelligence, capturing elements of behavior and cognition, and their neural basis. Recent advancements in deep learning have enabled neural network models to reach and even surpass human levels of intelligence in many respects, yet unlike humans, their ability to learn new tasks quickly remains a challenge. People can reason not only in familiar domains, but can also rapidly learn to reason through novel problems and situations, raising the question of how well modern neural network models capture human intelligence and in which ways they diverge. In this work, we explore this gap by investigating human adults' ability to learn an abstract reasoning task based on Sudoku from a brief instructional tutorial with explanatory feedback for incorrect responses using a narrow range of training examples. We find that participants who master the task do so within a small number of trials and generalize well to puzzles outside of the training range. We also find that most of those who master the task can describe a valid solution strategy, and such participants perform better on transfer puzzles than those whose strategy descriptions are vague or incomplete. Interestingly, fewer than half of our human participants were successful in acquiring a valid solution strategy, and this ability is associated with high school mathematics education. We consider the challenges these findings pose for building computational models that capture all aspects of our findings and point toward a possible role for learning to engage in explanation-based reasoning to support rapid learning and generalization.

Large language models have recently shown promising progress in mathematical reasoning when fine-tuned with human-generated sequences walking through a sequence of solution steps. However, the solution sequences are not formally structured and the resulting model-generated sequences may not reflect the kind of systematic reasoning we might expect an expert human to produce. In this paper, we study how to build stronger reasoning capability in language models using the idea of relational abstractions. We introduce new types of sequences that more explicitly provide an abstract characterization of the transitions through intermediate solution steps to the goal state. We find that models that are supplied with such sequences as prompts can solve tasks with a significantly higher accuracy, and models that are trained to produce such sequences solve problems better than those that are trained with previously used human-generated sequences and other baselines. Our work thus takes several steps toward elucidating and improving how language models perform on tasks requiring multi-step mathematical reasoning.

Retrieval induced forgetting (RIF) occurs when the retrieval of one item negatively impacts the recall probability of related items stored in memory (Anderson et al., 1994). Recently, Ritvo et al. (2023) demonstrated RIF emerges in a neural network model equipped with non-monotonic plasticity. Their finding supports the non-monotonic plasticity hypothesis (NMPH; Ritvo et al., 2019): the theory that connection changes in the brain follow a “U” shaped function of post-synaptic stimulation. Here, we apply a unique implementation of non-monotonic plasticity to a neural network model of an idealized hippocampus (HPC) and evaluate it with an adaptation of a classic RIF task. The model evidences the behavioral and representational characteristics of RIF, replicating Ritvo et al. (2023). As a monotonic baseline model failed these tests, we provide evidence of non-monotonic plasticity’s sufficiency for RIF. In addition to demonstrating the NMPH is robust to multiple implementations and evaluative paradigms, we conduct additional analysis to provide a mechanistic explanation for how non-monotonic plasticity brings about RIF. Lastly, we evaluate the model with an expansion of RIF: reverse RIF. The model fails this final test, raising questions for future research on the necessary parameters of non-monotonic plasticity and whether it must pair with complementary processes in the brain.

Background: Impaired mucociliary clearance is a central feature of CF to which many factors contribute.Pseudomonas aeruginosa commonly infects the CF airway, and many strains harbor filamentous bacteriophage (Pf ) genomes within their bacterial chromosome.Our laboratories have shown that Pfs organize polymers of biofilms into higher-order liquid crystalline (LC) structures [1].Pf is present in the sputum of 40% to 60% of people with CF and P. aeruginosa infection and is associated with poor clinical outcomes [2], but the mechanisms by which Pf contributes to the pathogenesis of CF lung infections are unclear.We hypothesize that Pf interacts with the ciliated epithelia and disrupts effective mucociliary clearance (MCC).Methods: Explanted lungs from a person with CF infected with P. aeruginosa-harboring Pf and a person with CF infected with P. aeruginosa without Pf were harvested, sectioned, and evaluated for birefringent content within the airways to look for evidence of Pf-induced LC.To look for Pf impact on MCC, we generated air-liquid interface (ALI) cultures of nasal epithelial cells from a person with CF (F508del/F508del) and a healthy control (HC).ALI cultures were then treated overnight with Pf and other phage controls.Mucus was harvested from the apical surface to evaluate for birefringence.MCC in the ALI cultures was then assessed, first by measuring ciliary beat frequency and then adding fluorescent micro beads to the apical surface to track particle transport using high-speed video microscopy.Finally, inserts were fixed and imaged using scanning electron microscopy to evaluate the interaction between Pf and the epithelial surface.To further test the effects of Pf on an intact airway, MCC was assessed ex vivo in newborn piglet tracheas using video monitoring of the transport of mucus in the surface marked with ink particles.Results: We found that explanted CF lungs infected with P. aeruginosa harboring Pf have birefringent LC material embedded in mucus plugs within the airways.This was replicated in our ALI cell cultures treated with Pf.Although we found no effect on ciliary beat frequency among conditions tested ( p > 0.1), particle transport was significantly slower in HC and CF ALI cultures exposed to Pf ( p < 0.0001).Transport in untreated CF cells at ALI was rescued by elexecaftor-tezacaftor-ivacaftor (ETI) treatment to values comparable with those of HC cells ( p > 0.2).This rescue of CF cells by ETI was abrogated in the presence of Pf ( p < 0.0001).MCC in ex vivo newborn piglet tracheas likewise demonstrated significantly lower transport when Pf was added to the mucosal surface ( p = 0.04).Imaging of ALI filters using scanning electron microscopy demonstrated bundles of material likely representing mucus with Pf that appeared to entangle the ciliary brush.This was not present under control conditions.Conclusions: Pf induced formation of LC in the CF airway and impaired MCC in ALI cultures and in an ex vivo model.These data suggest that Pf contributes to the pathogenesis of P. aeruginosa-associated CF lung disease by impairing MCC.

Background: Osmoregulation, which maintains the osmolarity of fluid surrounding cells, is a key feature of ionocytes across species and has been studied extensively in fish adaptation to environmental changes in salinity.In the mammalian kidney, osmoregulation is coordinated by principal cells and intercalated cells of the collecting ducts, which control water movement; acid-base regulation; and Na + , Cl -, K + , and Ca 2+ homeostasis.Given the overlap in expressed channels between pulmonary ionocytes and kidney principal cells and intercalated cells, we hypothesized that pulmonary ionocytes may also participate in osmoregulation by airway epithelia.Methods: To mimic fish gill ionocytes in freshwater and seawater environments, we created hypertonic and hypotonic air-liquid interface (ALI) culture media.FOXI1-Cre ERT2 lineage-traced proximal airway stem cells were differentiated under hypertonic or hypotonic conditions in ALI culture.At full differentiation (21 days), we quantified ionocyte numbers and ionocyte subtype gene expression patterns.Results: We exposed actively differentiating ferret basal cells to slightly hypertonic media (+77 mOsm/L NaCl) and observed a marked increase (7.32-fold; p < 0.0015) in the number of lineage-traced pulmonary ionocytes at full differentiation (21 days).Under hyperosmotic stress, greater numbers of ionocytes was correlated with high mRNA expression of key ionocyte markers FOXI1 and ASCL3.We have previously shown that three ionocyte subtypes exist (Type-A, Type-B, Type-C) in ferret airway ALI culture.We found that hyperosmotic stress increased expression of Type-A (BSND) and Type-C (CXCL17) ionocyte marker genes but decreased expression of the Type-B ionocyte marker ID3.The observed downregulation in CFTR expression under hyperosmotic stress is consistent with the expansion of Type-C ionocytes, which express significantly ( p = 2 × 10 -11 ) less CFTR than other ionocyte subtypes.Hyperosmotic stress imposed on fully differentiated ALI cultures did not alter the frequency of lineage-traced pulmonary ionocytes, suggesting that a hyperosmotic environment affects progenitor cell specification of Type-A and Type-C ionocytes.Exposure of differentiating ferret basal cells to hypotonic media (-77 mOsm/L) also led to expansion of ATP6V1G3 + ionocytes when fully differentiated at the ALI.Ongoing gene expression profiling is defining the dominant ionocyte subtype under hypotonic conditions.Conclusions: These findings indicate that osmolarity affects basal cell specification of pulmonary ionocytes and suggests that there are specialized ionocyte functions that adapt to environmental changes in airway osmolarity.Hyperosmotic fluid at the basolateral surface of the airway would be expected to be extracted water from cells and the airway surface liquid (ASL), leading to cell shrinkage and ASL dehydration.Our data suggest that apical membrane Cl -permeability is constrained by CFTR-expressing ionocytes and that the observed hyperosmotic downregulation of CFTR may be a compensatory mechanism to limit ASL dehydration.This finding may be analogous to the inactivation of ionocyte CFTR expression in fish gills during seawater to freshwater transition.

Accounts of human language processing have long appealed to implicit ``situation models'' that enrich comprehension with relevant but unstated world knowledge. Here, we apply causal intervention techniques to recent transformer models to analyze performance on the Winograd Schema Challenge (WSC), where a single context cue shifts interpretation of an ambiguous pronoun. We identify a relatively small circuit of attention heads that are responsible for propagating information from the context word that guides which of the candidate noun phrases the pronoun ultimately attends to. We then compare how this circuit behaves in a closely matched ``syntactic'' control where the situation model is not strictly necessary. These analyses suggest distinct pathways through which implicit situation models are constructed to guide pronoun resolution.

Transformer networks have seen great success in natural language processing and machine vision, where task objectives such as next word prediction and image classification benefit from nuanced context sensitivity across high-dimensional inputs. However, there is an ongoing debate about how and when transformers can acquire highly structured behavior and achieve systematic generalization. Here, we explore how well a causal transformer can perform a set of algorithmic tasks, including copying, sorting, and hierarchical compositions of these operations. We demonstrate strong generalization to sequences longer than those used in training by replacing the standard positional encoding typically used in transformers with labels arbitrarily paired with items in the sequence. We search for the layer and head configuration sufficient to solve these tasks, then probe for signs of systematic processing in latent representations and attention patterns. We show that two-layer transformers learn reliable solutions to multi-level problems, develop signs of task decomposition, and encode input items in a way that encourages the exploitation of shared computation across related tasks. These results provide key insights into how attention layers support structured computation both within a task and across multiple tasks.

How can artificial neural networks capture the advanced cognitive abilities of pioneering scientists? I suggest they must learn to exploit human-invented tools of thought and human-like ways of using them, and must engage in explicit goal-directed problem solving as exemplified in the activities of scientists and mathematicians and taught in advanced educational settings.