Computational models are central to cognitive neuroscience, but their rigorous application to experimental datasets is often constrained to a narrow set of canonical models that afford tractable analytical computations. We introduce the HSSM (Hierarchical Sequential Sampling Model) ecosystem, a Python toolbox that democratizes access to a broad, extensible array of neurocognitive process models through hierarchical Bayesian inference. Naturally leveraging simulation-based inference via likelihood surrogates, HSSM enables fast parameter estimation for models lacking closed-form likelihoods. Built atop PyMC and Bambi, HSSM provides a user-friendly formula syntax for specifying hierarchical mixed-effects regressions on model parameters, incorporating trial-by-trial neural or physiological covariates. The ecosystem allows fast model simulation and training data generation, as well as the neural network training utilities to deploy surrogate likelihood networks via HuggingFace. Contributions are designed to benefit not only the single researcher working on a problem, but organically, the entire research community. Together, the tools in the HSSM ecosystem bridge the interests of computational theorists as well as experimentalists, accelerating the cycle from model development to rigorous empirical testing.

Background: Major depressive disorder (MDD), bipolar disorder (BP), and schizophrenia (SCZ) involve learning impairments with poorly understood mechanisms. Understanding both the similarities and differences in these mechanisms is important to guide the development of new, targeted interventions. Methods: A total of 255 participants diagnosed with MDD (n = 54), BP (n = 47), or SCZ (n = 67) or without any clinical diagnoses (control [CTRL]) (n = 87) performed an associative learning task. Computational modeling quantified the mechanistic interplay between working memory (WM) and reinforcement learning (RL). The latent RL and WM signatures in the electroencephalography (EEG) dynamics showed shared and distinct neurocognitive mechanisms underlying learning. Results: All clinical groups showed learning impairments at the behavioral level. Model-based EEG analyses linked these impairments to distinct patterns in the dynamic interplay between latent RL and WM mechanisms, contrasting with the typical patterns observed in the CTRL group. SCZ was characterized by reduced neural markers of WM, weakening the cooperative influence of WM onto RL (reduced WM recruitment), and reduced integration of negative feedback. Conversely, MDD was characterized by reduced reciprocal influence of RL onto WM, reducing the tendency to upregulate WM contribution with reward history (impaired WM management). Finally, BP was characterized by deficits in both WM and RL recruitment, along with higher WM decay. Conclusions: Behavioral learning impairments that seem similar across clinical groups can be linked to distinct neurocognitive mechanisms via integrative neurocomputational modeling. Our approach provides insights into the interplay of underlying learning mechanisms and how they manifest differently across psychopathologies.

Computational modeling has become a central aspect of research in the cognitive neurosciences. As the field matures, it is increasingly important to move beyond standard models to quantitatively assess models with richer dynamics that may better reflect underlying cognitive and neural processes. For example, sequential sampling models (SSMs) are a general class of models of decision-making intended to capture processes jointly giving rise to RT distributions and choice data in n-alternative choice paradigms. A number of model variations are of theoretical interest, but empirical data analysis has historically been tied to a small subset for which likelihood functions are analytically tractable. Advances in methods designed for likelihood-free inference have recently made it computationally feasible to consider a much larger spectrum of SSMs. In addition, recent work has motivated the combination of SSMs with reinforcement learning models, which had historically been considered in separate literatures. Here, we provide a significant addition to the widely used HDDM Python toolbox and include a tutorial for how users can easily fit and assess a (user-extensible) wide variety of SSMs and how they can be combined with reinforcement learning models. The extension comes batteries included, including model visualization tools, posterior predictive checks, and ability to link trial-wise neural signals with model parameters via hierarchical Bayesian regression.

Computational modeling has become a central aspect of research in the cognitive neurosciences. As the field matures, it is increasingly important to move beyond standard models to quantitatively assess models with richer dynamics that may better reflect underlying cognitive and neural processes. For example, sequential sampling models (SSMs) are a general class of models of decision making intended to capture processes jointly giving rise to reaction time distributions and choice data in n-alternative choice paradigms. A number of model variations are of theoretical interest, but empirical data analysis has historically been tied to a small subset for which likelihood functions are analytically tractable. Advances in methods designed for likelihood-free inference have recently made it computationally feasible to consider a much larger spectrum of sequential sampling models. In addition, recent work has motivated the combination of SSMs with reinforcement learning (RL) models, which had historically been considered in separate literatures. Here we provide a significant addition to the widely used HDDM Python toolbox and include a tutorial for how users can easily fit and assess a (user extensible) wide variety of SSMs, and how they can be combined with RL models. The extension comes batteries included, including model visualization tools, posterior predictive checks, and ability to link trial-wise neural signals with model parameters via hierarchical Bayesian regression.

Motor skill learning involves the acquisition of sequential motor movements with practice. Studies have shown that we learn to execute these sequences efficiently by chaining several elementary actions in sub-sequences called motor chunks. Several experimental paradigms, such as serial reaction task, discrete sequence production, and m × n task, have investigated motor chunking in externally specified sequencing where the environment or task paradigm provides the sequence of stimuli, i.e., the responses are stimulus driven. In this study, we examine motor chunking in a class of more realistic motor tasks that involve internally guided sequencing where the sequence of motor actions is self-generated or internally specified. We employ a grid-navigation task as an exemplar of internally guided sequencing to investigate practice-driven performance improvements due to motor chunking. The participants performed the grid-sailing task (GST) (Fermin et al., 2010), which required navigating (by executing sequential keypresses) a 10 × 10 grid from start to goal position while using a particular type of key mapping between the three cursor movement directions and the three keyboard buttons. We provide empirical evidence for motor chunking in grid-navigation tasks by showing the emergence of subject-specific, unique temporal patterns in response times. Our findings show spontaneous chunking without pre-specified or externally guided structures while replicating the earlier results with a less constrained, internally guided sequencing paradigm.

Several canonical experimental paradigms (e.g., serial reaction time task, discrete sequence production task, m × n task) have been proposed to study the typical behavioral phenomenon and the nature of learning in sequential keypress tasks. A characteristic feature of most paradigms is that they are representative of externally-specified sequencing—motor tasks where the environment or task paradigm extrinsically provides the sequence of stimuli, i.e., the responses are stimulus-driven. Previous studies utilizing such canonical paradigms have largely overlooked the learning behaviors in a more realistic class of motor tasks that involve internally-guided sequencing—where the sequence of motor actions is self-generated or internally-specified. In this work, we use the grid-navigation task as an instance of internally-guided sequencing to investigate the nature of learning in such paradigms. The participants performed Grid-Sailing Task (GST), which required navigating (by executing sequential keypresses) a 5 × 5 grid from start to goal (SG) position while using a particular key-mapping (KM) among the three cursor-movement directions and the three keyboard buttons. The participants performed two behavioral experiments—Single-SG and Mixed-SG condition. The Single-SG condition required performing GST on a single SG position repeatedly, whereas the Mixed-SG condition involved performing GST using the same KM on two novel SG positions presented in a random, inter-mixed manner. In the Single-SG condition, we show that motor learning contributes to the sequence-specific learning in GST with the repeated execution of the same trajectories. In the Mixed-SG condition, since the participants utilize the previously learned KM, we anticipate a transfer of learning from the Single-SG condition. The acquisition and transfer of a KM-specific internal model facilitates efficient trajectory planning on novel SG conditions. The acquisition of such a KM-specific internal model amounts to trajectory-independent cognitive learning in GST. We show that cognitive learning contributes to the learning in GST by showing transfer-related performance improvements in the Mixed-SG condition. In sum, we show the role of cognitive and motor learning processes in internally-guided sequencing and further make a case for using GST-like grid-navigation paradigms in investigating internally guided skill learning.

Sequence learning plays a central role in the acquisition of many daily life motor skills such as typing or playing the piano. Several canonical experimental paradigms such as the serial reaction time task, discrete sequence production task and m × n task have been proposed to study the typical behavioral phenomenon in sequencing tasks. Such paradigms are externally-specified, where the environment or the task paradigm extrinsically provides the sequence of stimuli that guides the motor actions. Such paradigms differ from a class of more realistic motor tasks that are internally-guided, where the sequence of motor actions is self-generated or internally-specified. Most previous studies on discrete sequencing have employed externally-specified paradigms and therefore, the cognitive mechanisms underlying skill learning in internally-guided sequencing paradigms remain largely unexplored. This thesis presents an empirical and computational investigation of skill learning in internallyguided sequencing. We employ the Grid-Sailing Task (GST) as a canonical paradigm to study internallyguided sequence learning. The GST requires navigating by executing sequential keypresses, a n × n grid from start to goal (SG) position while using a particular key-mapping (KM) among the three cursormovement directions and the three keyboard buttons. In the first study, we investigate the learning processes involved in internally-guided sequencing. The participants performed two behavioral experiments – Single-SG and Mixed-SG condition. The participants first completed the Single-SG condition, which required performing GST on a single SG position repeatedly. By showing performance-related improvements in various behavioral measures such as the execution time and reward score, we show that motor learning contributes to the trajectory-specific learning in GST with the repeated execution of the same keypress sequences. The Mixed-SG condition involved performing GST using the same KM (from Single-SG condition) on two novel SG positions presented in a random, inter-mixed manner. Since the participants utilize the previously learned KM, we anticipate a transfer of learning from the Single-SG condition. The acquisition and transfer of a KM-specific internal model facilitate efficient trajectory planning on novel SG conditions. The acquisition of such a KM-specific internal model amounts to trajectory-independent cognitive learning in GST. We provide evidence for the role of cognitive learning in GST by showing transfer-related performance improvements in the Mixed-SG condition. In a subsequent study, we probe the involvement of a particular motor learning process called motor chunking. Motor chunking is a phenomenon which enables efficient execution of the motor sequences by chaining several elementary actions into sub-sequences called motor chunks. The participants per-

There have been numerous attempts in explaining the general learning behaviours using model-based and model-free methods. While the model-based control is flexible yet computationally expensive in planning, the model-free control is quick but inflexible. The model-based control is therefore immune from reward devaluation and contingency degradation. Multiple arbitration schemes have been suggested to achieve the data efficiency and computational efficiency of model-based and model-free control respectively. In this context, we propose a quantitative 'value of information' based arbitration between both the controllers in order to establish a general computational framework for skill learning. The interacting model-based and model-free reinforcement learning processes are arbitrated using an uncertainty-based value of information. We further show that our algorithm performs better than Q-learning as well as Q-learning with experience replay.

Several canonical experimental paradigms (serial reaction task, mxn task, etc.) have been proposed to study the typical behavioural phenomena in a sequential motor key-press task. The repeated execution of visuomotor sequences in such paradigms lead to overall performance improvement such that the inter-response intervals in between certain subsequences decreases as compared to that across other sub-sequences. This efficient and hierarchical cluster organisation is called motor chunking. We provide empirical evidence for motor chunking in grid-navigation sequencing tasks. The participants performed Grid-Sailing Task (GST) [Fermin et. al., 2010] that required navigating a 10x10 grid from start to goal position while using a particular key-mapping between the 3 cursor movement directions and the 3 keyboard buttons. This study confirms the emergence of subject-specific, unique temporal patterns related to chunking after substantial practice. 2133 ©2020 The Author(s). This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY).

There have been numerous attempts in explaining the general learning behaviours by various cognitive models. Multiple hypotheses have been put further to qualitatively argue the best-fit model for motor skill acquisition task and its variations. In this context, for a discrete sequence production (DSP) task, one of the most insightful models is Verweys Dual Processor Model (DPM). It largely explains the learning and behavioural phenomenon of skilled discrete key-press sequences without providing any concrete computational basis of reinforcement. Therefore, we propose a quantitative explanation for Verweys DPM hypothesis by experimentally establishing a general computational framework for motor skill learning. We attempt combining the qualitative and quantitative theories based on a best-fit model of the experimental simulations of variations of dual processor models. The fundamental premise of sequential decision making for skill learning is based on interacting model-based (MB) and model-free (MF) reinforcement learning (RL) processes. Our unifying framework shows the proposed idea agrees well to Verweys DPM and Fitts three phases of skill learning. The accuracy of our model can further be validated by its statistical fit with the human-generated data on simple environment tasks like the grid-world.