Motor adaptation to perturbations is provided by learning mechanisms operating in the cerebellum and basal ganglia. The cerebellum normally performs motor adaptation through supervised learning using information about movement error provided by visual feedback. However, if visual feedback is critically distorted, the system may disengage cerebellar error-based learning and switch to reinforcement learning mechanisms mediated by basal ganglia. Yet, the exact conditions and mechanisms of cerebellum and basal ganglia involvement in motor adaptation remain unknown. We use mathematical modeling to simulate control of planar reaching movements that relies on both error-based and non-error-based learning mechanisms. We show that for learning to be efficient only one of these mechanisms should be active at a time. We suggest that switching between the mechanisms is provided by a special circuit that effectively suppresses the learning process in one structure and enables it in the other. To do so, this circuit modulates learning rate in the cerebellum and dopamine release in basal ganglia depending on error-based learning efficiency. We use the model to explain and interpret experimental data on error- and non-error-based motor adaptation under different conditions.

In this study, we explore the functional role of striatal cholinergic interneurons, hereinafter referred to as tonically active neurons (TANs), via computational modeling; specifically, we investigate the mechanistic relationship between TAN activity and dopamine variations and how changes in this relationship affect reinforcement learning in the striatum. TANs pause their tonic firing activity after excitatory stimuli from thalamic and cortical neurons in response to a sensory event or reward information. During the pause striatal dopamine concentration excursions are observed. However, functional interactions between the TAN pause and striatal dopamine release are poorly understood. Here we propose a TAN activity-dopamine relationship model and demonstrate that the TAN pause is likely a time window to gate phasic dopamine release and dopamine variations reciprocally modulate the TAN pause duration. Furthermore, this model is integrated into our previously published model of reward-based motor adaptation to demonstrate how phasic dopamine release is gated by the TAN pause to deliver reward information for reinforcement learning in a timely manner. We also show how TAN-dopamine interactions are affected by striatal dopamine deficiency to produce poor performance of motor adaptation.

The motor cortex controls motor behaviors by generating movement-specific signals and transmitting them through spinal cord circuits and motoneurons to the muscles. Precise and well-coordinated muscle activation patterns are necessary for accurate movement execution. Therefore, the activity of cortical neurons should correlate with movement parameters. To investigate the specifics of such correlations among activities of the motor cortex, spinal cord network and muscles, we developed a model for neural control of goal-directed reaching movements that simulates the entire pathway from the motor cortex through spinal cord circuits to the muscles controlling arm movements. In this model, the arm consists of two joints (shoulder and elbow), whose movements are actuated by six muscles (4 single-joint and 2 double-joint flexors and extensors). The muscles provide afferent feedback to the spinal cord circuits. Cortical neurons are defined as cortical "controllers" that solve an inverse problem based on a proposed straight-line trajectory to a target position and a predefined bell-shaped velocity profile. Thus, the controller generates a motor program that produces a task-specific activation of low-level spinal circuits that in turn induce the muscle activation realizing the intended reaching movement. Using the model, we describe the mechanisms of correlation between cortical and motoneuronal activities and movement direction and other movement parameters. We show that the directional modulation of neuronal activity in the motor cortex and the spinal cord may result from direction-specific dynamics of muscle lengths. Our model suggests that directional modulation first emerges at the level of muscle forces, augments at the motoneuron level, and further increases at the level of the motor cortex due to the dependence of frictional forces in the joints, contractility of the muscles and afferent feedback on muscle lengths and/or velocities.

It is widely accepted that the basal ganglia (BG) play a key role in action selection and reinforcement learning. However, despite considerable number of studies, the BG architecture and function are not completely understood. Action selection and reinforcement learning are facilitated by the activity of dopaminergic neurons, which encode reward prediction errors when reward outcomes are higher or lower than expected. The BG are thought to select proper motor responses by gating appropriate actions, and suppressing inappropriate ones. The direct striato-nigral (GO) and the indirect striato-pallidal (NOGO) pathways have been suggested to provide the functions of BG in the two-pathway concept. Previous models confirmed the idea that these two pathways can mediate the behavioral choice, but only for a relatively small number of potential behaviors. Recent studies have provided new evidence of BG involvement in motor adaptation tasks, in which adaptation occurs in a non-error-based manner. In such tasks, there is a continuum of possible actions, each represented by a complex neuronal activity pattern. We extended the classical concept of the two-pathway BG by creating a model of BG interacting with a movement execution system, which allows for an arbitrary number of possible actions. The model includes sensory and premotor cortices, BG, a spinal cord network, and a virtual mechanical arm performing 2D reaching movements. The arm is composed of 2 joints (shoulder and elbow) controlled by 6 muscles (4 mono-articular and 2 bi-articular). The spinal cord network contains motoneurons, controlling the muscles, and sensory interneurons that receive afferent feedback and mediate basic reflexes. Given a specific goal-oriented motor task, the BG network through reinforcement learning constructs a behavior from an arbitrary number of basic actions represented by cortical activity patterns. Our study confirms that, with slight modifications, the classical two-pathway BG concept is consistent with results of previous studies, including non-error based motor adaptation experiments, pharmacological manipulations with BG nuclei, and functional deficits observed in BG-related motor disorders.

The activity of most motoneurons controlling one-joint muscles during locomotion are locked to either extensor or flexor phase of locomotion. In contrast, bifunctional motoneurons, controlling two-joint muscles such as posterior biceps femoris and semitendinosus (PBSt) or rectus femoris (RF), express a variety of activity patterns including firing bursts during both locomotor phases, which may depend on locomotor conditions. Although afferent feedback and supraspinal inputs significantly contribute to shaping the activity of PBSt and RF motoneurons during real locomotion, these motoneurons show complex firing patterns and variable behaviors under the conditions of fictive locomotion in the immobilized decerebrate cat, i.e., with a lack of patterned supraspinal and afferent inputs. This suggests that firing patterns of PBSt and RF motoneurons are defined by neural interactions inherent to the locomotor central pattern generator (CPG) within the spinal cord. In this study, we use computational modeling to suggest the architecture of spinal circuits representing the locomotor CPG and the connectivity pattern of spinal interneurons defining the behavior of bifunctional PBSt and RF motoneurons. The proposed model reproduces the complex firing patterns of these motoneurons during fictive locomotion under different conditions including spontaneous deletions of flexor and extensor activities and provides insights into the organization of spinal circuits controlling locomotion in mammals.

The motor cortex plays a major role in controlling voluntary movements by sending signals to the muscles through the spinal cord. Due to mapping complexity, the cortical representation of movement and the relationship between cortical neuronal activity and movement dynamics are not fully understood. How different movements are coded and represented in the cortical activity is a central question in motor control. Previous studies were based on two competing hypotheses. One approach attempts to correlate activity of primary motor cortex neurons during reaching movements with movement parameters (e.g., direction). A second approach suggests a dynamical systems model of motor control and assumes that neural activity in the motor cortex reflects temporal progression through dynamic motor programs corresponding to a particular motor task. We have developed a model of a neuronal controller which governs a two-joint arm actuated by six muscles. This model describes the dynamics of voluntary arm movement controlled by a spinal cord circuit with afferent feedback. A thalamo-cerebellar-cortical controller in the model provides the neuronal motor program necessary to simulate reaching movements with bell-shaped velocity profiles and a straight line trajectories. To do so the controller first calculates the joint torques which are necessary to move the arm along the specified trajectory and then generates these torques with agonist and antagonist muscles via specific activation of the low-level spinal circuit. The model demonstrates that patterns of activity in the motor cortex may indeed strongly correlate with parameters of the reaching movement. The correlation is caused by the afferent feedback signals and movement geometry. We conclude that the dynamical paradigm of cortical representation of movement implies coding of both dynamical movement parameters and its geometrical characteristics.