In primates, the Frontal Eye Field (FEF) controls saccades used to search the environment and foveate search targets. The receptive fields (RFs) of FEF neurons are not always statically fixed to a retinotopic location. Rather, they predictively remap to conform to the future fixation position just before a saccade begins. Two modes of perisaccadic remapping have been observed. In forward remapping, the RF jump follows a vector parallel to the upcoming saccade. In convergent remapping, the RF is pulled toward the endpoint of the upcoming saccade. Here, we ask what adaptive purpose RF remapping may serve for visual search behavior, characterized by exploratory and exploitative saccades. Exploratory saccades are high latency, low velocity saccades performed in a sequence to gather visual information, while exploitative saccades are low latency, high velocity, individual saccades performed to foveate the search target. We hypothesize that forward remapping is adaptive for exploratory saccades because it allows for a retinotopically accurate preview that will facilitate potential discovery of targets and planning of upcoming movements. Likewise, we hypothesize that convergent remapping occurs during exploitative saccades, and that this momentarily enhances visual representation around the target. To test this, we recorded from FEF neurons while monkeys searched for a target (gabor patch) embedded in a Perlin noise background. The monkeys were trained to ignore visual probes that were rapidly flashed at random locations during the search behavior. The visual responses elicited by the probes allowed us to infer RF location continuously throughout natural search behavior. We observed forward remapping around exploratory saccades and convergent remapping around exploitative saccades. This effect was strongest in cells that showed delay period activity during a memory guided saccade task. These results suggest that the dynamic reshaping of visual space in FEF facilitates different saccadic strategies. Meeting abstract presented at VSS 2018

When a saccade is expected to result in a reward, both neural activity in oculomotor areas and the saccade itself (e.g., its vigor and latency) are altered (compared with when no reward is expected). As such, it is unclear whether the correlations of neural activity with reward indicate a representation of reward beyond a movement representation; the modulated neural activity may simply represent the differences in motor output due to expected reward. Here, to distinguish between these possibilities, we trained monkeys to perform a natural scene search task while we recorded from the frontal eye field (FEF). Indeed, when reward was expected (i.e., saccades to the target), FEF neurons showed enhanced responses. Moreover, when monkeys accidentally made eye movements to the target, firing rates were lower than when they purposively moved to the target. Thus, neurons were modulated by expected reward rather than simply the presence of the target. We then fit a model that simultaneously included components related to expected reward and saccade parameters. While expected reward led to shorter latency and higher velocity saccades, these behavioral changes could not fully explain the increased FEF firing rates. Thus, FEF neurons appear to encode motivational factors such as reward expectation, above and beyond the kinematic and behavioral consequences of imminent reward.

Like humans, monkeys make saccades nearly three times a second. To understand the factors guiding this frequent decision, computational models of vision attempt to predict fixation locations using bottom-up visual features and top-down goals. How do the relative influences of these factors evolve over multiple time scales? Here we analyzed visual features at fixations using a retinal transform that provides realistic visual acuity by suitably degrading visual information in the periphery. In a task in which monkeys searched for a Gabor target in natural scenes, we characterized the relative importance of bottom-up and task-relevant influences by decoding fixated from nonfixated image patches based on visual features. At fast time scales, we found that search strategies can vary over the course of a single trial, with locations of higher saliency, target-similarity, edge–energy, and orientedness looked at later on in the trial. At slow time scales, we found that search strategies can be refined over several weeks of practice, and the influence of target orientation was significant only in the latter of two search tasks. Critically, these results were not observed without applying the retinal transform. Our results suggest that saccade-guidance strategies become apparent only when models take into account degraded visual representation in the periphery.

OPINION article Front. Integr. Neurosci., 31 March 2011 Volume 5 - 2011 | https://doi.org/10.3389/fnint.2011.00005

Generating sequences of multiple saccadic eye movements allows us to search our environment quickly and efficiently. Although the frontal eye field cortex (FEF) has been linked to target selection and making saccades, little is known about its role in the control and performance of the sequences of saccades made during self-guided visual search. We recorded from FEF cells while monkeys searched for a target embedded in natural scenes and examined the degree to which cells with visual and visuo-movement activity showed evidence of target selection for future saccades. We found that for about half of these cells, activity during the fixation period between saccades predicted the next saccade in a sequence at an early time that precluded selection based on current visual input to a cell's response field. In addition to predicting the next saccade, activity during the fixation prior to two successive saccades also predicted the direction and goal of the second saccade in the sequence. We refer to this as advanced predictive activity. Unlike activity indicating the upcoming saccade, advanced predictive activity occurred later in the fixation period, mirroring the order of the saccade sequence itself. The remaining cells without advanced predictive activity did not predict future saccades but reintroduced the signal for the upcoming saccade at an intermediate time in the fixation period. Together these findings suggest that during natural visual search the timing of FEF cell activity is consistent with a role in specifying targets for one or more future saccades in a search sequence.

Brain–machine interfaces (BMIs) have the potential to improve the quality of life for individuals with disabilities. We engaged in the development of neural mind-reading techniques for cognitive BMIs to provide a readout of decision processes. We trained 2 monkeys on go/no-go tasks, and monitored the activity of groups of neurons in their mid-brain superior colliculus (SC). We designed a virtual decision function (VDF) reflecting the continuous progress of binary decisions on a single-trial basis, and applied it to the ensemble activity of SC neurons. Post hoc analyses using the VDF predicted the cue location as well as the monkey’s motor choice (go or no-go) soon after the presentation of the cue. These results suggest that our neural mind-reading techniques have the potential to provide rapid real-time control of communication support devices.

To examine the function of the superior colliculus (SC) in decision-making processes and the application of its single trial activity for "neural mind reading," we recorded from SC deep layers while two monkeys performed oculomotor go/no-go tasks. We have recently focused on monitoring single trial activities in single SC neurons, and designed a virtual decision function (VDF) to provide a good estimation of single-dimensional decisions (go/no-go decisions for a cue presented at a specific visual field, a response field of each neuron). In this study, we used two VDFs for multidimensional decisions (go/no-go decisions at two cue locations) with the ensemble activity which was simultaneously recorded from a small group (4 to 6) of neurons at both sides of the SC. VDFs predicted cue locations as well as go/no-go decisions. These results suggest that monitoring of ensemble SC activity had sufficient capacity to predict multidimensional decisions on a trial-by-trial basis, which is an ideal candidate to serve for cognitive brain-machine interfaces (BMI) such as two-dimensional word spellers.

The purpose of this study was to develop an algorithm capable of transforming neural activity to correctly report behavioral outcome during a cognitive task. We recorded from small groups of 2-5 neurons in the superior colliculus (SC) while monkeys performed a go/no-go task. Depending upon the color of a peripheral stimulus, the monkey was required to either make a saccade to the stimulus (go) or maintain fixation (no-go). In order to replicate the progress of the decision-making process and generate a virtual decision function (VDF), we performed a multiple regression analysis, with 1 msec resolution, on neuron activity during individual trials. Post hoc analyses by VDFpredicted the monkey's choice with nearly 90% accuracy. These results suggest that monitoring of a limited number of SC neurons has sufficient capacity to predict go/no-go decisions on a trial-by-trial basis, and serves as an ideal candidate for a cognitive brain-machine interface (BMI).

Monkeys made saccades to one of two peripheral targets based on the brightness of a central stimulus. Task difficulty was manipulated by varying the ratio of stimulus black-and-white pixels. Correct response probability for two monkeys varied directly with difficulty. Deep layer SC neurons exhibited robust presaccadic activity the magnitude of which was unaffected by task difficulty when the stimulus specified a saccade toward a target within the neuron's response field. Activity after stimuli specifying saccades to targets outside the response field was affected by task difficulty, increasing as the task became more difficult. A quantitative model derived from studies of human decision-making was fit to the behavioral data. The model assumes that information from the stimulus drives two independent diffusion processes. Simulated paths from the model were compared with neuron activity, assuming that firing rate is linearly related to position in the accumulation process. The firing rate data show delayed availability of discriminative information for fast, intermediate, and slow decisions when activity is aligned on the stimulus and very small differences in discriminative information when aligned on the saccade. The model produces exactly these patterns of results. The accumulation process is highly variable, allowing the process both to make errors, as is the case for the behavioral performance, and also to account for the firing rate results. Thus the dual diffusion model provides a quantitative account for both the behavior in a simple decision-making task as well as the patterns of activity in competing populations of neurons.

The enhanced functional sensitivity offered by ultra-high field imaging may significantly benefit simultaneous EEG-fMRI studies, but the concurrent increases in artifact contamination can strongly compromise EEG data quality. In the present study, we focus on EEG artifacts created by head motion in the static B0 field. A novel approach for motion artifact detection is proposed, based on a simple modification of a commercial EEG cap, in which four electrodes are non-permanently adapted to record only magnetic induction effects. Simultaneous EEG-fMRI data were acquired with this setup, at 7 T, from healthy volunteers undergoing a reversing-checkerboard visual stimulation paradigm. Data analysis assisted by the motion sensors revealed that, after gradient artifact correction, EEG signal variance was largely dominated by pulse artifacts (81–93%), but contributions from spontaneous motion (4–13%) were still comparable to or even larger than those of actual neuronal activity (3–9%). Multiple approaches were tested to determine the most effective procedure for denoising EEG data incorporating motion sensor information. Optimal results were obtained by applying an initial pulse artifact correction step (AAS-based), followed by motion artifact correction (based on the motion sensors) and ICA denoising. On average, motion artifact correction (after AAS) yielded a 61% reduction in signal power and a 62% increase in VEP trial-by-trial consistency. Combined with ICA, these improvements rose to a 74% power reduction and an 86% increase in trial consistency. Overall, the improvements achieved were well appreciable at single-subject and single-trial levels, and set an encouraging quality mark for simultaneous EEG-fMRI at ultra-high field.

While some decision-making processes often result in the generation of an observable action, for example eye or limb movements, others may prevent actions and occur without an overt behavioral response. To understand how these decisions are made, one must look directly at their neuronal substrates. We trained two monkeys on a go/no-go task which requires a saccade to a peripheral cue stimulus (go) or maintenance of fixation (no-go). We performed binary regressions on the activity of single neurons in the superior colliculus (SC), with the go/no-go decision as a predictor variable, and constructed a virtual decision function (VDF) designed to provide a good estimation of decision content and its timing in a single trial decision process. Post hoc analyses by VDF correctly predicted the monkey's choice in more than 80% of trials. These results suggest that monitoring of SC activity has sufficient capacity to predict go/no-go decisions on a trial-by-trial basis.

The categorization performance of monkeys on a unidimensional perceptual categorization task was examined with reference to decision bound and exemplar theories of categorization. Three rhesus monkeys were presented with stimuli varying along a single dimension, the displacement of a target light from a fixation point. Left or right saccade responses were probabilistically reinforced according to one of three functions, two of which were nonmonotonic at one end of the stimulus space. The monkeys all showed a monotonic increase in response probability as a function of target light displacement in this region, consistent with decision bound theory. Fits of a single-boundary model (GRT; Ashby & Gott, 1988) and two exemplar models—one using a probabilistic response function (GCM; Nosofsky, 1986), the other using a deterministic response function (DEM; Ashby & Maddox, 1993)—revealed overall support for the decision bound model. The results suggest that monkeys used a perceptual decision boundary to perform the task.

Recently, models in psychology have been shown capable of accounting for the full range of behavioral data from simple two-choice decision tasks: mean reaction times for correct and error responses, accuracy, and the reaction time distributions for correct and error responses. At the same time, recent data from neural recordings have allowed investigation of the neural systems that implement such decisions. In the experiment presented here, neural recordings were obtained from superior colliculus prelude/buildup cells in two monkeys while they performed a two-choice task that has been used in humans for testing psychological models of the decision process. The best-developed psychological model, the diffusion model, and a competing model, the Poisson counter model, were explicitly fit to the behavioral data. The pattern of activity shown in the prelude/buildup cells, including the point at which response choices were discriminated, was matched by the evidence accumulation process predicted from the diffusion model using the parameters from the fits to the behavioral data but not by the Poisson counter model. These results suggest that prelude/buildup cells in the superior colliculus, or cells in circuits in which the superior colliculus cells participate, implement a diffusion decision process or a variant of the diffusion process.

The frontal eye field (FEF) of monkeys has been repeatedly implicated in the generation of saccadic eye movements by various experimental approaches. Electrical stimulation of most of the FEF produces saccadic eye movements, many cells have activities related to saccades, and it has anatomical connections with many other oculomotor areas. Surprisingly, complete lesions of the FEF have remarkably little effect on oculomotor behavior. Only when more cognitive aspects are tested is a deficit clearly detected. In contrast, acute inactivation of the FEF of monkeys with the GABA agonist muscimol produced much more severe oculomotor impairment. This difference is probably due to the acute nature of the muscimol effect, which does not allow time for reorganization of the control of eye movements before testing begins. In addition, acute activation of the FEF with the GABA antagonist bicuculline caused the monkey to make irrepressible saccades of the same dimensions as those electrically elicited at the site. These experiments further confirm the strong involvement of the FEF in the control of saccadic eye movements and fixation.

AbstractOn monkeys and cats, microstimulation of the superior colliculus evokes gaze movements containing both eye and head movement components. The superficial layers of the superior colliculus in these species contain a simple point-to-point representation of the retina. This retinotopic map is parallel with a movement activity map in the intermediate layers produced by neurons that are stimulated before and during saccades of specific amplitude and direction. Early reports of electrical stimulation of the monkey's superior colliculus with the head restrained focused on the correspondence of sensory and motor maps, finding that the trajectories of saccadic eye movements evoked by electrical stimulation, depended mainly upon the location of the stimulation site, with the eyes' orbital position having only a very slight effect upon the amplitude and direction of the evoked eye movement.

The saccadic system uses a muscular apparatus and motor programs that evolved long before the cerebral cortex assumed the dominant role in the generation of behavior that it occupies in the primate. The cortical role in eye movements therefore is to contribute aspects of sophisticated processing to the basic apparatus for rapid eye movements. Thus visual cortex is necessary for the integration of visual motion information into the saccadic system, because the superior colliculus in the primate cannot do adequate motion processing. Similarly, frontal cortex is necessary for performing saccades to remembered stimulus positions, whereas visually driven saccades can be performed by the colliculus alone. To generate saccade-related information, the cortex has activity that reflects all levels of processing, from the registration of the stimulus and the selection of a stimulus for a saccade, to the elaboration of the motor command for the saccade. Presumably the cortex also contains the decision mechanism, whereby a primate decides to make a saccade to a certain stimulus. How, and where that decision is made, or even if the decision occurs at a single place, is totally unknown.