Rewards have important influences on the motor planning of primates and the firing of neurons coding visual information and action. When eye movements to a target are differentially rewarded across locations, primates execute saccades towards the possible target location with the highest expected value, a product of sensory evidence and potentially earned reward (saccade to maximum expected value model, sMEV). Yet, in the natural world eye movements are not directly rewarded. Their role is to gather information to support subsequent rewarded search decisions and actions. Less is known about the effects of decision rewards on saccades. We show that when varying the decision rewards across cued locations following visual search, humans can plan their eye movements to increase decision rewards. Critically, we report a scenario for which five of seven tested humans do not preferentially deploy saccades to the possible target location with the highest reward, a strategy which is optimal when rewarding eye movements. Instead, these humans make saccades towards lower value but clustered locations when this strategy optimizes decision rewards consistent with the preferences of an ideal Bayesian reward searcher that takes into account the visibility of the target across eccentricities. The ideal reward searcher can be approximated with a sMEV model with pooling of rewards from spatially clustered locations. We also find observers with systematic departures from the optimal strategy and inter-observer variability of eye movement plans. These deviations often reflect multiplicity of fixation strategies that lead to near optimal decision rewards but, for some observers, it relates to suboptimal choices in eye movement planning.

During visual search, humans often direct their eyes towards image regions which resemble the target (Rao et al., 2002; Zelinsky, 2008) or toward objects which often spatially co-occur with search targets (Eckstein et al., 2006; Torralba et al., 2006). This behavior is consistent with a saccadic targeting (maximum a posteriori) model, which often approximates the ideal searcher. Here, we report a scenario in which an ideal searcher and humans initially look away from the target in the presence of remote spatial cues to optimize performance. Method: Observers performed a free eye movement search task in which they were to indicate the presence of a Gabor (12 cycles/º) embedded in white luminance noise. The target appeared at one of ten locations arranged in two semi-circles. The orientation of three remote spatial cues (large C’s), situated between the two groups of target locations (eccentricity 5º), indicated where the target would appear if it were present. Observers performed task variants in which the cues differed in whether they provided target location information (C’s vs. O’s), how visible the information in the cues was (large vs. small gap), or whether the cues could be foveated. Results: Performance when remote cues were predictive (95.1±.3%) vastly exceeded that of when the cues did not predict location (60.9±.2%). Moreover, observers systematically directed early saccades towards the cues when they provided target location information. When cues were non-predictive, they were largely ignored and a sequence of eye movements which surveyed as many target locations as possible prevailed. Critically, when cues were predictive but observers were prevented from foveating them, performance suffered (62.5±.5%) compared to when saccades were unrestricted. Conclusions: Humans display initial eye movement patterns which depart from a well-established saccadic targeting strategy when these atypical saccades can enhance performance, as is the case with remote spatial cues. Meeting abstract presented at VSS 2013

Visual attention is commonly studied by using visuo-spatial cues indicating probable locations of a target and assessing the effect of the validity of the cue on perceptual performance and its neural correlates. Here, we adapt a cueing task to measure spatial cueing effects on the decisions of honeybees and compare their behavior to that of humans and monkeys in a similarly structured two-alternative forced-choice perceptual task. Unlike the typical cueing paradigm in which the stimulus strength remains unchanged within a block of trials, for the monkey and human studies we randomized the contrast of the signal to simulate more real world conditions in which the organism is uncertain about the strength of the signal. A Bayesian ideal observer that weights sensory evidence from cued and uncued locations based on the cue validity to maximize overall performance is used as a benchmark of comparison against the three animals and other suboptimal models: probability matching, ignore the cue, always follow the cue, and an additive bias/single decision threshold model. We find that the cueing effect is pervasive across all three species but is smaller in size than that shown by the Bayesian ideal observer. Humans show a larger cueing effect than monkeys and bees show the smallest effect. The cueing effect and overall performance of the honeybees allows rejection of the models in which the bees are ignoring the cue, following the cue and disregarding stimuli to be discriminated, or adopting a probability matching strategy. Stimulus strength uncertainty also reduces the theoretically predicted variation in cueing effect with stimulus strength of an optimal Bayesian observer and diminishes the size of the cueing effect when stimulus strength is low. A more biologically plausible model that includes an additive bias to the sensory response from the cued location, although not mathematically equivalent to the optimal observer for the case stimulus strength uncertainty, can approximate the benefits of the more computationally complex optimal Bayesian model. We discuss the implications of our findings on the field's common conceptualization of covert visual attention in the cueing task and what aspects, if any, might be unique to humans.

A vast literature has shown that the presence of predictive spatial cues nearly invariably improves performance in a variety of perceptual tasks. However, we previously demonstrated that, for a letter identification search task with predictive spatial cues, human performance was actually hindered by cue presence at high signal contrasts (Mack & Eckstein, VSS 2011). We suggested this could be explained by suboptimal overutilization of cue information. Here, we develop a foveated Maximum a Posteriori (MAP) eye movement model to assess this claim. Task: Humans performed a search task in which they were to indicate which of five letters at one of five contrasts was embedded in a noisy image. In cued sessions, colored cue circles indicated likely target locations with varying probability (80% predictive overall). In the remaining 20% of cued trials, targets appeared outside the circles. In ‘uncued’ sessions, the statistics of the images were identical (including the probabilistic target location structure) except for the absence of the cue circles. Model: A MAP eye movement model, which weighted visual evidence by the prior probability of target location, was modulated by eccentricity-dependent white internal noise which simulated decreasing visual acuity in the periphery of the human foveated visual system. Internal noise levels were fit to approximate human performance on uncued trials. Results and Conclusions: In model simulations which implemented optimal weighting of visual information, performance for both cued and uncued trials increased with signal contrast, though performance was consistently superior for cued trials. However, for simulations in which visual information at cued locations was suboptimally overweighted, a reversal in performance was seen at high signal contrasts, as uncued performance exceeded cued performance, qualitatively mirroring our behavioral results. Thus, it appears the misuse of even strongly predictive cue information can, in some circumstances, hinder visual search performance. Meeting abstract presented at VSS 2012

There is evidence that the presence of predictive cues typically improves performance in visual search tasks (e.g. Eckstein et al., 2000; Palmer et al., 2000). Here, we present a novel finding in which search performance was hindered by predictive cues at high signal contrasts. Method: Participants performed a visual search task in which they were to identify one of five letters (A–E) at one of five signal contrasts embedded in white noise. In cued trials, four colored circles were overlaid on the image with their colors indicative of how likely they were to contain the target (Red:40%, Green:20%, Blue:10%, Yellow:10%). The arrangement of the cued regions was arbitrary and identical across trials and participants. In the remaining 20% of trials, the target appeared outside of the cue circles. For uncued trials, stimuli were constructed identically (including the probabilistic structure of the target location) except for the absence of the circles. Results: Performance at low signal contrasts was higher for cued trials than uncued trials, mirroring traditional cueing effects. However, at high signal contrasts, observers achieved higher performance on uncued trials than cued trials. We suggest that these results arose from the overweighting of information at cued locations which impeded performance when targets were easily detectable. Behavioral results support this hypothesis, as performance for the 20% trials in which targets appeared outside the four probabilistically-defined cue areas was significantly lower for cued trials (circles present) than uncued trials (circles absent). Additionally, the frequency of eye movements towards high contrast targets appearing outside of the four probabilistically-defined cue areas sharply diminished when the cue circles were present. Conclusions: While observers often exploit predictive cues to enhance visual search performance, suboptimal overutilization of that statistical information in the guidance of saccadic eye movements can hinder performance when targets are easily detectable.

There is accumulating evidence that scene context can guide and facilitate visual search (e.g., A. Torralba, A. Oliva, M. S. Castelhano, & J. M. Henderson, 2006). Previous studies utilized stimuli of restricted size, a fixed head position, and context defined by the global spatial configuration of the scene. Thus, it is unknown whether similar effects generalize to natural viewing environments and to context defined by local object co-occurrence. Here, with a mobile eye tracker, we investigated the effects of object co-occurrence on search performance under naturalistic conditions. Observers searched for low-visibility target objects on tables cluttered with everyday objects. Targets were either located adjacent to larger, more visible "cue" objects that they regularly co-occurred in natural scenes (expected condition) or elsewhere in the display, surrounded by unrelated objects (unexpected condition). Mean search times were shorter for targets at expected locations as compared to unexpected locations. Additionally, context guided eye movements, as more fixations were directed toward cue objects than other non-target objects, particularly when the cue was contextually relevant to the current search target. These results could not be accounted for by image saliency models. Thus, we conclude that object co-occurrence can serve as a contextual cue to facilitate search and guide eye movements in natural environments.

Studies have shown that when humans search for a known target, eye movements are often guided towards target features (e.g., Findlay, 1997) with occasional fixational compensations for inhomogeneous processing across the visual field (Najemnik & Giesler, 2005). In a different type of search (oddity search), targets and distractors are not known beforehand and observers have to determine whether an element in the display differs from others (distractors) along some feature. Previous research has shown that oddity search has different behavioral properties from target known search, such as different effects of set-size (Bravo & Nakayama, 1992; Schoonveld et al., 2007). Here, we measure human saccadic eye movements during an oddity search task and develop an optimal foveated oddity searcher, which fixates locations that maximize search accuracy, to evaluate human eye movement strategies. Observers performed a yes/no search task in which they were to determine whether one of the five potential target locations contained a singleton target. Stimuli consisted of Gabor patches at three orientations (5° left, vertical, 5° right) embedded in white noise at five locations around an imaginary circle (6° radius). For target present trials, the target was randomly selected from the three possible elements, while one of the remaining elements occupied the other four locations (distractors). For target absent trials, all five locations contained the same randomly selected element. Participants viewed each stimulus for 700 ms (allowing approximately three eye movements) before making a decision. Unlike typical strategies for target known search, observers adopted an eye movement strategy for the oddity search in which they systematically fixated three locations regardless of the stimulus configuration. Such strategy shows little guidance towards the target location. We find that the optimal foveated oddity searcher, like humans, seeks to survey as many of the potential target locations as possible and shows little target guidance.

Although the influence of predictive spatial cues on perceptual decisions has been studied in humans and monkeys, few studies have directly compared cueing effects across species (Bowman et al., 1993). Here, we investigate the effects of spatial cueing and its interaction with target detectability in a similarly structured paradigm across humans, monkeys, and bees and compare the results to a Bayesian ideal observer. Methods: Humans and monkeys participated in the same spatial two alternative forced choice task in which a Gaussian signal of varying detectability (SNRs=0, 2.7, 4.0) embedded in white noise had to be localized. Subjects indicated the target location by making a rapid eye movement towards it. Prior to the onset of the stimulus, a brief precue was presented indicating the target location with 75% accuracy. Bees were trained to fly to one of two boxes containing a target of colored cardboard. The distractor box contained a similar piece of cardboard with a color that varied in its discriminability from the target (e.g. blue/blue vs. blue/grey). A secondary black cardboard served as a cue and co-occurred with the target on 80% of the trials. Results: Cueing effects, defined as the difference in proportion correct for validly and invalidly cued trials, were present for all three species but less than those predicted by an optimal Bayesian observer. These effects were comparable for humans and monkeys, but smaller for bees. However, consistent with ideal observer predictions, cueing effects increased with decreasing detectability of the target for all three organisms. Conclusions: Our results show that the influence of spatial cues on perceptual decisions is pervasive across species. The modulation of the cueing effect with signal strength for all three organisms is consistent with a Bayesian mechanism whereby sensory data are weighted by prior probabilities.

Introduction: There is a growing literature showing how contextual cues guide and facilitate visual search (Chun & Yiang, 1998; Chen & Zelinsky; 2006; Eckstein et al., 2006; Torralba et al., 2006). However, all of these studies used 2-D images and a limited field of view. Here, we investigate the effects of contextual cues on search times and eye movements in a real 3-D scene. Methods: Observers were instructed to search for low visibility objects (e.g., straw, knife) placed on one of four elongated tables. Other distracting objects also cluttered the tables to increase the difficulty of the task. For each observer half the target objects were placed next to highly visible contextual cues (contextual condition; e.g., straw next to a red cup, knife next to plate) while the other half were placed on other table locations surrounded by unrelated items (non-contextual condition). Retinal eccentricity and local salience of the target against the background were matched for each object across conditions. Eye movements were monitored using an Applied Science Laboratories (ASL) mobile eye tracker which monitored the position of the right eye at an effective sampling rate of 30 Hz. Results: Mean human search times to fixate the target were shorter when the object co-occurred with a highly visible contextual cue than when it appeared elsewhere (1.97 vs. 3.8 seconds, p Conclusions: The results extend previous work with 2-D images to show that contextual cues also aid search in a more ecologically valid 3-D environment.