In this article, we discuss operationalizations and examples of experimental design in eye-tracking research. First, we distinguish direct operationalization for entities like saccades, which are closely aligned with their original concepts, and indirect operationalization for concepts not directly measurable, such as attention or mind-wandering. The latter relies on selecting a measurable proxy. Second, we highlight the variability in algorithmic operationalizations and emphasize that changing parameters can affect outcome measures. Transparency in reporting these parameters and algorithms is crucial for comparisons across studies. Third, we provide references to studies for common operationalizations in eye-tracking research and discuss key operationalizations in reading research. Fourth, the IO-model is introduced as a tool to help researchers operationalize difficult concepts. Finally, we present three example experiments with useful methods for eye-tracking research, encouraging readers to consider these examples for inspiration in their own experiments.

How do researchers in psychology view the relation between scientific knowledge, its applicability, and its societal relevance? Most research on psychological science and its benefits to society is discussed from a bird's eye view (a meta-scientific perspective), by identifying general trends such as psychology's dominant focus on lab-based experiments and general descriptive theories. In recent years, several critics have argued that this focus has come at the cost of reduced practical and societal relevance. In this study, we interviewed Dutch psychology professors to gauge their views about the relation between psychological research and its relevance to society. We found that psychology professors engaged in a variety of activities to engage science with society, from work in clinical and applied settings, to consultancy, education, and science communication. However, we found that the role of theory when applying scientific knowledge to practical problems is far from straightforward. While most participants regarded theories as relevant to understanding general contexts of application, psychological theories were seldom directly related to specific applications. We compare and discuss our findings in the light of recent discussions about the lack of applicability and societal relevance of psychological science.

We disagree with Almaatouq et al. that no realistic alternative exists to the "one-at-a-time" paradigm. Seventy years ago, Egon Brunswik introduced representative design, which offers a clear path to commensurability and generality. Almaatouq et al.'s integrative design cannot guarantee the external validity and generalizability of results which is sorely needed, while representative design tackles the problem head on.

Eye contact is essential for human interactions. We investigated whether humans are able to avoid eye contact while navigating crowds. At a science festival, we fitted 62 participants with a wearable eye tracker and instructed them to walk a route. Half of the participants were further instructed to avoid eye contact. We report that humans can flexibly allocate their gaze while navigating crowds and avoid eye contact primarily by orienting their head and eyes towards the floor. We discuss implications for crowd navigation and gaze behavior. In addition, we address a number of issues encountered in such field studies with regard to data quality, control of the environment, and participant adherence to instructions. We stress that methodological innovation and scientific progress are strongly interrelated.

A primary mode of human social behavior is face-to-face interaction. In this study, we investigated the characteristics of gaze and its relation to speech behavior during video-mediated face-to-face interactions between parents and their preadolescent children. 81 parent–child dyads engaged in conversations about cooperative and conflictive family topics. We used a dual-eye tracking setup that is capable of concurrently recording eye movements, frontal video, and audio from two conversational partners. Our results show that children spoke more in the cooperation-scenario whereas parents spoke more in the conflict-scenario. Parents gazed slightly more at the eyes of their children in the conflict-scenario compared to the cooperation-scenario. Both parents and children looked more at the other's mouth region while listening compared to while speaking. Results are discussed in terms of the role that parents and children take during cooperative and conflictive interactions and how gaze behavior may support and coordinate such interactions.

A popular goal in psychological science is to understand human cognition and behavior in the ‘real-world.’ In contrast, researchers have typically conducted their research in experimental research settings, a.k.a. the ‘psychologist’s laboratory.’ Critics have often questioned whether psychology’s laboratory experiments permit generalizable results. This is known as the ‘real-world or the lab’-dilemma. To bridge the gap between lab and life, many researchers have called for experiments with more ‘ecological validity’ to ensure that experiments more closely resemble and generalize to the ‘real-world.’ However, researchers seldom explain what they mean with this term, nor how more ecological validity should be achieved. In our opinion, the popular concept of ecological validity is ill-formed, lacks specificity, and falls short of addressing the problem of generalizability. To move beyond the ‘real-world or the lab’-dilemma, we believe that researchers in psychological science should always specify the particular context of cognitive and behavioral functioning in which they are interested, instead of advocating that experiments should be more ‘ecologically valid’ in order to generalize to the ‘real-world.’ We believe this will be a more constructive way to uncover the context-specific and context-generic principles of cognition and behavior.

Research conducted in laboratories is unlikely to be representative of human functioning in the “real world.” Or so one is led to believe when one sifts through the psychological literature of the last century. Discussions about the generalisability of laboratory research beyond its confines have arisen in the fields of perception, attention, memory, and social cognition as well as many applied sciences (see Holleman et al., 2020b, for an overview). More often than not, papers advocating research in the “real world” are published in highimpact journals. Of course, this is in itself no reason for particular concern. What does concern us is that some of these papers make unrealistic claims about how “real-world research”—whatever its precise definition—is to be conducted. The fact that such claims are made in high-impact journals is worrisome, as papers in these journals are often the basis for grant applications or serve as a starting point for new researchers. A recent claim that particularly concerns us is that techniques such as wearable eye trackers, mobile electroencephalography (EEG), and mobile functional near-infrared spectroscopy (fNIRS) will allow researchers to finally conduct “real-world research” (P erezEdgar et al., 2020; Shamay-Tsoory & Mendelsohn, 2019). The gist is that with wearable technology, one can freely roam the world, while (eye-movement) behaviour and its neural correlates are continuously measured, allowing for profound insights into human perception and cognition. In our view, there is a stark contrast between how this claim is presented and the real problems of the empirical work. We have often found that researchers vastly underestimate the reality of conducting research with wearable technology, whether it is conducted inside or outside of the laboratory. Using wearable eye tracking as an example, we show that there are theoretical, conceptual, and methodological problems when moving from classical laboratory-based research to research using wearable technology, which many researchers seem to be unaware of. Although our focus is on eye-tracking research, as that is where our expertise lies, we expect that similar concerns can be raised about research using mobile EEG or mobile fNIRS. The fact that some researchers overestimate what wearable eye trackers can deliver is perhaps best illustrated by a quote from a recent paper advocating the use of wearable technology for “real-world research.” Shamay-Tsoory & Mendelsohn (2019) state that “. . . newly available portable eye-tracking systems offer a cost-effective, easy to apply, and reliable measure of eye gaze and saccades in an ecological environment” (p. 853, our emphasis). If this were true, wearable eye trackers might indeed be wonderful tools for studying “real-life” behaviour. However, we contest these three postulated characteristics of wearable Perception 2020, Vol. 49(6) 611–615 ! The Author(s) 2020

Face-to-face interaction is a primary mode of human social behavior which includes verbal and non-verbal expressions, e.g. speech, gazing, eye contact, facial displays, and gestures (Holler & Levinson, 2019). In this study, we investigated the relation between speech and gaze behavior during 'face-to-face' dialogues between parents and their preadolescent children (9-11 years). 79 child-parent dyads engaged in two semi-structured conversations about family-related topics. We used a state-of-the-art dual-eye tracking setup (Hessels et al. 2019) that is capable of concurrently recording eye movements, frontal video recordings, and audio from two conversational partners. Crucially, the setup is designed in such a way that eye contact can be maintained using half-silvered mirrors, as opposed to e.g. Skype where the camera is located above the screen. Parents and children conversed about two different topics for five minutes each, one 'conflict' (e.g. bedtime, homework) and one 'cooperation' (e.g. organize a party) topic. Preliminary analyses of speech behavior (Figure 1) show that children talked more in the cooperative task and talked less when discussing a topic of disagreement with their parents. Conversely, parents talked more during the conflict-task and less during the cooperative-task. The next step is to combine measures of speech and gaze to investigate the interplay and temporal characteristics of verbal and non-verbal behavior during face-to-face interactions.

Human crowds provide an interesting case for research on the perception of people. In this study, we investigate how visual information is acquired for (1) navigating human crowds and (2) seeking out social affordances in crowds by studying gaze behavior during human crowd navigation under different task instructions. Observers ( n = 11) wore head-mounted eye-tracking glasses and walked two rounds through hallways containing walking crowds ( n = 38) and static objects. For round one, observers were instructed to avoid collisions. For round two, observers furthermore had to indicate with a button press whether oncoming people made eye contact. Task performance (walking speed, absence of collisions) was similar across rounds. Fixation durations indicated that heads, bodies, objects, and walls maintained gaze comparably long. Only crowds in the distance maintained gaze relatively longer. We find no compelling evidence that human bodies and heads hold one’s gaze more than objects while navigating crowds. When eye contact was assessed, heads were fixated more often and for a total longer duration, which came at the cost of looking at bodies. We conclude that gaze behavior in crowd navigation is task-dependent, and that not every fixation is strictly necessary for navigating crowds. When explicitly tasked with seeking out potential social affordances, gaze is modulated as a result. We discuss our findings in the light of current theories and models of gaze behavior. Furthermore, we show that in a head-mounted eye-tracking study, a large degree of experimental control can be maintained while many degrees of freedom on the side of the observer remain.

The main thrust of Shamay-Tsoory and Mendelsohn’s ecological approach is that “the use of real-life complex, dynamic, naturalistic stimuli provides a solid basis for understanding brain and behavior” (p. 851). Although we support the overall goal and objectives of Shamay-Tsoory and Mendelsohn’s approach to “real-life” neuroscience, their review refers to the terms “ecological validity” and “representative design” in a manner different from that originally introduced by Egon Brunswik. Our aim is to clarify Brunswik’s original definitions and briefly explain how these concepts pertain to the larger problem of generalizability, not just for history’s sake, but because we believe that a proper understanding of these concepts is important for researchers who want to understand human behavior and the brain in the context of everyday experience, and because Brunswik’s original ideas may contribute to Shamay-Tsoory and Mendelsohn’s ecological approach. Finally, we argue that the popular and often misused concept of “ecological validity” is ill-formed, lacks specificity, and may even undermine the development of theoretically sound and tractable research.

As humans move through parts of their environment, they meet others that may or may not try to interact with them. Where do people look when they meet others? We had participants wearing an eye tracker walk through a university building. On the way, they encountered nine “walkers.” Walkers were instructed to e.g. ignore the participant, greet him or her, or attempt to hand out a flyer. The participant's gaze was mostly directed to the currently relevant body parts of the walker. Thus, the participants gaze depended on the walker's action. Individual differences in participant's looking behavior were consistent across walkers. Participants who did not respond to the walker seemed to look less at that walker, although this difference was not statistically significant. We suggest that models of gaze allocation should take social motivation into account.

Researchers have increasingly focused on how the potential for social interaction modulates basic processes of visual attention and gaze behavior. In this study, we investigated why people may experience social interaction and what factors contributed to their subjective experience. We furthermore investigated whether implying social interaction modulated gaze behavior to people's faces, specifically the eyes. To imply the potential for interaction, participants received either one of two instructions: 1) they would be presented with a person via a 'live' video-feed, or 2) they would be presented with a pre-recorded video clip of a person. Prior to the presentation, a confederate walked into a separate room to suggest to participants that (s)he was being positioned behind a webcam. In fact, all participants were presented with a pre-recorded clip. During the presentation, we measured participants' gaze behavior with an eye tracker, and after the presentation, participants were asked whether they believed that the confederate was `live' or not, and, why they thought so. Participants varied greatly in their judgements about whether the confederate was `live' or not. Analyses of gaze behavior revealed that a large subset of participants who received the live-instruction gazed less at the eyes of confederates compared with participants who received the pre-recorded-instruction. However, for both the live-instruction group and the prerecorded instruction group, another subset of participants gazed predominantly at the eyes. The current findings may contribute to the development of experimental designs aimed to capture the interactive aspects of social cognition and visual attention.

Gaze allocation to human faces has recently been shown to be greatly dependent on the social context. However, what has not been considered explicitly here, is how gaze allocation may be supportive of the specific task that individuals carry out. In the present study, we combined these two insights. We investigated (1) how gaze allocation to facial features in face-to-face communication is dependent on the task-structure and (2) how gaze allocation to facial features is dependent on the gaze behavior of an interacting partner. To this end, participants and a confederate were asked to converse, while their eye movements were monitored using a state-of-the-art dual eye-tracking system. This system is unique in that participants can look each other directly in the eyes. We report that gaze allocation depends on the sub-task being carried out (speaking vs. listening). Moreover, we show that a confederate's gaze shift away from the participants affects their gaze allocation more than a gaze shift towards them. In a second experiment, we show that this gaze-guidance effect is not primarily stimulus-driven. We assert that gaze guidance elicited by the confederate looking away is related to the participants' sub-task of monitoring the confederate for when they can begin speaking. This study exemplifies the importance of both task structure and social context for gaze allocation during face-to-face communication.

Most modern video eye trackers deliver binocular data. Many researchers take the average of the left and right eye signals (the version signal) to decrease the variable error (precision) up to a factor of [Formula: see text]. What happens to the systematic error (accuracy) if the left and right eye signals are averaged? To determine the systematic error, we conducted a calibration validation in two experiments (n= 79 and n = 64). The systematic error was computed for the left eye, right eye, and version signals separately. In respectively 29.5 and 25.8% of the participants, the systematic error of a single eye signal was lower than that of the version signal at the cost of a higher variable error. If a small variable error is desirable, and the difference between the left and the right eye is not the topic of study, one should average position data from the left and the right eye (in other words, use the version signal). If a small systematic error is desirable, one should use the signal (from left eye, right eye or version) that delivers the best accuracy. In the latter case, this may cause worse precision than that of the version signal.

Research on social impairments in psychopathology has relied heavily on the face processing literature. However, although many sub-systems of facial information processing are described, recent evidence suggests that generalizability of these findings to social settings may be limited. The main argument is that in social interaction, the content of faces is more dynamic and dependent on the interplay between interaction partners, than the content of a non-responsive face (e.g. pictures or videos) as portrayed in a typical experiment. The question beckons whether gaze atypicalities to non-responsive faces in certain disorders generalize to faces in interaction. In the present study, a dual eye-tracking setup capable of recording gaze with high resolution was used to investigate how gaze behavior in interaction is related to traits of Autism Spectrum Disorder (ASD), and Social Anxiety Disorder (SAD). As clinical ASD and SAD groups have exhibited deficiencies in reciprocal social behavior, traits of these two conditions were assessed in a general population. We report that gaze behavior in interaction of individuals scoring high on ASD and SAD traits corroborates hypotheses posed in typical face-processing research using non-responsive stimuli. Moreover, our findings on the relation between paired gaze states (when and how often pairs look at each other’s eyes simultaneously or alternately) and ASD and SAD traits bear resemblance to prevailing models in the ASD literature (the ‘gaze aversion’ model) and SAD literature (the ‘vigilant-avoidance’ model). Pair-based analyses of gaze may reveal behavioral patterns crucial to our understanding of ASD and SAD, and more general to our understanding of eye movements as social signals in interaction.

An important mode of human social behavior is face-to-face interaction. Face-to-face interactions consist of various behaviors, such as gaze, speech, facial expressions, and gestures. Researchers interested in the social functions of gaze, also referred to as social attention, have often used eye-tracking to measure where people are looking. In recent ... read more years, several researchers have criticized some laboratory procedures used in research on social attention because these procedures are not representative of ‘real world’ social situations. To counter this problem, researchers have advocated that research should be conducted in the real world instead of the laboratory. This dissertation consists of two parts: a theoretical part and an empirical part. The theoretical part is about the question how and to what extent research conducted in a laboratory is representative of human cognition and behavior in the real world. I analyze the conceptual and historical interpretations of the concept of ecological validity. This concept has often been used to qualify to what extent behavior in a laboratory resembles behavior in the real world. The empirical part consists of two eye-tracking experiments in which human gaze behavior during face-to-face interactions is investigated, for example, conversations between parents and children. These studies show that some people are eye- and mouth lookers, that parents gaze more at the eyes of their child during conflicting conversations than when parents and children need to collaborate, and that people can experience having a ‘live’ video-interaction with another person when in fact, they are looking at a pre-recorded videoclip. show less