: In content viewing activities, such as movies and paintings, it is important to retain and utilize the viewing experience in memory. We have been studying the effect of the content of visual and auditory information provided during viewing activities and presentation timing on content memory. We have clarified the appropriate timing of presenting visual information that should be supplemented by auditory information. We have also found that the inclusion of emotion induction words in the auditory information is effective in forming content memory. In this study, we present a framework for examining the effects of emotion-evoking characteristics of short sentences while taking into account individual differences in memory. Subjects were presented with a short sentence with an emotion-inducing word at the beginning of the sentence, in which the impression of the entire short sentence would appear at the end of the sentence. We designed an experimental system to clarify the relationship between subject-specific pupillary responses to the emotion induction words and memory for short sentences. Our findings indicate a scheme that relates the pupillary response to short sentence memory.

This study aimed to examine the possibility of using emotion-induction words in audio guides for education via visual content. This was performed based on the findings of a previous study that focused on the provision timings of visual and auditory information [6]. Thirty emotion-induction words were extracted from the database and categorized into positive, negative, and neutral words, and three experiments were performed. The first experiment was conducted to confirm the reliability of emotional values. The results revealed a strong consistency between the values in the database and the ratings given by the participants. The second experiment assessed whether consistency was maintained if the words appeared in the sentences. The results confirmed that a certain degree of consistency was maintained, as expected, but showed larger individual differences compared with the first experiment. The third experiment was conducted to probe the effect of emotion-induction words used in the audio guide to explain the visual content of memory. Our results revealed that participants who were exposed to positive and negative emotion-induction words remembered the content better than those who were presented with neutral words. Per the three experiments, the emotion value of the neutral words was found to be sensitive to the context in which they were embedded, which was confirmed by observing the changes in pupillary reactions. Suggestions for designing audio and visual content using emotion-induction words for better memory are provided.

: This paper proposes a novel method for analyzing the relationships between the difficulty of tasks and the effort of learners to accomplish them using biometric information. The biometric information we adopted was as follows: 1) pupil diameter variation for estimating subjective task difficulty, and 2) eye movements indicative of answer selection times for assessing subjective efforts and strategies to solve the problems. The data used in this study are eye movement data obtained in a different study for studying brain activities during arithmetic calculations in terms of electroencephalography (EEG) data (Suzuki et al., 2021). This study re-analyzed the eye movement data by introducing the following two variables: 1) the duration times in the characteristic areas for solving the tasks to understand how the participants strategically retrieved the task information, and 2) the changes in the sizes of pupil diameter to understand the levels of engagement of the participants while solving the tasks. This study suggests that the relationships found in these variables should characterize the participants’ learning attitudes and could be related to confidence and satisfaction in the attention-relevance-confidence-satisfaction (ARCS) model, indicating the possibility of applying the results to educational systems.

The goal of this paper is to examine the possibility of using emotion-induction words in audio guide for the learning of visual contents by extending the study that focused on the provision timings of visual and auditory information (Hirabayashi et al., 2020). Thirty emotion-induction words were extracted from the database and categorized into positive, negative, and neutral words. Three experiments were carried out. The first experiment was conducted to confirm the reliability of the emotional values. The result showed a good consistency between the values on the database and the ratings given by the participants. The second experiment was for examining whether the consistency is maintained if the words appeared in sentences. The result confirmed the expectation but showed larger individual differences compared with the first experiment. The third experiment was conducted to examine the effect of emotion-induction words used in audio guide for explaining the visual contents on memory. The results showed that the participants who were exposed to the positive and negative emotion-induction words, remembered the content better than those who were presented with neutral triggers. Through the three experiments, the emotion value of the neutral words were found to be sensitive to the context in which they were embedded, which was confirmed by observing the changes of pupil diameter. Suggestions for designing audio and visual contents by using emotion-induction words for better memory are provided.

Meme is a type of behavior that is passed from one member of a group to another, not through the genes but by other means. This paper claims that the concept of resonance should play an important role in the propagation of memes among group members and the development of meme in individual members. Dinet et al. pointed out that the concept of resonance originally issued from physics that has been successfully applied to cognitive processes for behavior selection; the Model Human Processor with Real-Time Constraints (MHP/RT) model proposed by Kitajima and Toyota is a unique model that incorporates a resonance mechanism to connect perceptual-cognitive-motor (PCM) processes that work synchronously, and memory processes that work asynchronously with the environment. This paper elaborates the resonance mechanism implemented in MHP/RT and its associated multi-dimensional memory frames from three perspectives: how resonance works in a single-action selection process; what types of memes can exist in multi-dimensional memory frames; how PCM processes and memory develop from birth to the end of adolescence in terms of the detailed workings of resonance. This paper concludes with the implications of the mechanistic understanding of the role of resonance, the effect of resonance malfunction and the effect of the existence of memory that does not resonate with the real environment. Keywords–Resonance; Meme; MHP/RT; Development.

This study focuses on audio guide as a support for smooth information acquisition for visual stimuli. The interval between provision timing of visual guidance part, which explains explicit features of the object, and information addition part, which explains implicit features of the object, is set as a parameter and its effect on memory is measured as an indicator for estimating the degree of smoothness in information acquisition. Eye tracking experiments were conducted in a dome theater with the omnidirectional movie using three timing interval conditions: shorter than two seconds (Short Interval), longer than three seconds and shorter than five seconds (Medium Interval), and longer than six seconds (Long Interval). The results showed that the memory scores for the movie presented in the Medium Interval condition was the largest. This paper discusses how the presentation in the Medium Interval condition allowed effective integration of visual information and the auditory information provided by audio guide: the visual guidance part of audio guide helped the viewer to find the objects at the best timing before the presentation of information addition part. This would have enabled the participants to elaborate the visual scene with the relevant long-term memory for integration with the auditory information.

The UN report, “The Future is Now: Science for Achieving Sustainable Development,” expressed expectations for contributions from cognitive science for the achievement of the Sustainable Development Goals (SDGs). This is because achievement of the SDGs can be regarded as an extension of problem-solving activities in individuals’ daily behavior choices. However, as categorized in Newell’s “time scale of human action,” the SDGs belong to the SOCIAL BAND, while individuals’ daily problem-solving belongs to the COGNITIVE BAND, which makes it difficult to construct predictive models. In other words, it is impossible to define a well-defined problem space that spans between the non-linearly connected BANDs. As an alternative approach, this paper proposes an adapted version of the Cognitive Chrono-Ethnography (CCE), a study methodology integrating cognitive science and ethnography, for understanding individuals’ daily behavior and specifying their action selection activities that would eventually lead to the achievement of some of the SDGs. Keywords–Sustainable Development Goals (SDGs); Cognitive Chrono-Ethnography (CCE); real world problem-solving; adaptive problem-solving; happiness goals.

Museums offer the opportunity to acquire knowledge about artistic, cultural, historical or scientific interest through a large number of exhibitions. However, even if these masterpieces are visually accessible to all visitors, the background of these works is not necessarily acquired because visitors do not have enough knowledge to fully appreciate them. An audio guide is a tool commonly used to fill this gap. The purpose of this study is to understand the relationships between the eye movements of visitors for the acquisition of information by seeing, the content of the audio guide that should help them understand the objects by hearing, and the contentment level of museum experience. This paper reports the results of an eye-tracking experiment in which eighteen participants were invited to appreciate a variety of images with or without an audioguide used in an actual museum, to complete a questionnaire on subjective feelings and to attend an interview. It is found that the relationship between the viewing time or the frequency of fixation and the satisfaction of the sight, and the effect of the audio-guide on these eye movements. And also found that participants could be categorized into four categories, suggesting an effective way to provide an audio guide.

—Behavior of users interacting with multimodal inter- faces looks complex because the degrees of freedom of sensory input and motor output is large. This paper suggests that this complexity can be alleviated by applying the Simon’s ant metaphor to the multimodal interaction situations, i.e., “What users do is simple. 1) they use perceptual input to generate a mirror image of the real world surrounding the self to be shared in conscious and unconscious processes, 2) they select next actions by consciously planning ahead and unconsciously tuning motor movements for the event to happen, and after performing the action, they unconsciously modify the participated neural network and consciously reflect on the result of action, and 3) they perform 1 and 2 in synchronous with the ever-changing external environment, which this paper calls “weak synchronization.” A cognitive architecture, Model Human Processor with Realtime Constraints (MHP/RT) and its associated memory structure, Multi-Dimensional Memory Frames, developed by the authors is briefly introduced considering the situation of users interacting with multimodal environment. Then, the above three items are derived as the essential principles for organizing user’s behavior in multimodal interaction environment. Future work on designing mixed reality multimodal interaction environment is introduced that has its basis on the perspectives for multimodal interactions this paper claims.

In many large cities worldwide, traffic congestion is a critical issue. In general, secondary tasks are considered dangerous while driving. However, in heavy traffic, where the major driving task is significantly reduced, secondary tasks may reduce burden on drivers, maintain their arousal level, and cause them to perceive time as moving more quickly. In this study, we conducted an experiment using virtual congestion to clarify drivers' stress, reaction delay, and perceived time under in-car activities of listening to music (passive task), and completing a quiz and talking with a passenger (active tasks). As a result, under "do nothing," drivers perceived time as moving slowly and felt higher stress and drowsiness. Meanwhile, under the passive and active tasks, they perceived time as moving quickly and both the passive and active tasks showed stress reduction effects. Thus, it is necessary to develop driving support technology focusing on psychological, behavioral, and motivational aspects of drivers.

There are three methods for deriving a solution for a problem with which a person is facing, which are 1) retrieval of an existing solution from his/her own memory or from available external resources including human resources, digital resources, and so on, 2) clarifying the constraints to meet and discovering a solution that should satisfy them by exploring the problem space, or 3) deriving a solution by applying inference rules successively until the goal state is achieved. This paper describes the distinctive cognitive processes that respective methods should use when deriving a solution. On the assumption that the ultimately needed problem solving skill would be the one which makes a person solve any problem by himself or herself without reliance on any external resources other than himself/herself, i.e., adaptive problem solving, this paper discusses the implications of the respective methods of problem solving to acquiring the required problem solving skill.

Immersive Virtual Environments are distinct from other types of multimedia learning environments. But, if immersion defined the subjective impression that one is participating in a comprehensive and a realistic experience, immersive-ness is generally defined only from a systemic point of view (e.g., capacity to track users' movements, facial expressions and gestures, quality of appearance, combination of multi-sensory information, design of the virtual world). Moreover, nowadays, it does not exist a robust theoretical framework to describe and to predict immersive-ness from a user-point of view. So this paper is aiming to assume that (a) immersive-ness should be defined from a cognitive user point of view, and that (b) the cognitive architecture called MHP/RT (for Model Human Processor with Realtime Constraints) is relevant to understand and to predict immersive-ness. After a presentation of the MHP/RT model and the distributed memory system related to conscious and unconscious processes, we present the conditions necessary to produce an "immersive experience" for the user, and a case study is described as an example. Theoretical and methodological perspectives are discussed.

Reality is the true situation which consists of a set of things that are actually experienced. Human beings live in the environment filled with artifacts, part of which is real and the rest is virtual. The purpose of this paper is to show how the perceptual-cognitive-motor processes along with the memory process of human beings result in memorable experiences. Since memory system is cumulative, any external stimuli that do not resonate with the existing memory are not real but virtual. However, repetitive experiences should strengthen its memory trace along with its associated memory which in turn makes the virtual change to the real. It is argued from the dual processing perspective necessary conditions for the memory system to work to make the virtual to the real by drawing preliminary results from a study that showed how omnidirectional movies in virtual reality augmented with audio-guide made the experience memorable by timely synchronization and integration of multi-modal information.