This paper reviews recent advances in assistive devices based on multimodal information fusion control, designed for individuals with motor dysfunction. The prevalence of motor dysfunction is increasingly concerning amidst global population aging. Information fusion technology, widely adopted in rehabilitation, enhances the efficacy and specificity of rehabilitation treatments. This paper introduces the concept of multimodal information fusion control into rehabilitation equipment design. It highlights the advantages and disadvantages of data-level, feature-level, and decision-level fusion, along with commonly employed fusion algorithms. By summarizing and analyzing the current state of research, this paper aims to provide a valuable reference for the further development and optimization of assistive devices for motor dysfunction.

Inner speech, or internal utterance, is fundamental to cognitive activities like language comprehension, planning, and control. However, its psychophysiological mechanisms, particularly the link between verbal and sensorimotor processes, remain poorly understood. Embodied cognition theories suggest that understanding action words involves the reactivation of sensorimotor experiences tied to those words. This study investigates neural differences in the perception, internal utterance, and mental imagery of action words through eventrelated potentials (ERPs). Three experimental series were conducted: (1) word perception and pronunciation, (2) word perception and imagery representation, and (3) sequential perception and utterance of action words. Eight audio stimuli representing distinct actions (e.g., run, stand, up, left) were used. EEG recordings were obtained from 130 participants using a 19-channel Neuro-KM electroencephalograph, with brain activity analyzed via the ‘BrainSys’ software and the ‘Virtual Implanted Electrode’ method. During utterance and representation, similar ERP patterns were observed, differing in amplitude at N140 and N400 latencies across multiple leads (e.g., Cz, Fz, Pz). Significant neural differences were observed, involving regions such as the brainstem, midbrain, right amygdala, cingulate gyrus (BA24), Broca’s area (BA44), parahippocampal gyri, thalamus, and hippocampus. Specific activations during utterance included the left caudate nucleus and right orbitofrontal cortex (BA47).The results highlight distinct neural mechanisms underlying perception, internal utterance, and imagery of action words, supporting embodied cognition theories and offering insights into the verbal-sensorimotor connection during cognitive tasks.

Existing studies in the field of speech disorders do not provide a systematic understanding of the relationship between the bioelectrical activity of the brain and the nature of speech disorders, the characteristics of the processes of speech perception and internal pronunciation. This work is aimed at comparing the activity of the brain during the internal pronunciation of words by a group of people without speech disorders and a group of people with rhotacism. For the first time, an analysis and comparison of event-related potentials (ERP) of the brain in the process of internal pronunciation in people with and without rhotacism was carried out. The electroencephalographic (EEG) study involved 36 people, 18 of them had a speech disorder in the form of rhotacism. The subjects were presented with auditory stimuli (words) spoken by a speaker with standard sound pronunciation. The subject’s task was to mentally repeat the word, maintaining the intonation and pronunciation features, as in external speech. The results obtained in this study using a new method of localization of brain activity demonstrate significant differences in ERP during the mental pronunciation of words between the studied groups of people in a number of brain structures, including cortical and subcortical formations. The group of people with rhotacism is characterized by the presence of a pronounced ERP N200 in more evolutionarily early brain structures, such as the midbrain, medulla oblongata and insular lobe on the left. The group of people without speech disorders is characterized by the presence of pronounced ERP in the following structures: caudate nuclei on the right and left, right globus pallidus, cingulate cortex, striatum, dorsomedial prefrontal cortex, anterior cingulate cortex, field 17 on the right and left, Broca’s area on the right, Wernicke’s area on the right, angular gyrus on the right, anterior prefrontal cortex on the right and left. All differences are obtained with an estimate of 95

Face-name memory is a special kind of memory that includes visual and semantic memory. Existing research suggests that name retrieval is located at the final stage of face recognition, but the exact timing has not been fully investigated. This study used ERPs and a method of spatially localizing brain activity to investigate neural mechanisms underlying face-name memory. Participants were given four tasks: perceiving unfamiliar faces, learning face-name pairs, recalling a name by a face, and recognizing familiar faces but without names. We found that recently learned face-name pairs had the same highly activated brain regions as long-term familiar faces, but the long-term familiar faces exhibit larger amplitudes on the P100 component in the ventral occipital cortex and the N400 component in the thalamus and Gpi. Faces that can be recognized by name elicit a stronger response in the N400 component, particularly in the left hemisphere-dominant thalamus, Gpi, hippocampus, and putamen, compared to faces that are only familiar but not known by name. Results suggest that N400 may represent the retrieval of semantic information related to the name and the depth of retrieval of face-name pairs.

In the current work, the subjects internally pronounced dictated Russian words “caxap” (“sahar”, sugar) and “шaшлык” (“shashlik”, shish kebab) under two conditions: with and without the use of external articulatory interference. Articulatory interference presents itself as a wooden stick and it supposed to be clamped between the teeth horizontally. The study revealed that articulatory pre-setting systematically influences the process of internal word pronouncing, manifested in event-related brain potentials in the brain structures responsible not only for articulatory, but also for sensory components of speech. Myographic sensor detected minimal muscle activity during the study. The analysis of the obtained data was carried out using the author’s method of “virtually implanted electrode”. The results showed that articulatory pre-setting affects the process of internal word pronunciation depending on the influence of artificial interference that hinders the reproduction of the habitual muscle pattern characteristic of the external speech. For the analysis based on the new method of brain activity localization “virtually implanted electrode” ([4], Patent RU 2 785 268 C1, developer A.V. Vartanov), 41 points selected according to the MNI152 atlas, representing the centers of the main structures of brain. As a result, Broca’s and Wernicke’s areas were selected, which demonstrate the most revealing distinctions expressed in event-related brain potentials under the condition of presence or absence of the external articulatory interference that alters the pre-setting of the speech apparatus during the process of internal word pronunciation.

This paper investigates one of the processes of internal representations—specifically, the mental representation of melodies with suppressed verbalization. The objective of this study is to explore the involvement of brain structures during the mental representation of melodies. An fMRI study was conducted with 33 healthy, right-handed participants. Participants were tasked with listening to musical passages accompanied by lyrics in their original form and then mentally reproducing these excerpts with and without the lyrics. The findings suggest that the encoding of musical patterns may exhibit individual variations; however, general trends were identified. The results demonstrate that both cortical and subcortical structures contribute to the internal representation of melodies. Specifically, Wernicke's area and its right hemisphere homologue are instrumental in internal representations, while the caudate nucleus, cingulate gyrus, and superior temporal gyrus play key roles in the inhibition of the verbal component.

The article discusses the problems of identifying the emotional component of inner pronunciation using psychophysiological methods. In the course of preliminary analysis, P200, N400 and LPC were identified, associated with various parameters of prosody regulation during inner pronunciation. An experimental study of inner pronunciation using event-related potentials from EEG was conducted to isolate these components. In addition, a new method of localizing sources of activity using EEG “virtually implanted electrode” was applied in order to study possible sources of the isolated components. The results show the connection of EEG components with various characteristics of prosody (P200 − the beginning of prosody encoding, N400 − the valence of the emotion, LPC − the intensity of the emotion). Based on the results, the participation of various brain structures in the generation of each of the components was also analyzed.

Background. The problem of concept formation in the visual modality remains largely unexplored. There are many studies on the formation of verbal concepts in the learning process. However, it has not yet been fully investigated how concept formation occurs in conditions when both the signified and the signifier (sign) are represented only in the visual modality and what brain mechanisms are involved in this process. Objectives. The aim of the present work was to identify the brain mechanisms of visual concepts formation, on the basis of EEG registration with subsequent localization of the sources of electrical activity. Another task is to evaluate the possibility of actualizing the process of indirect learning in the formation of visual concepts. Study Participants. 26 Russian-speaking subjects without neurological disorders: 10 males and 16 females (aged 18 to 40 years, mean age 22.92 years, SD = 6.38) participated in the study. Methods. Chinese hieroglyphs, unfamiliar to the subjects, were taken as signs. The designated ones were emoticons (schematic faces expressing various emotions). A total of 10 pairs of stimuli sign-designated were presented. A 19-channel electroencephalogram (EEG) was recorded in three successive series: 1) the initial perception of hieroglyphic signs without assigning them a category; 2) categorization of these signs in the process of associative learning, with only 8 designating stimuli-signs directly associated with the designated stimulus, while 2 stimuli-signs were never associated with the stimulus-designator, their meaning was established indirectly; 3) subsequent perception of signs with the meaning already assigned to them. All stimuli were presented to the subjects visually. A new method of localization of brain activity, “Virtually implanted electrode”, developed by A.V. Vartanov (patent RU No. 2 785 268) was applied. Results. The analysis of the subjects' responses showed that during the learning process all signs (including those formed indirectly) were assigned a certain meaning (designated). Differences in event-related potentials (ERP) were found in leads C3 and CZ. Significant differences in ERP as a result of learning were revealed in a number of brain structures. It was found that a number of functional connections between the left area of the secondary visual cortex and the right part of the cerebellum changed significantly as a result of learning. Conclusions. The development of visual categories is ensured by the coordinated work of the right part of the cerebellum, parahippocampal gyrus and primary visual cortex, which is confirmed by the discovered differences in the corresponding ERPs.

In this work we investigate the phenomenon of emotional mirroring using remotely diagnosed dynamic parameters of facial expressions. The research is based on the fact that mirroring is the subconscious adjustment and copying of the dynamics of another person. We considered a reflection of face expression as a reproduction of emotions of one person by another. To obtain this behavior we used an induced cognitive–emotional conflict in the process of telecommunication dialogue. The conflict was initiated by a psychologist or by short videoclips with surprise endings. Since the communication in a telecommunication form limits non-verbal information about the interlocutor with respect to the normal dialogue, we have also investigated the hypothesis of whether the phenomenon of mirroring is detectable in such conditions. We developed a computer program using VGG16-based artificial neural network to mark people’s emotional reactions in video data automatically. The processed material consisted of 24 interview recordings with the participants of both genders and three qualified expert psychologists. We used different types of interviews: interviews based on self-attitude techniques, problematic interviews based on transactional analysis, free reasoning about controversial and topical situations. The communication topics were selected with respect to the age and other indicators of the group of participants. It was found that the parameters of facial expressions of the participant and the experimenter (psychologist) identified by the program strongly correlate with emotions such as happiness, sadness and surprise. Notable negative correlations were found between the parameters of the happiness of participant and fear of psychologist, sad of the participant and happiness of the psychologist, sad of psychologist and surprise of the participant. A direct relationship between sad of participant and fear of psychologist was detected. All of the identified correlations appear both in the situation with and without cognitive–emotional conflict. However, the degree of their manifestation was quite different for these two cases.

Inner speech is one of the most important human cognitive processes. Nevertheless, until now, many aspects of inner speech, particularly the emotional characteristics of inner speech, remain poorly understood. The main objectives of our study are to identify the neural substrate for the emotional (prosodic) dimension of inner speech and brain structures that control the suppression of expression in inner speech. To achieve these goals, a pilot exploratory fMRI study was carried out on 33 people. The subjects listened to pre-recorded phrases or individual words pronounced with different emotional connotations, after which they were internally spoken with the same emotion or with suppression of expression (neutral). The results show that there is an emotional component in inner speech, which is encoded by similar structures as in spoken speech. The unique role of the caudate nuclei in the suppression of expression in the inner speech was also shown.

We analyze a classification problem of mentally pronounced Russian phonemes based on data obtained by means of an electroencephalography device. We describe the data collection method as well as the methods of the obtained data processing. To solve the small sample size problem we present the augmentation techniques that use the time stretching and the white noise adding. Our approach uses an algorithm based on the convolutional neural networks and it is applicable to solving the binary and multiclass classification problems. The conducted experiments allow us to estimate the accuracy of our algorithms and to compare them to the existing algorithms based on the support vector machine.

The contradiction between the conscious and emotional assessment of situations often arises when there is a discrepancy between reality and ideal view. The purpose of this study is to identify a system of remotely diagnosed parameters (by video recording using Skype tools), which can objectively indicate the presence of a cognitive-emotional conflict during a dialogue. The current study has become particularly relevant in the COVID-19 environment, where face-to-face communication is limited, and all interactions must occur in a remote online format. In such an environment, the identification of formal signs indicating latent conflicts in online interactions comes to the fore. To stimulate an outburst and fix the cognitive-emotional conflict remotely, various schemes were used: interviews based on the self-attitude technique, discussions based on the transactional analysis model, free reasoning about situations that were selected by the age, and other parameters of participants, as well as watching video clips with an unexpected end. For automatic marking of people's emotional reactions in video data, a computer program was developed using neural network algorithms based on VGG16.Furthermore, the neural network for emotion recognition was pretrained based on open sources by the P. Ekman FACS model. Finally, video recordings were marked according to the expert assessment results regarding the presence of cognitive-emotional conflict. We conducted 27 interviews with the participation of both genders, including ten high school students (from 12 to 17 years old), a total of 338,075 frames were received. As a result, it was found that the two facial expression parameters identified by the program reliably distinguish the presence of a cognitive-emotional conflict. Furthermore, in all the studied schemes of telecommunication dialogue, the «happy» parameter was significantly more pronounced in a situation of conflict.

This article describes the fMRI study of the processes of internal and external speech. The purpose of this work is to compare the processes of internal and external speech in conjunction with those activated structures. The experiment included 3 series for each type of stimulus (words, syllables, phonemes): simple listening to stimulus material, listening followed by internal pronunciation, and listening followed by pronunciation aloud. The study involved 33 respondents (10 men and 23 women ranging from 20 to 30 years old, mean age = 24 years). The results of study is connected with the identification of zones specifically associated with inner speech. Areas such as BA37, BA38, and cerebellar areas were found to be associated with the type of pronunciation, external or internal. A significant activation of the vACC zone (BA24) was found during external speech, in contrast to inner speech, which is most likely associated with a complex function of social reinforcement of this zone.

We propose a spatial localization method for scalp electroencephalography (EEG). The technique allows reliable and unambiguous identification of the activity for any intracerebral source by its spatial coordinates. Our "virtually implanted electrode" is based on the dynamics and correlation analysis of signals in the EEG leads with the addition of artificially generated data. The generated data helps us accurately model the electrical potential distribution between the studied source and the scalp electrodes. Each modeled intracerebral source is analyzed independently, so the proposed method does not require an estimate of the possible number of sources. The method produces output values that can be interpreted as the "local field" elec-trical activity for the implanted electrode in the corresponding brain point. The signal cleaning procedure makes it possible to exclude the influence from neighboring regions and detect a given brain region's isolated electrical activity. For verification, the method was applied in the empathy study on a sample of 16 people. We compared the background EEG of calm wakefulness with the EEG of empathic behavior for the 33 preselected brain regions.

In this article it was decided to pay attention to the mechanisms of mental articulation of words with semantic meaning and words without meaning. The research is based on scalp EEG registration and application of a new method of source localization “Virtual implanted electrode” for calculation of the analogue of the local field potential in the brain regions specified by its coordinates. The purpose of the study is to compare EPs to the signal for internal utterance (repetition) of words in the native language and words with no sense, pronounced in a non-familiar language (Japanese). In Caput n.Caudati there is a complex of N150 - P200, whereas in Putamen the opposite picture is observed - a complex of P120 - N200 peaks. There is also a significant difference in Gl. pallidus Med., especially on the left side. At the same time, the meaningful words show large amplitude peaks - P100, P120, N150, P220, P300. A strong and significant distinction is found for average latent peaks, in the range of P230 - P350, the amplitude of peaks when uttering words with a semantic load is greater. For left- and right-sided areas of Parietal c. BA7, as well as in the middle part of the cingular cortex (G. Cingulate Med. BA24), the potentials are similar and characterized by the two-component structure P120 - N230, analogous to the potentials in Putamen.

The solution of many clinical problems, including the issues of the origin of pain as a component of individual pathologies, lies in the plane of quantitative objectification of the underlying disease and elucidation of the role of pain syndrome (PS) in it. The analysis of literature data on methods of pain assessment, including quantitative scales and questionnaires, biochemical and instrumental studies, was carried out. Various PS correlates used in fundamental and applied research are considered. The neurophysiological mechanisms of pain formation and specific methods for assessing PS in endometriosis were studied.

In this article, the process of internal pronunciation (covert speech) is associated with the internal speech through an intellectual process such as silent reading. The objective of the research is to compare the EP of visual and auditory perception and internal pronunciation of phonemes and syllables; to classify phonemes, words and syllables from covert speech, according to the EEG-data. Electrical activity was measured in tasks: for visual and auditory perception and pronunciation; for perception and pronunciation of conditioned stimuli. Electrophysiological experiment registrated using 19-channel EEG. Seven phonemes (A, B, F, G, M, R, U) and ten syllables composed of these phonemes (BA, FA, GA, MA, RA, BU, RU, MU, FU, GU) were selected for the experiment. Japanese words constructed using current phonemes were used as conditional stimuli (a trigger for the stimulus's pronunciation). The obtained data analysis was carried out using the statistical programming language R. Results, based on the ANOVA have significant differences for all experimental stages. Pronunciation as a reaction to a conditioned stimulus in the form of Japanese words were compared with two types of of covert speech initialization. The averaged reconstruction was between 63–67

This study is part of a large interdisciplinary project to create objective criteria for building a classification system for a polymorphic schizophrenic disease characterized by disorders of thinking and perception. Within the framework of the project, for the first time in patients with schizophrenia with hallucinatory-delusional syndrome, neuropsychological testing, complete structural (morphometry and tractography) and functional MRI studies involving a detailed clinical study, determination of the patient’s immunological and genetic status are simultaneously performed. On the basis of a comprehensive analysis of MRI data, it is planned to determine the key architecture of the neural networks of the brain producing the disease, to assess the success of disease therapy and to determine the targets of subsequent therapy based on the application of the method of transcranial magnetic stimulation. This paper presents the primary results of work on the study of the difference between the structural connectomes of schizophrenic patients and the norm. The work is of a search and preliminary nature. As a result of the work, the possibility of classifying the disease was shown and the key structural links for it were identified.

Prosocial behavior is progressively being studied because of its enormous role in the changing conditions of the modern world. We study the prosocial behavior by analyzing the brain features mechanisms during the custom setting - empathy game “Stone-Paper-Scissors”. Observed results for the cohort of 55 participants (28 women and 27 men) were obtained from the test questionnaires and EEG. Our approach relates to the field of medicine and neuroscience, in particular, to a method for studying the activity of individual brain structures predefined by their spatial position according to the scalp multichannel EEG. The results of the questionnaires were tabulated and processed by the factor analysis. Six factors were identified that accounted for 41.45