Conversations between two peers exhibit a large amount of emotional content that dynamically creates an emotional climate (EC) during the conversation. The recognition of this EC using artificial intelligence (AI), gives an idea of how the conversation is emotionally interpreted by both interlocutors and external parties. This paper presents a new method for EC detection called DeepBispec that is based on deep bispectral processing of conversational speech. The latter is segmented based on emotional labels and subjected to windowed bispectral analysis. The calculated 2D-bispectrums are inputted as colored images to a convolutional neural network (CNN). The latter detects and extracts features from the bispectrum images, that are then fused with affect dynamics (AD) to classify arousal (A) and valence (V) into (low/high) classes. Extensive experiments on the IEMOCAP dataset with 2D emotions (i.e., A and V) show that DeepBispec outperforms previous state-of-the-art methods, achieving an accuracy of 0.826A/0.749V, sensitivity of 0.898A/0.774V, and area under the curve (AUC) of 0.845A/0.824V. The findings reveal the effectiveness of DeepBispec in detecting the emotional tone of conversations between peers, providing a deeper understanding of the emotional dynamics at play in social interactions.
Gait, or the unique way every human walks is widely recognised as an important indicator of overall health and predictor of fall risks. Recently, machine learning based classifiers have been acknowledged and adopted to monitor personal safety specially to detect or predict falls as health and safety anomalies among the older population. It is well known that other animals and birds, including chickens can also suffer from health problems that frequently manifest themselves as gait disorders, one of which is known as lameness. Comparing broiler lameness detection with human fall prediction and detection is meaningful for many reasons, one possible argument being that there are very few bipedal animal models, of which poultry is one. In our multidisciplinary collaborative search for a possible AI-based solution to detecting early lameness among broiler chickens, we discovered formidable challenges in applying anomaly detection techniques. However, prior to this work we had already gained valuable experience in gait anomaly detection for humans. This extension of our experience from humans to broilers has revealed new challenges in anomaly detection that are also relevant to human gait analysis, particularly in monitoring gait to enhance fall prediction and detection through assistive technologies.
Predicting and understanding the changes in cognitive performance, especially after a longitudinal intervention, is a fundamental goal in neuroscience. Longitudinal brain stimulation-based interventions like transcranial direct current stimulation (tDCS) induce short-term changes in the resting membrane potential and influence cognitive processes. However, very little research has been conducted on predicting these changes in cognitive performance post-intervention. In this research, we intend to address this gap in the literature by employing different EEG-based functional connectivity analyses and machine learning algorithms to predict changes in cognitive performance in a complex multitasking task. Forty subjects were divided into experimental and active-control conditions. On Day 1, all subjects executed a multitasking task with simultaneous 32-channel EEG being acquired. From Day 2 to Day 7, subjects in the experimental condition undertook 15 minutes of 2mA anodal tDCS stimulation during task training. Subjects in the active-control condition undertook 15 minutes of sham stimulation during task training. On Day 10, all subjects again executed the multitasking task with EEG acquisition. Source-level functional connectivity metrics, namely phase lag index and directed transfer function, were extracted from the EEG data on Day 1 and Day 10. Various machine learning models were employed to predict changes in cognitive performance. Results revealed that the multi-layer perceptron and directed transfer function recorded a cross-validation training RMSE of 5.11% and a test RMSE of 4.97%. We discuss the implications of our results in developing real-time cognitive state assessors for accurately predicting cognitive performance in dynamic and complex tasks post-tDCS intervention
A methodology based on deep recurrent models for maritime surveillance, over publicly available Automatic Identification System (AIS) data, is presented in this paper. The setup employs a deep Recurrent Neural Network (RNN)-based model, for encoding and reconstructing the observed ships’ motion patterns. Our approach is based on a thresholding mechanism, over the calculated errors between observed and reconstructed motion patterns of maritime vessels. Specifically, a deep-learning framework, i.e. an encoder-decoder architecture, is trained using the observed motion patterns, enabling the models to learn and predict the expected trajectory, which will be compared to the effective ones. Our models, particularly the bidirectional GRU with recurrent dropouts, showcased superior performance in capturing the temporal dynamics of maritime data, illustrating the potential of deep learning to enhance maritime surveillance capabilities. Our work lays a solid foundation for future research in this domain, highlighting a path toward improved maritime safety through the innovative application of technology.
The environmental hazards and climate change effects causes serious problems in land and coastal areas. A solution to this problem can be the periodic monitoring over critical areas, like coastal region with heavy industrial activity (i.e., ship-buildings) or areas where a disaster (i.e., oil-spill) has occurred. Today there are several Earth and non-Earth Observation data available from several data providers. These data are huge in size and usually it is needed to combine several data from multiple sources (i.e., data with format differences) for a more effective evaluation. For addressing these issues, this work proposes the Ocean-DC framework as a solution in data harmonization and homogenization. A strong advantage of this Data Cube implementation is the generation of a single NetCDF product that contains Earth Observation data of several data types (i.e., Landsat-8 and Sentinel-2). To evaluate the effectiveness and efficiency of the Ocean-DC implementation, it is examined a case study of an oil-spill in Saronic gulf in September of 2017. The generated 4D Data Cube considers both Landsat-8,9 and Sentinel-2 products for a time-series analysis, before, during, and after the oil-spill event. The Ocean-DC framework successfully generated a NetCDF product, containing all the necessary remote sensing products for monitoring the oil-spill disaster in the Saronic gulf.
Recent advancements in Human-Robot Collaboration (HRC) have brought to light the significance of ethical, psychological, and attitudinal factors in advanced work and industrial settings, whereby collaborative robots assist humans in work tasks. In these environments, individual factors, attitudes, and trust beliefs of human workers towards robots have a direct impact on the perceived efficiency and safety of HRCs, contributing to worker well-being in the workplace. However, most of the existing research on these topics has been concentrated on social robots and much less on industrial ones. This study aims to fill this gap by exploring the relationships between Negative Attitudes toward robots (NARS) and Trust in industrial HRC. Results demonstrated how, while the overall correlation between NARS and Trust was non-significant, unexpected trends also arose. Gender-dependent dynamics added complexity, with women exhibiting stronger correlations between emotional attitudes and trust. Men, on the other hand, demonstrated a link between stronger NARS and enhanced trust, particularly in robot motion speed perceptions. These intricate findings emphasize the need for tailored design considerations in cobot development, acknowledging the nuanced interplay between dispositional attitudes and trust in shaping human perceptions of robotic technologies in practical scenarios.
This work presents an extension for a coffee-machine that is intended to facilitate its use by people with disabilities. For this purpose, a control method was developed using three wireless buttons and a user interface that allows the selection of several coffee specialties. This selection is translated by a Python script into stepper motor movements fixed to the coffee-machine. With this setup, it is possible to incorporate multiple input modalities such as eye tracking and voice control. Detailed instructions can be found in [1].
Type 2 diabetes mellitus (T2DM) presents a multifaceted challenge, often exacerbated by sedentary lifestyles, poor dietary choices, and heightened stress levels. Addressing these complexities, SugarControl offers a comprehensive solution by empowering users to monitor vital signs, blood glucose levels, stress indicators, dietary habits, and physical activity. By integrating guided yoga, meditation, and chanting practices, the app advocates for a holistic approach to diabetes management. Notably, in a rigorous evaluation involving 287 diabetic, 265 non-diabetic individuals, and 33 diverse app users, SugarControl consistently received ratings of >=4/5 in 70% of cases, affirming its effectiveness in key areas like performance, motivation, stress reduction, goal attainment, and data accuracy. Moreover, over 80% of critics rated these aspects >=4/5, underscoring the app’s ability to inspire healthy behaviors and alleviate stress. Beyond its efficacy, demographic insights reveal higher vulnerability among adults and older individuals compared to teenagers, alongside gender disparities, with males exhibiting a higher prevalence at 28.8% compared to females at 23% in a sample size of 552 patients. Additionally, findings indicate that 44.6% of diabetic individuals lead sedentary lifestyles, emphasizing the critical role of interventions like SugarControl in promoting physical activity. Furthermore, despite dietary preferences, over 67% of diabetic individuals continue consuming sugar-rich foods, highlighting the imperative for motivational and awareness interventions to facilitate healthier choices. Thus, SugarControl meets the urgent need of educating individuals and helping them make proper lifestyle choices to manage diabetes.
The use of Inertial Measurement Units (IMUs) as sensors in wearable devices for human movement monitoring helps the development of remote rehabilitation systems. Pathologies of musculoskeletal system can affect the execution of the basic activities of daily living (ADL), hence in rehabilitation treatments it is important to also evaluate functional gestures related to ADL execution. In this work, we assess the capability of an IMU-based device to detect abnormalities in motor patterns of the shoulder joint during the execution of functional movements. Results from a preliminary assessment suggest good possibilities of detecting differences in motor patterns. The final direction of this work should be to integrate the findings in a telerehabilitation system as an indicator of functionality of the shoulder joints.
User Persona is a structured Interaction Design method for representing a group of users as conceptual models in text and pictorial formats. The regular user Persona descriptions are limited for designing Digital Behaviour Change Interventions for Health. First, they lack the specification of behaviour change goals and target behaviours needed to design behaviour change Interventions. Second, the regular user Personas are mostly static, and they represent only the user's current state and do not address the users' dynamic, long-term behavioural change needs. Third, the user Persona does not help estimate the possible behavioural patterns of the users in the specific context of behaviour change, considering various determinants that influence the users' behaviour. The current work proposes enriching the regular user Persona descriptions with the behaviour change theory. The COM-B Model (Capability, Opportunity, Motivation - Behaviour) and Transtheoretical Model of Change were chosen to enrich user Persona descriptions with behaviour change theory. The current study included the first expert-based evaluation of the theory-enriched Persona description with an Interaction Design expert. The usefulness and overall approach of the enriching Persona descriptions were positively assessed, and future work on making the approach more explicit and precise is recommended.
This paper explores an innovative approach to enhance the interaction of visitors with the cultural heritage embedded in the shepherd settlements, known as "Mitata" nestled within the captivating landscapes of the Psiloritis Mountains in Crete. These unique settlements known for their architectural uniqueness, are repositories of rich cultural traditions and practices that have endured over generations. Our research introduces a novel method for cultural immersion through the integration of geolocated points of interest accessible via mobile devices and augmented reality (AR). The proposed system aims to facilitate a dynamic and engaging experience for visitors, allowing them to navigate the Psiloritis Mountains with their mobile phones, unlocking content linked to the historical and cultural significance of Mitata. Through AR, users can experience their historical evolution, explore traditional agricultural practices and art inspired by these shepherd settlements and gain insights into the daily lives of the communities inhabiting this rugged terrain. By blending modern technology with cultural heritage, our approach seeks to bridge the gap between past and present, fostering a deeper understanding of the symbiotic relationship between the landscape and its inhabitants. This research highlights the potential of augmented reality in cultural heritage preservation and underscores its role in promoting sustainable tourism.
The potential of LLMs to generate context-specific content for psychiatric patients could possibly be used to support treatment. Patients diagnosed with Severe Mental Illnesses face a significant challenge in the realm of goal setting. Caregivers work closely with patients with SMI to establish treatment goals. However, most goals are often immeasurable, hindering their integration into mHealth. This is a missed opportunity since mHealth has the potential to aid healthcare professionals in tracking a patient’s progress and help motivate patients to work on their goals using behavior change concepts like personalization and gamification. Recognizing the time-consuming nature of creating measurable goals. This study validates an LLM-powered goal system aiming to provide a time-efficient way of goal creation for patients with SMI, on the quality of the goals it generates in collaboration with caregivers.
The paper proposes a method to conduct a memorability test to be applied to TactCube, a tangible and tactile User Interface relying solely on the sense of touch. Two Research questions are given, along with the description of the expected experiment structured into two phases. At the end, some metrics are introduced, whose aim is to assign the achieved memorability rate.
Now that the technology is more widely available than ever before, protecting information assets is paramount, especially in the face of escalating cyber threats. The healthcare sector has experienced a significant increase in cyber-attacks, particularly those affecting patient data and business continuity. This trend, exacerbated by the widespread adoption of Internet of Things (IoT) devices, poses significant challenges to securing healthcare infrastructure, given its critical role and unique operational constraints. This paper presents the SEPTON Horizon Europe project, a comprehensive cybersecurity toolkit designed to improve the security of networked medical devices in hospitals and healthcare centres. The mechanisms, techniques and methodologies used in the development of the SEPTON toolkit are presented. The scope and level of application of each component is briefly described, and how each can enhance the overall security of the medical system flow. Finally, a description of the use case with a wearable medical device and the tools of the SEPTON toolkit that will be integrated and tested in it is given.
This paper presents an initial investigation of a speech-based human-robot interaction system for locating items in a warehouse environment. The system uses a 2D SLAM map and visual servoing with deep learning-based barcode recognition to identify and locate items based on user speech commands. The system was tested with and without item location in the SLAM map and achieved a 100% success rate in identifying and localizing items. The average speech processing time was recorded at 9.28 seconds, and the system demonstrated a best-case timing of 15.46 seconds and a worst-case timing of 3 minutes for identifying items on different tables. The proposed system has the potential to improve the employment opportunities and experiences of blind or visually impaired workers in the warehouse industry. Future work will focus on testing the system in real-world environments and improving its performance in cluttered and dynamic settings.
Serious Games (SGs) offer significant potential in healthcare for disease assessment and intervention, enhancing patients’ quality of life. Exercise SGs, or Exergames, promote physical activity and mobility in a gamified environment, encouraging tailored exercises based on individual abilities and needs. Here, within the Horizon Europe iPROLEPSIS project that targets Psoriatic Arthritis (PsA), we describe the design process of Exergames to address PsA symptoms, including morning stiffness, impaired finger/hand and wrist functionality, and motor asymmetry. From a methodological point of view, the storyboarding technique was adopted to facilitate the visualization of the proposed Exergame scenarios, including the graphical user interface, interactive game elements, narratives, characters, and contexts related to the PsA condition. This approach has been implemented within an agile game development methodology, which includes gathering feedback early on to iteratively refine both the game ideas and prototype. Informed by the results of the game design via two agile sessions incorporating 25 participants (PsA patients, clinicians, technology developers, facilitators/observers), the current efforts are focused on developing the game prototypes and corresponding mock-ups, preparing them for upcoming clinical validation trials. The Exergames development lays the groundwork for an innovative framework to improve gross and fine motor symptoms in PsA patients. This framework could inform healthcare providers and policymakers about its potential inclusion in routine PsA management.
Communication challenges faced by individuals with speech impairments present a unique set of difficulties, often hindering effective interaction. This research paper centers on addressing these challenges by employing ChatGPT, a sophisticated large language model (LLM) developed by OpenAI, within the framework of Human-Computer Interaction (HCI). The study investigates the intricate landscape of speech impairments, emphasizing the inherent complexities in vocal expression. Our work highlights the pivotal role of ChatGPT in offering a tailored and adaptable communication solution.The paper highlights the contributions to HCI principles and assistive technologies, showcasing the innovative integration of ChatGPT. Emphasizing interdisciplinary collaboration, the study positions itself at the forefront of leveraging large language models to provide a comprehensive theoretical framework tailored to the unique needs of individuals with speech impairments.
TimeML serves as a foundational annotation scheme for capturing temporal information within textual data. Despite the availability of several gold-standard TimeML corpora, the adherence to TimeML annotation rules and principles of temporal reasoning remains uncertain. This paper proposes a fully automatic sanity check algorithm for TimeML corpus validation and presents a comprehensive evaluation of four mainstream TimeML corpora. Through this investigation, numerous previously unrecognized issues were identified, potentially impacting downstream applications such as graphs and timelines. The analysis revealed a total of 682 errors across the evaluated corpora, encompassing TimeML compatibility errors, violations of TimeML annotation guidelines, temporal reasoning inaccuracies, annotation inconsistencies, and redundancy. These findings underscore the suboptimal nature of current gold-standard TimeML corpora, emphasizing the necessity for corpus validation procedures prior to their release.
The shift towards a more human-centric manufacturing approach in Industry 5.0 emphasizes the integration of technologies that augment rather than replace human capabilities, highlighting the role of collaborative robots (cobots). These cobots, designed to work closely with human operators, bring adaptability and efficiency to the manufacturing floor, adjusting to various tasks and production needs. This integration, while promising, introduces challenges, especially in terms of human adaptation and learning in dynamic work settings. To date, research has primarily focused on the technological advancement of cobots, often overlooking the human component in this collaborative equation. Our study seeks to bridge this gap by employing pupillometry to explore learning effects within human-robot collaboration (HRC), specifically examining human adaptation to complex and extended tasks reflective of industrial environments. Through a multifactorial design involving 19 participants engaged in three trials repeated for two task difficulty levels, the research analyzes performance metrics along with changes in pupil diameter. The results discovered that repetitive task execution is related to decreased operation time and pupil diameter, suggesting reduced cognitive load levels. These findings imply the potential utility of pupillometry as an indicator of human adaptation to complex task execution, promoting further investigation into physiological measures to optimize cobot integration into the workplace.
The privacy of personal data is paramount in the realm of assisted living and digital healthcare. Federated Learning (FL), with its decentralised model training approach, has emerged as a compelling solution to reconcile the need for personalised models with the requirement to protect sensitive personal information. By allowing model training to occur locally on user devices without centralising raw data, FL is intended to strike a balance between personalisation and privacy. While the potential benefits of FL in assisted living and digital healthcare are substantial, practical implementation poses significant challenges. One of them is the non-Independently and Identically Distributed (non-IID) nature of personal data. Unlike centralised datasets, non-IID data exhibits inherent variability across different individuals, as well as their surrounding contexts. Unfortunately, many research approaches in this domain often overlook the nuances of non-IID data, potentially leading to models that lack robust generalisation across diverse healthcare scenarios. To highlight the importance of this challenge, in this paper, we report on our hands-on experience of building a FL system for drowsiness detection using non-IID data. We compare this federated setup with a traditional, centralised approach to model training by identifying and discussing the associated challenges from multiple perspectives, as well as possible solutions and recommendations for further research.
