The presented research focuses on Hand Gesture Recognition (HGR) utilizing Surface-Electromyogram (sEMG) signals. This is due to its unique potential for decoding wearable data to interpret human intent for immersion in Mixed Reality (MR) environments. The existing solutions so far rely on complicated and heavy-weighted Deep Neural Networks (DNNs), which have restricted practical application in low-power and resource-constrained wearable systems. In this work, we propose a light-weight hybrid architecture ( $$\text {HDCAM}$$ ) based on Convolutional Neural Network (CNN) and attention mechanism to effectively extract local and global representations of the input. The proposed $$\text {HDCAM}$$ model with 58, 441 parameters reached a new state-of-the-art (SOTA) performance with $$83.54\%$$ and $$82.86\%$$ accuracy on window sizes of 300 ms and 200 ms for classifying 17 hand gestures. The number of parameters to train the proposed $$\text {HDCAM}$$ architecture is $$18.87 \times $$ less than its previous SOTA counterpart. Furthermore, the model is trained based on a hybrid loss function consisting of two-fold: (i) Cross Entropy (CE) loss which focuses on identifying the helpful features to perform the classification objective, and (ii) Supervised Contrastive (SC) loss which assists to learn more robust and generic features by minimizing the ratio of intra-class to inter-class similarity.
Ultrasound (US) is the most widely used medical imaging modality due to its low cost, portability, real time imaging ability and use of non-ionizing radiation. However, unlike other imaging modalities such as CT or MRI, it is a heavily operator dependent, requiring trained expertise to leverage these benefits. Recently there has been an explosion of interest in AI across the medical community and many are turning to the growing trend of deep learning (DL) models to assist in diagnosis. However, due to possible differences in training and deployment, model performance suffers which can lead to misdiagnosis and operator hesitancy. This issue is known as dataset shift. Two aims to address dataset shift were proposed. The first was to quantify how US operator skill and hardware affects acquired images. The second was to use this skill quantification method to screen and match data to deep learning models to improve performance. A CAE Healthcare BLUE phantom with mock lesions was scanned by three operators using three different US systems (Siemens S3000, Clarius L15, and Ultrasonix SonixTouch) producing 39013 images. DL models were trained on a specific set to classify the presence of a simulated tumour and tested with data from differing sets. Principle Component Analysis (PCA) for dimension reduction was applied, then K-Means clustering was used to separate images generated by operator and hardware into clusters. This clustering algorithm was then used to screen incoming images during deployment to best match input to an appropriate DL model which is trained specifically to classify that type of operator or hardware. Results showed a noticeable difference when models were given data from differing datasets with the largest accuracy drop being 81.26% to 31.26%. Overall, operator differences more significantly affected DL model performance. Clustering models had much higher success separating hardware data compared to operator data. The proposed method reflects this result with a much higher accuracy across the hardware test set compared to the operator data.
Fast-paced and ever-growing advances in Artificial Intelligence (AI) and Deep Neural Network (DNN) models have initiated works on autonomous monitoring/screening systems to assess individuals’ cognitive state. In conventional cognitive assessment systems, a physician evaluates the mental abilities of the brain by rating the patient’s numerical, verbal, and logical responses. Development of an autonomous cognitive assessment system that assist the physician is both of significant importance and a critically challenging task. As a first step towards achieving this objective, in the paper an Automated Virtual Cognitive Assessment ( $$\text {AVCA}$$ ) framework is proposed that integrates Natural Language Processing (NLP) and hand gesture recognition techniques. The proposed $$\text {AVCA}$$ framework provides individual scores in the seven major cognitive domains, i.e., orientation, attention, language, contractual ability, memory, calculation, and reasoning. More specifically, the $$\text {AVCA}$$ framework is an autonomous cognitive assessment system that receives audio and video signals in a real-time fashion, and performs semantic and synthetic analysis using NLP techniques and DNN models. Real-time video processing engines of the $$\text {AVCA}$$ monitors hand motions and facilitate simpler engagement for visual evaluation. Additionally, we propose an efficient model to facilitate human-machine interactions from speech recognition to text classification. In particular, an unsupervised contrastive learning framework is proposed using Bidirectional Encoder Representations from Transformers (BERT) that outperforms its state-of-the-art unsupervised counterparts achieving an average of 77.84% Sparsman’s correlation on standard Semantic Textual Similarity (STS) tasks.
The objective of this paper is to formalize the stress contagion protocols in first responder teams which respond to life-threatening crises on a daily basis. Such teams include both human and autonomous machine teammates. While the stress contagion is an attribute of human relations, collaboration between the humans under stress and, and the stressed humans and machines, impacts the mission performance. Thus, it is important to model this process and use the modeling to train the first responder teams. This paper proposes a framework for modeling the human-machine communication protocols using the advances in computational multivalued logic. We also separate the formalization of the protocol from the choice of an appropriate model. This separation allows to mitigate the constraints posed by the limited real-world initial data of absence of thereof. The proposed formalization based on multivalued logic is well-supported by the computational and benchmarking tools.
