In the expanding realm of sensor-based applications, the reliance on time-series data has surged, posing challenges in explaining the decisions of complex black-box time-series models. Existing Explainable Artificial Intelligence (XAI) approaches such as SBXAI, MCXAI and TS-MULE offer insights into these models but face limitations in generating multiple explanations, exploring time-series-specific characteristics, optimizing found cognitive blocks, and setting appropriate hyperparameters. Addressing these challenges, we introduce an EXplainable artificial intelligence method targeting Time-series model based on Evolutionary Algorithm (ExTea). ExTea conceptualizes explanations as evolving individuals and employs an innovative pyramidal structure for optimizing potential explanations, categorized into newborn, tested, and elite stages. This approach incorporates time-series characteristics into the fitness function of individual evaluation, thereby enhancing the overall explanatory power. Extensive experiments on six benchmark datasets with four target models demonstrate that the performance of ExTea significantly exceeds the state-of-the-art time-series XAI algorithms, SBXAI and MCXAI.

Passive Haptic Learning (PHL) is the process of learning to perform a certain task without paying attention to it. In this paper, we invited 24 participants (evenly divided into 3 groups), and taught them a simple rhythm through the use of haptic, auditory and haptic & auditory signals. The subjects were then tested to reproduce the rhythms on a keyboard and ukulele. We inspected and analyzed the data from two aspects: duration and timing. We found that the difference in instruments and learning groups will be of significance when looking at the duration of notes and pauses. The participants provided both haptic and auditory signals during the learning phase achieved the best performance on duration of notes and pauses. Regarding the timing of the notes within the rhythm.

Deep learning has proven to be an effective approach in the field of Human activity recognition (HAR), outperforming other architectures that require manual feature engineering. Despite recent advancements, challenges inherent to HAR data, such as noisy data, intra-class variability and inter-class similarity, remain. To address these challenges, we propose an ensemble method, called randomHAR. The general idea behind randomHAR is training a series of deep learning models with the same architecture on randomly selected sensor data from the given dataset. Besides, an agent is trained with the reinforcement learning algorithm to identify the optimal subset of the trained models that are utilized for runtime prediction. In contrast to existing work, this approach optimizes the ensemble process rather than the architecture of the constituent models. To assess the performance of the approach, we compare it against two HAR algorithms, including the current state of the art, on six HAR benchmark datasets. The result of the experiment demonstrates that the proposed approach outperforms the state-of-the-art method, ensembleLSTM.

Passive Haptic Learning (PHL) is a method by which users are able to learn motor skills without paying active attention. In past research, vibration is widely applied in PHL as the signal delivered on the participant's skin. The human somatosensory system provides not only discriminative input (the perception of pressure, vibration, slip, and texture, etc.) to the brain but also an affective input (sliding, tapping and stroking, etc.). The former is often described as being mediated by low-threshold mechanosensitive (LTM) units with rapidly conducting large myelinated (Aᵬ) afferents, while the latter is mediated by a class of LTM afferents called C-tactile afferents (CTs). We investigated whether different tactile sensations (tapping, light stroking, and vibration) influence the learning effect of PHL in this work. We built three wearable systems corresponding to the three sensations respectively. 17 participants were invited to learn to play three different note sequences passively via three different systems. The subjects were then tested on their remembered note sequences after each learning session. Our results indicate that the sensations of tapping or stroking are as effective as the vibration system in passive haptic learning of piano songs, providing viable alternatives to the vibration sensations that have been used so far. We also found that participants on average made up to 1.06 errors less when using affective inputs, namely tapping or stroking. As the first work exploring the differences in multiple types of tactile sensations in PHL, we offer our design to the readers and hope they may employ our works for further research of PHL.

The vulnerability of the high-performance machine learning models implies a security risk in applications with real-world consequences. Research on adversarial attacks is beneficial in guiding the development of machine learning models on the one hand and finding targeted defenses on the other. However, most of the adversarial attacks today leverage the gradient or logit information from the models to generate adversarial perturbation. Works in the more realistic domain: decision-based attacks, which generate adversarial perturbation solely based on observing the output label of the targeted model, are still relatively rare and mostly use gradient-estimation strategies. In this work, we propose a pixel-wise decision-based attack algorithm that finds a distribution of adversarial perturbation through a reinforcement learning algorithm. We call this method Decision-based Black-box Attack with Reinforcement learning (DBAR). Experiments show that the proposed approach outperforms state-of-the-art decision-based attacks with a higher attack success rate and greater transferability.

The performance of machine learning models depends heavily on the feature space and feature engineering. Although neural networks have made significant progress in learning latent feature spaces from data, compositional feature engineering through nested feature transformations can reduce model complexity and can be particularly desirable for interpretability. To find suitable transformations automatically, state-of-the-art methods model the feature transformation space by graph structures and use heuristics such as $$\epsilon $$ -greedy to search for them. Such search strategies tend to become less efficient over time because they do not consider the sequential information of the candidate sequences and cannot dynamically adjust the heuristic strategy. To address these shortcomings, we propose a reinforcement learning-based automatic feature engineering method, which we call Monte Carlo tree search Automatic Feature Engineering (mCAFE). We employ a surrogate model that can capture the sequential information contained in the transformation sequence and thus can dynamically adjust the exploration strategy. It balances exploration and exploitation by Thompson sampling and uses a Long Short Term Memory (LSTM) based surrogate model to estimate sequences of promising transformations. In our experiments, mCAFE outperformed state-of-the-art automatic feature engineering methods on most common benchmark datasets.

The skin, as the largest organ distributed all over the human body, offers excellent opportunities for different kinds of input stimuli. However, most of the haptic devices can only render single sensations or they need to combine multiple complex components for generating multiple sensations. We present “DragTapVib” in this paper, a novel, ultra-low-cost, wearable actuator that can reliably provide dragging, tapping, and vibrating sensations to the user. Our actuator is fully electromagnetically-actuated with a moving tactor that can render three haptic feedbacks through systematically controlling the current inside the flexible PCBs. The actuator can be arranged with varying parts of the body which enriches the potentials to implement promising application scenarios including delivering the notification and providing immersive haptic feedback either in virtual reality or in gameplay. A prototypical technical evaluation demonstrated the mechanical properties of our actuator. We quantitatively conducted a series of psychophysical user studies (N= 12) to reveal the feasibility of our prototype. The overall absolute identification study for distinguishing three sensations accuracy at two body locations reached up to 97.2%.

This work contributes an investigation of on-skin electromagnetic actuator for indicating directions through tactile cues. In the study (N = 16), the actuator was tested on three locations on participants' arms. We compared the perception accuracy on these locations and draw the conclusion that the participants could identify the stimulus clearly with the actuator begin placed on the underside of the wrist best. Taken together, our result demonstrates that our work would be used as an additional stimulus combined with other haptic interaction to deliver rich information to the users in potential.

To this day, a variety of approaches for providing local interpretability of black-box machine learning models have been introduced. Unfortunately, all of these methods suffer from one or more of the following deficiencies: They are either difficult to understand themselves, they work on a per-feature basis and ignore the dependencies between features and/or they only focus on those features asserting the decision made by the model. To address these points, this work introduces a reinforcement learning-based approach called Monte Carlo tree search for eXplainable Artificial Intelligent (McXai) to explain the decisions of any black-box classification model (classifier). Our method leverages Monte Carlo tree search and models the process of generating explanations as two games. In one game, the reward is maximized by finding feature sets that support the decision of the classifier, while in the second game, finding feature sets leading to alternative decisions maximizes the reward. The result is a human friendly representation as a tree structure, in which each node represents a set of features to be studied with smaller explanations at the top of the tree. Our experiments show, that the features found by our method are more informative with respect to classifications than those found by classical approaches like LIME and SHAP. Furthermore, by also identifying misleading features, our approach is able to guide towards improved robustness of the black-box model in many situations.

With mixed reality (MR) becoming widely available, it could enhance learning because special equipment like musical instruments or access to instructors will be less of a concern. Furthermore, passive haptic learning systems to learn piano are promising research subjects. We combine both trends of MR and haptic learning to build a piano learning application. Through a study with diverse participants, we evaluate the piano application. The study results show the potentiality of the on-skin actuators and we hope our work could foster the future iterations of the actuators for a fun and effective learning environment.

Earables are a hot topic in wearable research. An increasing number of off-the-shelf devices support collecting sensor data for different use cases. Following this trend, we introduce EarRecorder – a unifying, open-source app that connects to multiple earables (eSense, Cosinuss°, regular earphones). The app can collect data from multiple sensor streams at the same time and lets users label the data during recording. Two user studies allowed us to optimize the usability of the app further and to validate its functionalities. In the future, we hope to include new earables with open APIs and establish a community that advances the app further.

When designing tactile displays on the wrist for spatial cues, it is crucial to keep the natural movement of the body in mind. Depending on the movement of the wrist, different reference frames can influence the output of the wristband. In this paper, we compared two possible reference frames, one where spatial cues are fixed in a wrist-centered frame of reference, and an allocentric frame of reference which fixes spatial cues in the global coordinate system. We compared both conditions in terms of reaction time, achievable accuracy and cognitive load. Our study with 20 participants shows that utilizing the allocentric reference frame reduces cognitive load (avg. 38% reduction) and reaction time (avg. 240ms reduction), with no statistically significant difference in accuracy.

Rapid prototyping and fast manufacturing processes are critical drivers for implementing wearable devices. This paper shows an exemplary method for building flexible, fully elastomeric, vibrotactile electromagnetic actuators based on the Lorentz force law. This paper also introduces the design parameters required for well-functioning actuators and studies the properties of such actuators. The crucial element of actuator is a helical planer coil manufactured from "capillary" silver TPU (Thermoplastic polyurethane), an ultra-stretchable conductor. This paper leverages the novel material to manufacture soft vibration actuators in fewer and simpler steps than previous approaches. Best practice and procedure for building a wearable actuator are reported. We show that dimension of actuators are easily configurable and can be printed in batch-size-one using 3D printing. Actuators can be attached directly to the skin as all the components of FLECTILE are made from biocompatible polymers. Tests on the driving properties have confirmed that the actuator could reach a broad scope of frequency up to 200 Hz with a small voltage (5 V) required. A user study showed that vibrations of the actuator are well perceivable by six study participants under an observing, hovering, and resting condition.