Wearable sensor-based fall detection systems that rely solely on inertial measurement unit (IMU) data are prone to high false-positive rates, particularly for subtle “soft falls”. While incorporating multimodal data can substantially improve detection accuracy, existing multimodal datasets typically depend on specialized, lab-grade IMU sensors and often lack demographic diversity, limiting their applicability to real-world deployment in commodity wearable devices and generalizability across age groups. To bridge this gap, we present SmartFallMM, a multimodal dataset collected using a combination of commodity and specialized hardware: smartwatches, smartphones, Kinect motion cameras, and Meta sensors. The dataset captures synchronized skeletal motion (32 joint positions), accelerometer and gyroscope readings from 51 participants spanning two distinct age groups (20-30 and 60-93), performing 14 activities encompassing both falls and activities of daily living (ADLs). Since multimodal models are too computationally demanding for edge devices, we propose Modality Aware Knowledge Distillation (MAKD), a cross-modal approach that distills multimodal knowledge into an IMU-only model. MAKD improves F1-score by up to 7.85% on SmartFallMM and 8.65% on the UP-Fall dataset over IMUonly baselines, showing the practical value of SmartFallMM and the effectiveness of cross-modal knowledge transfer.

Android apps rely on application programming interfaces (APIs) to access various functionalities of Android devices. These APIs however are regularly updated to incorporate new features while the old APIs get deprecated. Even though the importance of updating deprecated API usages with the recommended replacement APIs has been widely recognized, it is non-trivial to update the deprecated API usages. Therefore, the usages of deprecated APIs linger in Android apps and cause compatibility issues in practice. This paper introduces GUPPY, an automated approach that utilizes large language models (LLMs) to update Android deprecated API usages. By employing carefully crafted Chain-of-Thoughts prompts, GUPPY leverages GPT-4, one of the most powerful LLMs, to update deprecated-API usages, ensuring compatibility in both the old and new API levels. Additionally, GUPPY uses GPT-4 to generate tests, identify incorrect updates, and refine the API usage through an iterative process until the tests pass or a specified limit is reached. Our evaluation, conducted on 360 benchmark API usages from 20 deprecated APIs and an additional 156 deprecated API usages from the latest API levels 33 and 34, demonstrates GUPPY's advantages over the state-of-the-art techniques.

Knowledge distillation (KD) has emerged as a widely adopted approach for cross-modal knowledge transfer, where transferring knowledge from a semantically rich teacher model to a lightweight student model enhances the performance of the student model. However, existing KD approaches rely on logit-based distillation, which suffers from limited logit diversity inherent in binary classification tasks such as fall detection. Moreover, existing KD methods transfer knowledge indiscriminately from both correct and incorrect predictions of the teacher model, which can undermine the reliability of the student model. We propose RightFeatKD, a novel feature-based distillation framework that captures rich semantic knowledge from the intermediate representations of the teacher model using a specialized loss function. Importantly, RightFeatKD focuses more on distilling knowledge from the teacher’s correct predictions, ensuring reliable and meaningful supervision. Experimental evaluations of RightFeatKD with two public fall detection datasets SmartFallMM and Up-Fall respectively demonstrate 4.04

Human Action Recognition (HAR) has recently achieved significant success through the analysis of human behavior using non-visual data (e.g., sensor data) and visual data (e.g., skeleton data). However, sensor-based methods face challenges due to the inherent limitations of sensor data, including the absence of 3D body pose information, high volatility, and vulnerability to noise. Meanwhile, skeleton-based methods, while effective due to their rich spatial and temporal information, are constrained by the stringent requirements for data acquisition and problems like occlusion, limiting their feasibility in real-world outdoor scenarios. Therefore, to solve these challenges, we resort to the cross-modal generation strategy and aim to generate hard-to-collect but information-rich skeleton data conditioned on easy-to-monitor sensor data. In our work, we propose a novel Sensor-to-Skeleton Diffusion Model with Lipschitz Regularization, named SSDL. Specifically, we first design an Angular Variation module and extract angular variation information of joint movements with time information. Subsequently, noise is added to the skeleton key points and angular variation during the forward diffusion process. To address noisy sensor data and improve training stability, we incorporate Lipschitz regularization with the diffusion model's loss to prevent overfitting. We verify the generalizability and effectiveness of our methods on two benchmark multimodal human action datasets: UTD-MHAD, Berkeley-MHAD, and SmartFall-MHAD dataset. Extensive results demonstrate the superiority of leveraging generated skeleton information conditioned on the sensor data for accurate human activity recognition with limited computational demands. Code will be available at https://github.com/nikhiliit/SSDL.

Training fall detection systems is challenging due to the scarcity of real-world fall data, particularly from elderly individuals. To address this, we explore the potential of Large Language Models (LLMs) for generating synthetic fall data. This study evaluates text-to-motion (T2M, SATO, and ParCo) and text-to-text models (GPT4o, GPT4, and Gemini) in simulating realistic fall scenarios. We generate synthetic datasets and integrate them with four real-world baseline datasets to assess their impact on fall detection performance using a Long Short-Term Memory (LSTM) model. Additionally, we compare LLM-generated synthetic data with a diffusion-based method to evaluate their alignment with real accelerometer distributions. Results indicate that dataset characteristics significantly influence the effectiveness of synthetic data, with LLM-generated data performing best in low-frequency settings (e.g., 20 Hz) while showing instability in high-frequency datasets (e.g., 200 Hz). While text-to-motion models produce more realistic biomechanical data than text-to-text models, their impact on fall detection varies. Diffusion-based synthetic data demonstrates the closest alignment to real data but does not consistently enhance model performance. An ablation study further confirms that the effectiveness of synthetic data depends on sensor placement and fall representation. These findings provide insights into optimizing synthetic data generation for fall detection models.

Human Action Recognition (HAR) using wearable sensors has become a topic of great interest in research due to its wide range of applications. However, achieving performance improvements in HAR often involves increasing model sizes, hindering the feasibility of deployment on resource-constrained wearable devices. Spiking Neural Networks (SNNs), inspired by biological neural networks, offer a promising alternative by leveraging the spiking nature of brain neurons and eventdriven activations. SNNs can thus reduce training complexity and power consumption significantly. Nevertheless, SNNs usually face accuracy degradation and require a substantial number of time steps to attain competitive performance when compared to conventional artificial neural networks (ANNs). To address these challenges, we propose a novel Semantic-aware Spiking neural network Self-distillation ((SKD)-K-3) framework to improve the accuracy of wearable-based HAR. First, we replace all activation functions with the Leaky-Integrate-and-Fire (LIF) neurons to construct SNN models. Next, we design a semantic-aware MixUp technique to construct more reliable soft labels for the (SKD)-K-3 process. Furthermore, we introduce a knowledge preview scheme aimed at bolstering the transferability of model knowledge within SNNs. We validate the effectiveness of (SKD)-K-3 through extensive experiments on three HAR datasets. The results indicate its superiority over previous SNN-based approaches, achieving competitive performance enhancements.

The lack of real-world data in clinical fields poses a major obstacle in training effective AI models for diagnostic and preventive tools in medicine. Generative AI has shown promise in increasing data volume and enhancing model training, particularly in computer vision and natural language processing (NLP) domains. However, generating physiological time-series data, a common type in medical AI applications, presents unique challenges due to its inherent complexity and variability. This paper introduces TransConv-DDPM, an enhanced generative AI method for biomechanical and physiological time-series data generation. The model employs a denoising diffusion probabilistic model (DDPM) with U-Net, multi-scale convolution modules, and a transformer layer to capture both global and local temporal dependencies. We evaluated TransConv-DDPM on three diverse datasets, generating both long and short-sequence time-series data. Quantitative comparisons against state-of-the-art methods, TimeGAN and Diffusion-TS, using four performance metrics, demonstrated promising results, particularly on the SmartFallMM and EEG datasets, where it effectively captured the more gradual temporal change patterns between data points. Additionally, a utility test on the SmartFallMM dataset revealed that adding synthetic fall data generated by TransConv-DDPM improved predictive model performance, showing a 13.64% improvement in F1-score and a 14.93% increase in overall accuracy compared to the baseline model trained solely on fall data from the SmartFallMM dataset. These findings highlight the potential of TransConv-DDPM to generate high-quality synthetic data for real-world applications.

Time-series data is widely used across domains but presents challenges due to high dimensionality and noise. This study proposes a hybrid feature selection approach to enhance forecasting accuracy by reducing irrelevant features. We first trained a 1D-CNN-based forecasting model using all features as a baseline. Then, we evaluated four feature selection methods: two traditional (Variance and Dynamic Mode Decomposition) and two XAI-based (SHAP and LIME). To address the limitations of individual methods, we introduced a hybrid approach combining Variance and LIME. Experiments on four diverse datasets, including EEG (seizure forecasting), Pole-balancing (pre-fall detection), Weather (meteorological forecasting), and Electricity (demand prediction), show that the hybrid method consistently outperformed all others. It achieved the highest gains in EEG (+7.34%), Pole-balancing (+9.02%), Weather (+8.62%), and Electricity (+6.72%), demonstrating robust, interpretable, and generalizable performance across tasks.

We provide a retrospective on our article “QoS-Aware Middleware for Web Service Composition” published in the May 2004 issue of IEEE Transactions on Software Engineering. We start by defining the concept of Web service composition, and reviewing key paradigms emerging from early research on this topic. We then explain how the 2004 QoS article laid down a conceptual foundation that spawned a prolific line of research into the estimation and optimization of Quality of Service (QoS) properties of composite Web services. We close with a reflection on how the challenges tackled by the 2004 QoS article and other research on service composition are reemerging in the field of Large Language Model (LLM) agents. We argue that the design of LLM agents raises challenges similar to those that the 2004 QoS article tackled, while bringing in new challenges due to the non-deterministic nature of LLMs and the fact that they consume and produce unstructured data.

AI-Powered Edge Computing is accelerating the integration of the cyber world with the ever-growing list of new physical IoT devices and will fundamentally change and empower the way humans interact with the world. In this paper, we prototyped and analyzed three edge computing architectures for running SmartFall, a real-time fall detection application that uses accelerometer data from the watch, to compare the trade-off in relationship to battery consumption, potential data loss, machine learning model's prediction accuracy, and latency in model inferencing. Our experiments show that running the machine learning prediction on the server using the TensorFlow native model format has achieved the best model accuracy with-out draining the battery power of the smartwatches. However, the optimal selection of the software architecture depends on the intended deployment environment, projected user numbers, users' privacy concerns, and network stability.

Human Activity Recognition (HAR) using wearable sensors has gained significant attention due to its portability and unobtrusiveness. However, the data obtained from wearable sensors are limited to inertial data from predefined locations on the human body. In contrast, skeletal data from motion capture devices, such as the Kinect camera, offer richer information by capturing the whole body dynamics of a human action. Unfortunately, the use of skeletal data is impractical in wearable sensor-based HAR for real-world deployment. Currently, transformer neural networks, known for their self-attention mechanism, have shown effective handling of data from diverse modalities in wearable sensor-based HAR. However, the deployment of multimodal transformer on wearable devices is challenging due to their inherent large model size. We propose a Lightweight HAR Transformer (LightHART) framework that trains an unimodal Inertial Transformer (IT) network by transferring knowledge from a large multimodal transformer using a knowledge distillation approach. We evaluate the proposed framework on three public multimodal human activity datasets and compare the performance of the LightHART student model with various state-of-the-art approaches. Experimental results demonstrate that our LightHART model achieves competitive performance in terms of effectiveness and scalability with a model size of only 1.43 Mb. We are the first to deploy and validate the LightHART fall detection model on a SmartFall App running on a WearOS-compatible smartwatch showcasing its potential in advancing wearable sensor-based HAR research.

Recognition of human activity is crucial in emer-gency healthcare applications. Multi-modal deep learning learning algorithms are gaining attention for Human Activity Recog-nition (HAR) due to their success in various domains. The application of multi-modal learning to HAR continues to present challenges, particularly in addressing noisy data and achieving effective fusion of information from disparate modalities. We propose a new Multimodal-ConvTransformer (CT-HAR) that aims to efficiently extract and fuse complementary spatial and temporal information. Experiments conducted on the public UTD-Mhad and Berkley-Mhad datasets demonstrate significant performance enhancements, with CT-HAR achieving accuracy rates of 89.81 % and 85.69% on these datasets, respectively.

The combination of increased life expectancy and falling birth rates is resulting in an aging population. Wearable Sensor-based Human Activity Recognition (WSHAR) emerges as a promising assistive technology to support the daily lives of older individuals, unlocking vast potential for human-centric applications. However, recent surveys in WSHAR have been limited, focusing either solely on deep learning approaches or on a single sensor modality. In real life, our human interact with the world in a multi-sensory way, where diverse information sources are intricately processed and interpreted to accomplish a complex and unified sensing system. To give machines similar intelligence, multimodal machine learning, which merges data from various sources, has become a popular research area with recent advancements. In this study, we present a comprehensive survey from a novel perspective on how to leverage multimodal learning to WSHAR domain for newcomers and researchers. We begin by presenting the recent sensor modalities as well as deep learning approaches in HAR. Subsequently, we explore the techniques used in present multimodal systems for WSHAR. This includes inter-multimodal systems which utilize sensor modalities from both visual and non-visual systems and intra-multimodal systems that simply take modalities from non-visual systems. After that, we focus on current multimodal learning approaches that have applied to solve some of the challenges existing in WSHAR. Specifically, we make extra efforts by connecting the existing multimodal literature from other domains, such as computer vision and natural language processing, with current WSHAR area. Finally, we identify the corresponding challenges and potential research direction in current WSHAR area for further improvement.

Falls are the second leading cause of unintentional injury deaths worldwide. While numerous wearable fall detection devices incorporating AI models have been developed, none of them are used successfully in a fall detection application running on commodity-based smartwatches in real time. The system misses some falls, and generates an annoying amount of False Positives for practical use. We have investigated and experimented with an LSTM model for fall detection on a smartwatch. Even though the LSTM model has high accuracy during offline testing, the good performance of offline LSTM models cannot be translated to the equivalence of real-time performance. Transformers, on the other hand, can learn long-sequence data and patterns intrinsic to the data due to their self-attention mechanism. This paper compares three variants of LSTM and two variants of Transformer models for learning fall patterns. We trained all models using fall and activity data from three datasets, and the real-time testing of the model was performed using the SmartFall App. Our findings showed that in the offline training, the CNN-LSTM model was better than the Transformer model for all the datasets. However, the Transformer is a preferable choice for deployment in real-time fall detection applications.

In an era where Android dominates the mobile operating system market, it is important to ensure high quality Android app delivery. In this paper, we delve into the essential need for effective Android app testing in a market characterized by diversity and widespread usage and empirically investigate testing practices for Android apps. We conduct an experimental study on 2965 open-source Android apps, examining the quantity and coverage of the tests used for open-source Android app development. We further conduct a survey to gather more insights in testing practices from Android app developers and testers. The results reveal a limited adoption of testing among Android app development, a restricted range of testing tools and frameworks being used, and low code and API coverage in testing. This investigation shows that current Android app testing practices are lacking the use of automated testing tools and embarks on a need for more awareness and adoption of state-of-the-art testing tools and techniques.

Introduction: There is increasing interest in developing mathematical and computational models to forecast adverse events in physiological systems. Examples include falls, the onset of fatal cardiac arrhythmias, and adverse surgical outcomes. However, the dynamics of physiological systems are known to be exceedingly complex and perhaps even chaotic. Since no model can be perfect, it becomes important to understand how forecasting can be improved, especially when training data is limited. An adverse event that can be readily studied in the laboratory is the occurrence of stick falls when humans attempt to balance a stick on their fingertips. Over the last 20 years, this task has been extensively investigated experimentally, and presently detailed mathematical models are available.Methods: Here we use a long short-term memory (LTSM) deep learning network to forecast stick falls. We train this model to forecast stick falls in three ways: 1) using only data generated by the mathematical model (synthetic data), 2) using only stick balancing recordings of stick falls measured using high-speed motion capture measurements (human data), and 3) using transfer learning which combines a model trained using synthetic data plus a small amount of human balancing data.Results: We observe that the LTSM model is much more successful in forecasting a fall using synthetic data than it is in forecasting falls for models trained with limited available human data. However, with transfer learning, i.e., the LTSM model pre-trained with synthetic data and re-trained with a small amount of real human balancing data, the ability to forecast impending falls in human data is vastly improved. Indeed, it becomes possible to correctly forecast 60%–70% of real human stick falls up to 2.35 s in advance.Conclusion: These observations support the use of model-generated data and transfer learning techniques to improve the ability of computational models to forecast adverse physiological events.

Lack of real-world data in clinical fields poses a major obstacle for training deep learning models. Using data augmentation can increase data volume, making the training of deep learning models more effective. This paper aims to investigate different techniques for generating realistic multivariate synthetic fall data, addressing the challenge of limited fall data availability. We experimented with three traditional time series data augmentation techniques, a generative AI approach with diffusion, and extraction of data from public video recordings of older adults falling. We evaluated the effectiveness of the generated data with both an LSTM model trained offline and using the SmartFall App running the LSTM model in real-time. Initial results indicate a 7–10

We present an edge-cloud collaborative personalized fall detection pipeline called P-Fall. A personalized fall detection model requires real-time adaptation of a pre-trained model using real-time feedback data provided by the user. We, herein, highlight the design of the software architecture for a collaborative framework, the smart-watch's UI for the ease of collecting a user's feedback data, and the automation of the personalization process.

Wearable sensor-based Human Action Recognition (HAR) has made significant strides in recent times. However, the accuracy performance of wearable sensor-based HAR is currently still lagging behind that of visual modalities-based systems, such as RGB video and depth data. Although diverse input modalities can provide complementary cues and improve the accuracy performance of HAR, wearable devices can only capture limited kinds of non-visual time series input, such as accelerometers and gyroscopes. This limitation hinders the deployment of multimodal simultaneously using visual and non-visual modality data in parallel on current wearable devices. To address this issue, we propose a novel Physical-aware Cross-modal Adversarial (PCA) framework that utilizes only time-series accelerometer data from four inertial sensors for the wearable sensor-based HAR problem. Specifically, we propose an effective IMU2SKELETON network to produce corresponding synthetic skeleton joints from accelerometer data. Subsequently, we imposed additional constraints on the synthetic skeleton data from a physical perspective, as accelerometer data can be regarded as the second derivative of the skeleton sequence coordinates. After that, the original accelerometer as well as the constrained skeleton sequence were fused together to make the final classification. In this way, when individuals wear wearable devices, the devices can not only capture accelerometer data, but can also generate synthetic skeleton sequences for real-time wearable sensor-based HAR applications that need to be conducted anytime and anywhere. To demonstrate the effectiveness of our proposed PCA framework, we conduct extensive experiments on Berkeley-MHAD, UTD-MHAD, and MMAct datasets. The results confirm that the proposed PCA approach has competitive performance compared to the previous methods on the mono sensor-based HAR classification problem.