Sepsis is a major cause of premature mortality, high healthcare costs, and disability-adjusted life years. Digital interventions such as continuous cardiac monitoring solutions can help to monitor the patient’s status and provide valuable feedback to clinicians to detect early warning signs and provide effective interventions. This paper presents i-CardiAx, a wearable sensor based on low-power high-sensitivity accelerometers that measures vital signs essential for cardiovascular health monitoring, namely, heart rate (HR), blood pressure (BP), and respiratory rate (RR). A dataset has been collected from 10 healthy subjects with the i-CardiAx wearable chest patch to develop low-complexity, lightweight vital sign measurement algorithms and evaluate their performance. The experimental evaluation demonstrates high-performance vital sign measurement for RR (-0.11±0.77 breaths per minute), HR ( 0.82± 2.85 beats per minute), and systolic BP (-0.08 ± 6.245 mm of Hg). The proposed algorithms are embedded on the ARM Cortex-M33 processor supporting Bluetooth Low Energy (BLE). Estimation of HR and RR achieved an inference time of only 4.2 ms and 8.5 ms for BP. Moreover, a multi-channel quantized Temporal Convolutional Neural (TCN) Network has been proposed and trained on the open-source HiRID dataset to have a large number of patients with data for sepsis ground truth. The model has been trained and evaluated using only digitally acquired vital signs as input-data that could be collected by i- CardiAx to detect the onset of Sepsis in a real-time scenario. The TCN has been fully quantized to 8-bit integers and deployed on i-CardiAx.The network showed a median predicted time to sepsis of 8.2 hours with an energy per inference of 1.29mJ. i-CardiAx has a sleep power of 0.152 mW and an averages a power of 0.77 mW for always-on sensing and periodic on-board processing and BLE transmission. With a small 100 mAh battery, the operational longevity of the wearable has been estimated at two weeks (432 hours) for measuring the three cardiovascular parameters (HR, BP and RR) at a granularity of 30 measurements per hour per vital sign, running inference every 30 minutes. Thus, the wearable i-CardiAx system can provide a method to monitor the cardiovascular parameters of patients with energy-efficient, high- sensitivity sensors to provide predictive alerts for life-threatening adverse events of sepsis, over a long period of time.
There’s a significant move towards the adoption of Post-Quantum Cryptography (PQC). While there have been initiatives to transition conventional TCP/IP-based networks to PQC, Quantum Resilient Internet of Things (IoT) networks have not been as widely discussed. Presently, Bluetooth Low Energy (BLE) employs the Elliptic Curve Diffie-Hellman algorithm for Secure Connection (SC) pairing, which is vulnerable to quantum threats. In this study, we introduce a backward-compatible Post-Quantum Key Exchange (PQKE) protocol for BLE, utilizing the Kyber-512 algorithm that has been adopted by the National Institute of Standards and Technology as a post quantum Key Encapsulation Mechanism. Although Kyber-512 is quantum resilient, it has large key pairs and ciphertexts, which presents critical challenges for the limited computational resources of IoT devices. Our performance assessment reveals that with an Attribute Protocol Maximum Transmission Unit (ATT MTU) of 65 bytes, the pairing time increases by approximately 9 folds using our PQKE in comparison to the traditional BLE SC pairing, mainly due to increased data transmission. Nevertheless, by employing a larger ATT MTU, the pairing time of our PQKE mechanism can be minimized to be of the same order of magnitude as current pre-quantum key exchange for BLE. We therefore advocate for the adoption of larger ATT MTU sizes in quantum resilient BLE pairing to ensure the performance and usability of the technology in a post-quantum world.
In recent years, there has been a notable shift towards automation in agricultural technology. Furthermore, driven by environmental concerns, green energy is gradually replacing conventional power sources. However, green energy sources are vulnerable to external environmental factors, resulting in an unreliable power supply. Such instability can disrupt the functioning of monitoring systems which are critical for agricultural automation, leading to operational issues. To address these challenges, we propose an intermittent edge computing system for agricultural automation. It consists of battery-less intermittent computing devices, wireless intermittent communications, and deep-capable edge computing. Our demo illustrates the process of tomato monitoring including reliable on- device tomato detection and Bluetooth Low Energy (BLE) image data transmission, even in the absence of stable power conditions. Upon receiving the data, we employ edge computing to detect the location, color, and quantity of tomatoes.
Emergency Medical Services (EMS) responders often operate under time-sensitive conditions, facing cognitive overload and inherent risks, requiring essential skills in critical thinking and rapid decision-making. This paper presents CognitiveEMS, an end-to-end wearable cognitive assistant system that can act as a collaborative virtual partner engaging in the real-time acquisition and analysis of multimodal data from an emergency scene and interacting with EMS responders through Augmented Reality (AR) smart glasses. CognitiveEMS processes the continuous streams of data in real-time and leverages edge computing to provide assistance in EMS protocol selection and intervention recognition. We address key technical challenges in real-time cognitive assistance by introducing three novel components: (i) a Speech Recognition model that is fine-tuned for real-world medical emergency conversations using simulated EMS audio recordings, augmented with synthetic data generated by large language models (LLMs); (ii) an EMS Protocol Prediction model that combines state-of-the-art (SOTA) tiny language models with EMS domain knowledge using graph-based attention mechanisms; (iii) an EMS Action Recognition module which leverages multimodal audio and video data and protocol predictions to infer the intervention/treatment actions taken by the responders at the incident scene. Our results show that for speech recognition we achieve superior performance compared to SOTA (WER of 0.290 vs. 0.618) on conversational data. Our protocol prediction component also significantly outperforms SOTA (top-3 accuracy of 0.800 vs. 0.200) and the action recognition achieves an accuracy of 0.727, while maintaining an end-to-end latency of 3.78s for protocol prediction on the edge and 0.31s on the server.
Due to new regulations regarding privacy and the increasing reservations of citizens about surveillance systems, cities are exploring privacy-friendly technologies for crowd monitoring. An alternative gaining significant interest is mmWave radar for people counting applications. However, most studies in mmWave monitoring primarily focus on indoor or outdoor deployments under ideal conditions. Working with the municipality of Ams-terdam, we propose a method for counting people with mmWave in a real outdoor scenario. These realistic scenarios pose two significant difficulties. First, it is hard to estimate the number of people when they walk together because the radar captures the group as a single-point cloud. Second, the dynamic environment adds random points (noise) that may get reflected as false counts. To tackle these challenges, we optimize a state-of-the-art deep learning model to attain noise filtering and cluster disaggregation. We deploy the radar across a street and measure its performance under real traffic conditions. Our system achieves an accuracy of 96% for Presence detection and 69% for People counting. Over the long term, the system can monitor the crowd level with an error below 2.5%. Furthermore, considering the needs of our application, we reduce the complexity of the original SoA model by 93%. Our results indicate that mmWave is a promising solution for privacy-friendly people counting in urban scenarios and exposes unique challenges that have not been reported or considered in the SoA.
Sign Language is widely used by over 500 million Deaf and hard of hearing (DHH) individuals in their daily lives. While prior works made notable efforts to show the feasibility of recognizing signs with various sensing modalities both from the wireless and wearable domains, they recruited sign language learners for validation. Based on our interactions with native sign language users, we found that signal diversity hinders the generalization of users (e.g., users from different backgrounds interpret signs differently, and native users have complex articulated signs), thus resulting in recognition difficulty. While multiple solutions (e.g., increasing diversity of data, harvesting virtual data from sign videos) are possible, we propose ASLRing that addresses the sign language recognition problem from a meta-learning perspective by learning an inherent knowledge about diverse spaces of signs for fast adaptation. ASLRing bypasses expensive data collection process and avoids the limitation of leveraging virtual data from sign videos (e.g., occlusions, overexposure, low-resolution). To validate ASLRing, instead of recruiting learners, we conducted a comprehensive user study with a database with 1080 sentences generated by a vocabulary size of 1057 from 14 native sign language users and achieved a 26.9% word error rate, and we also validated ASLRing in diverse settings 1 .
Making Internet-of-Things (IoT) applications adaptive under unexpected failures and resource fluctuations can be challenging. In this demo, we present ImmunoPlane, a middleware system that brings adaptivity to IoT applications. Using a distributed stream-processing application, we demonstrate how ImmunoPlane transparently configures and deploys the application such that it can adapt to network congestions and random device failures.
Modern IoT/edge applications require one-to-many wireless communication (e.g., multi-drone coordination, data sharing among vehicles, synchronized IoT light shows). Due to the constantly varying wireless medium, thus reception quality, the sender must adjust its transmitting rate on a per-frame basis to meet the goodput and loss requirements of multiple receivers. Deciding an optimal rate within tens of milliseconds from nearly a hundred or more choices and continuing to chase that moving target is extremely challenging. Existing wireless technologies have little support for multicast rate control: most works are designed for unicast, where one receiver sends explicit per frame feedback, which is infeasible to scale to multiple receivers; a few works for multicast have rigid structures and high overhead unsuitable for IoT/edge; and most designs are based on a common implicit assumption: higher rates incur more losses. In this paper, we conduct systematic experiments and find that only a small fraction of data rates are practically useful, and higher rates can incur similar or even lower losses, thus cutting the data rate table size by 3.8X, making it manageable to select the optimal rate within a short duration. We further design an application-adaptive multicast rate control feedback protocol (r-DACK) with two policies enabling receivers to specify their desired loss rate, or loss rate and goodput requirements. r-DACK enables most receivers to meet their goodput/loss requirements while not being "bogged down" by some stragglers with bad reception quality. We build a prototype leveraging 802.11ac radio hardware and show that r-DACK can meet various goodput (15-50Mbps) and loss rate (10-50%) requirements successfully, both indoors and outdoors.
Hierarchical Federated Learning (HFL) has shown great promise over the past few years, with significant improvements in communication efficiency and overall performance. However, current research for HFL predominantly centers on supervised learning. This focus becomes problematic when dealing with semi-supervised learning, particularly under non-IID scenarios. In order to address this gap, our paper critically assesses the performance of straightforward adaptations of current state-of-the-art semi-supervised FL (SSFL) techniques within the HFL framework. We also introduce a novel clustering mechanism for hierarchical embeddings to alleviate the challenges introduced by semi-supervised paradigms in a hierarchical setting. Our approach not only provides superior accuracy, but also converges up to 5.11× faster, while being robust to non-IID data distributions for multiple datasets with negligible communication overhead. 1
A central challenge in machine learning deployment is maintaining accurate and updated models as the deployment environment changes over time. We present a hardware/software framework for simultaneous training and inference for monocular depth estimation on edge devices. Our proposed frame-work can be used as a hardware/software co-design tool that enables continual and online federated learning on edge devices. Our results show real-time training and inference performance, demonstrating the feasibility of online learning on edge devices.
In deep learning (DL) based human activity recognition (HAR), sensor selection seeks to balance prediction accuracy and sensor utilization (how often a sensor is used). With advances in on-device inference, sensors have become tightly integrated with DL, often restricting access to the underlying model used. Given only sensor predictions, how can we derive a selection policy which does efficient classification while maximizing accuracy? We propose a cascaded inference approach which, given the prediction of any one sensor, determines whether to query all other sensors. Typically, cascades use a sequence of classifiers which terminate once the confidence of a classifier exceeds a threshold. However, a threshold-based policy for sensor selection may be suboptimal; we define a more general class of policies which can surpass the threshold. We extend to settings where little or no labeled data is available for tuning the policy. Our analysis is validated on three HAR datasets by improving upon the F1-score of a threshold policy across several utilization budgets. Overall, our work enables practical analytics for HAR by relaxing the requirement of labeled data for sensor selection and reducing sensor utilization to directly extend a sensor system’s lifetime.
In industrial countries, adults spend a considerable amount of time sedentary each day at work, driving and during activities of daily living. Characterizing the seated upper body human poses using mmWave radars is an important, yet under-studied topic with many applications in human-machine interaction, transportation and road safety. In this work, we devise SUPER, a framework for seated upper body human pose estimation that utilizes dual-mmWave radars in close proximity. A novel masking algorithm is proposed to coherently fuse data from the radars to generate intensity and Doppler point clouds with complementary information for high-motion but small radar cross section areas (e.g., upper extremities) and low-motion but large RCS areas (e.g. torso). A lightweight neural network extracts both global and local features of upper body and output pose parameters for the Skinned Multi-Person Linear (SMPL) model. Extensive leave-one-subject-out experiments on various motion sequences from multiple subjects show that SUPER outperforms a state-of-the-art baseline method by 30 - 184%. We also demonstrate its utility in a simple downstream task for hand-object interaction.
We contribute SEAGULL, a novel pervasive sensing approach for monitoring and identifying underwater plastics. SEAGULL builds on an innovative light (LED) sensing solution that takes advantage of convolutional sparse coding to classify plastic debris according to their material (resin identification code). This enables SEAGULL to determine the composition of plastics in-situ, unlike existing plastic analysis methods which require taking the samples to a laboratory where they are analyzed using high precision measurement instruments. Through extensive experiments we demonstrate that SEAGULL correctly distinguishes between the main plastic categories (over 85% accuracy), is able to operate robustly against diverse water conditions (turbulence, turbidity, luminosity), and works with different sensing resolutions. We also demonstrate the practicality of SEAGULL by carrying out field tests in an ocean and a river, demonstrating that the performance of SEAGULL translates to real in-the-wild environments. Our work demonstrates how low-cost pervasive sensing solutions help to tackle environmental sustainability challenges, offering a new way to collect information about the extent and characteristics of underwater plastics and improving the scale at which monitoring can operate while overcoming the main constraints of existing techniques.
Deep Neural Networks (DNNs) are commonly used in camera systems for video surveillance. However, the computational demands of DNN inference pose challenges for on-edge video analytics due to potential delay. Additionally, edge cameras typically employ lightweight models, which are susceptible to data drift. In this demo, we present EdgeCam, an open-source distributed camera operating system that incorporates inference scheduling and continuous learning for video analytics. EdgeCam comprises multiple edge nodes and the cloud, enabling collaborative video analytics. Edge nodes also collect drift data to support continuous learning and maintain recognition accuracy. We have implemented essential functionalities and algorithms, ensuring modularity and ease of configuration. The source code of EdgeCam is at https://github.com/MSNLAB/EdgeCam.
In a world driven by data, cities are increasingly interested in deploying networks of smart city devices for urban and environmental monitoring. To be successful, these networks must be reliable, low-cost, and easy to install and maintain—criteria that are all significantly affected by the design choices around power and can seemingly be satisfied with the use of solar energy. However, solar power is not ubiquitous throughout cities, making it difficult to know where to place nodes to avoid charging issues and thus potentially increasing maintenance costs. This abstract describes the development of a machine learning model that predicts whether any arbitrary location in a city will have solar charging issues. Using data from a large-scale real-world solar-powered sensor deployment in Chicago, Illinois and open data about building location and height, the binary classification model outputs the probability of adequate solar charging at a node location with 77% accuracy on the held-out test set. This work lays the foundation for those deploying future solar-powered urban sensor networks to have more confidence in the reliability of their chosen node locations.
Small-scale autonomous airborne vehicles, such as micro-drones, are expected to be a central component of a broad spectrum of applications ranging from exploration to surveillance and delivery. This class of vehicles is characterized by severe constraints in computing power and energy reservoir, which impairs their ability to support the complex state-of-the-art neural models needed for autonomous operations. The main contribution of this paper is a new class of neural navigation models - NaviSlim - capable of adapting the amount of resources spent on computing and sensing in response to the current context (i.e., difficulty of the environment, current trajectory, and navigation goals). Specifically, NaviSlim is designed as a gated slimmable neural network architecture that, different from existing slimmable networks, can dynamically select a slimming factor to autonomously scale model complexity, which consequently optimizes execution time and energy consumption. Moreover, different from existing sensor fusion approaches, NaviSlim can dynamically select power levels of onboard sensors to autonomously reduce power and time spent during sensor acquisition, without the need to switch between different neural networks. By means of extensive training and testing on the robust simulation environment Microsoft AirSim, we evaluate our NaviSlim models on scenarios with varying difficulty and a test set that showed a dynamic reduced model complexity on average between 57-92%, and between 61-80% sensor utilization, as compared to static neural networks designed to match computing and sensing of that required by the most difficult scenario.
With the proliferation of Internet of Things (IoT) applications relying on LoRa to gather data from dispersed devices, LoRa communications become prone to jamming attacks, which can cause massive packet loss, reduced throughput, and depleting batteries. Existing work related to jamming in LoRa mainly considers the impact of jamming and does not provide any anti-jamming techniques. Mitigating jamming in a LoRa network is extremely challenging as the devices have low computation power and limited energy typically supplied by small batteries. In this paper, we present a jamming mitigation technique for LoRa that imposes no overhead for energy-constrained nodes and can decode packets even when the SNR is ultra-low. We do this by exploiting the temporal and spatial variations of jammed signals of the same packet at different locations. Our design requires no change in LoRa nodes’ physical layer, making it usable with all commercial off-the-shelf (COTS) LoRa devices. It works even under a powerful jammer with variable jamming signals, length of jamming packets, and mobility. Our jamming mitigation technique can be combined with existing collision recovery techniques for multiple LoRa packets to recover packets that collide with either jamming signal or with other LoRa packets of the same network. Finally, we evaluate the effectiveness of our jamming mitigation technique through outdoor experiments. The results show that our technique mitigates jamming by improving packet reception rate and energy consumption per packet up to 83.91 and 115.65 times, respectively, compared to the traditional LoRa.
Littering is a significant environmental concern that causes significant damage to the natural ecosystem and contributes adversely to human health. Monitoring litter accumulation is currently labour-intensive and costly, often resulting in action being taken only once the environment has already become polluted. We contribute LIZARD, a novel pervasive sensing solution for detecting and monitoring plastics that is tailored to autonomous vehicles. LIZARD relies on an innovative sensing pipeline that combines thermal imaging and optical sensing. The intuition is to rely on thermal dissipation patterns to identify larger (macro) plastics and use optical sensing to sample area with the highest density of plastics to identify smaller (micro and meso) plastics. Ours is the first pervasive sensing solution that can detect microplastics in the environment and be integrated into autonomous vehicles. Indeed, state-of-the-art solutions are either limited to laboratory analysis with special instruments or rely on manual observation without being able to identify the smallest plastics – which often are the most dangerous. We evaluate LIZARD through rigorous experiments that combine controlled laboratory settings and in-the-field measurements carried out in three real-world locations to evaluate LIZARD. Our results show that LIZARD can be used to detect plastics of different sizes with an accuracy of up to 80%. The performance depends on the diameter of the plastics, the background surface, and the luminosity of the environment. We also demonstrate that our solution can be easily integrated with ground drones, enabling (semi-)autonomous litter monitoring. Our work offers an innovative way to harness pervasive sensing to address an important global (environmental) sustainability challenge while paving the way toward improved monitoring of the accumulation of harmful plastic fragments in the environment.
Inertial Measurement Unit (IMU) sensors are commonly used for estimating device orientation. However, due to the irregular movements of devices and distortions of magnetic fields, IMU sensors may present varying data quality. Conventional data fusion approaches such as Complementary Filter (CF) and Kalman Filter struggle to adapt to these variations. Recent efforts have explored the utilization of deep learning to directly infer orientation from IMU sensor data. Nevertheless, when facing new scenarios that have different data distributions from training (e.g., different movement patterns or magnetic fields), deep learning methods cannot accurately infer orientation. In this paper, we conduct extensive experiments and identify two critical parameters for CF-based orientation estimation. We propose employing deep learning to adjust these two parameters, rather than directly inferring the final orientation outcomes. Since the relationship between sensor data and the settings of CF parameters is relatively simpler than the relationship between sensor data and orientation, a deep learning model of the same size can learn the first relationship more effectively and efficiently. We develop DRLPilot which leverages Deep Reinforcement Learning (DRL) to pilot CF-based orientation estimation based on the data quality of IMU sensors. Our DRL framework incorporates novel state design and reward function to accommodate the unique features of IMU sensor data and orientation estimation. Extensive experiment results demonstrate DRLPilot outperforms baseline systems by 27% in orientation accuracy.
Wearable IoT devices rely on batteries, which pose challenges for long-term sustainable health monitoring due to the need for recharging or replacement. Batteryless sensing approaches, which harvest energy from the environment, offer an appealing alternative. However, given the discontinuous supply of harvested energy, it is unclear how to leverage sparse, asynchronous data from batteryless sensors for machine learning (ML) tasks such as human activity recognition (HAR). To this end, we present and profile a prototype of a system to simulate data acquisition from a set of kinetic energy harvesting devices. Our results demonstrate that there is a need to jointly optimize (1) when sensors should spend energy to communicate data, and (2) the training of the ML model that will receive the data.
