Multi-agent reinforcement learning is a promising approach to solve decentralized control problems. However, if states of agents are dependent on other agents’ states, knowing the whole system state is crucial for decision-making. Exchanging state information between agents requires a communication infrastructure, which in practice might be unreliable. In this paper, we focus on a setting where a central controller is present that can store and forward state information of agents. We propose a reconstruction approach that allows agents to predict the state of other agents based on their own observations. During episodes of lost communication, agents can still act according to a previously learned joint policy. We show that the proposed approach is able to achieve a similar performance as a centralized approach, even when the communication between agents is unreliable.

Running offers substantial health benefits, but improper gait patterns can lead to injuries, particularly without expert feedback. While prior gait analysis systems based on cameras, insoles, or body-mounted sensors have demonstrated effectiveness, they are often bulky and limited to offline, post-run analysis. Wrist-worn wearables offer a more practical and non-intrusive alternative, yet enabling real-time gait recognition on such devices remains challenging due to noisy Inertial Measurement Unit (IMU) signals, limited computing resources, and dependence on cloud connectivity. This paper introduces StrikeWatch, a compact wrist-worn system that performs entirely on-device, real-time gait recognition using IMU signals. As a case study, we target the detection of heel versus forefoot strikes to enable runners to self-correct harmful gait patterns through visual and auditory feedback during running. We propose four compact DL architectures (1D-CNN, 1D-SepCNN, LSTM, and Transformer) and optimize them for energy-efficient inference on two representative embedded Field-Programmable Gate Arrays (FPGAs): the AMD Spartan-7 XC7S15 and the Lattice iCE40UP5K. Using our custom-built hardware prototype, we collect a labeled dataset from outdoor running sessions and evaluate all models via a fully automated deployment pipeline. Our results reveal clear trade-offs between model complexity and hardware efficiency. Evaluated across 12 participants, 6-bit quantized 1D-SepCNN achieves the highest average F1 score of 0.847 while consuming just 0.350 μJ per inference with a latency of 0.140 ms on the iCE40UP5K running at 20 MHz. This configuration supports up to 13.6 days of continuous inference on a 320 mAh battery. All datasets and code are available in the GitHub repository https://github.com/tianheng-ling/StrikeWatch.

Unreliable predictions limit the usability of AI in pervasive systems, especially in time series forecasting of critical sectors like medicine. Incomplete data and missing measurements degrade model performance and obscure their impact on predictions. Hardware failures and limited resources to collect data at required frequencies exacerbate these challenges. This research proposes XTRTimeS—eXplainable Transformer for Robust Time Series forecasting—an approach that leverages explainability techniques to improve forecasting with incomplete data while providing a confidence evaluation. The evaluation offers users insights into how missing data influence predictions, highlighting both likely and extreme scenarios. Additionally, it can be employed in machine-to-machine interactions, where predictions with low confidence are automatically discarded. By enhancing transparency and robustness, XTRTimeS supports reliable time series predictions, particularly in high-stakes applications such as healthcare and autonomous systems.

This paper presents a camera-based optical transmission system and architecture of secure access to Internet of Things (IoT) devices in ubiquitous computing environments. Devices emit light tokens, referred to as local identifiers (LIDs), which are captured and decoded by a nearby camera. A LID initiates a local discovery process, where a bootstrapping service links the LID to an IoT device within a local network, allowing interaction without physical contact or prior pairing. The system supports device setup and control in ad-hoc scenarios using standard commodity hardware. By connecting a temporary physical signal to a network device identity, it enables access without continuous connectivity or complex setup. The light signal serves as a temporary authentication mechanism, while the camera verifies physical proximity and enables positional discovery of nearby devices. It provides a simple, secure way to interact briefly with nearby devices and is well suited for deployment in smart environments, augmented reality interfaces, and other ubiquitous computing contexts.

In healthcare time series forecasting, missing data is a prevalent challenge that undermines both model performance and interpretability. This paper investigates how different imputation strategies affect not only the predictive accuracy of machine learning models but also the internal logic behind their decisions. We propose a systematic evaluation using LSTM-based models trained on the PhysioNet Sepsis Challenge dataset, applying three common imputation methods, mean imputation, backward/forward fill, and linear interpolation, across varying feature sets. To assess interpretability, we employ Layer-wise Relevance Propagation (LRP), revealing that the choice of imputation significantly alters feature relevance patterns, even when predictive metrics remain similar. Our findings show that imputation is not a neutral preprocessing step; it fundamentally shapes what models learn and how they explain their outputs. These insights highlight the need for careful imputation design in clinical AI systems, especially where transparency and trust are critical.

The rapid advancement and accessibility of large language models (LLMs) have significantly enhanced their utility across various disciplines. However, this pervasive presence of LLMs raises substantial security and trust concerns, particularly at the user-level, within the context of cybersecurity. This paper investigates the potential misuse of LLMs in crafting spam emails that are convincingly legitimate, thus bypassing traditional spam filters. We already conducted an experiment utilizing ChatGPT 3.5 Turbo to alter spam emails, resulting in a 70% success rate of these emails being misclassified as legitimate. We present a pipeline demonstrating the ease with which LLMs can be exploited to undermine email security, highlighting the urgency for improved defensive mechanisms. To counteract these threats, we propose a novel methodology designed to enhance the robustness of spam filters against such sophisticated attacks. This methodology comprises three phases: assessing the vulnerability of current systems to LLM-modified spam, detailed examination of the changes imposed by LLMs, and applying insights gained to fortify existing security infrastructures.

Conducting online studies via crowdsourcing platforms is a widely used approach in the field of pervasive computing, as it enables efficient access to diverse, and scalable participant samples. However, certain study designs are not well suited for online implementation, risking compromised data quality. This issue became evident during our study on human interaction with AI assistance. By leveraging Prolific, a crowdsourcing website enabling online studies, we investigated how humans interact with AI-generated hints while solving a pipe maze game. The study employed "Poor Man’s Eye Tracking," combining mouse tracking with obscured vision fields, to monitor participant behaviour. Contrary to our hypothesis, participants did not increase diligence in verifying AI hints after AI errors were pointed out. Analysing our data suggests a lack of intrinsic motivation and attention among participants. This falsified the reliability and validity of the collected data. Our study showed that even a careful pre-selection of participants through a crowdsourced website cannot prevent these issues. Moreover, identifying missing motivation or inattentiveness in non-questionnaire components, such as game scenarios, requires additional data collection and analysis, such as mouse tracking, to retrospectively filter invalid datasets. This paper discusses potential causes and methods to mitigate these issues and proposes strategies to identify and prevent data distortion caused by missing motivation and inattentiveness of subjects.

Spam and phishing remain critical threats in cybersecurity, responsible for nearly 90% of security incidents. As these attacks grow in sophistication, the need for robust defensive mechanisms intensifies. Bayesian spam filters, like the widely adopted open-source SpamAssassin, are essential tools in this fight. However, the emergence of large language models (LLMs) such as ChatGPT presents new challenges. These models are not only powerful and accessible, but also inexpensive to use, raising concerns about their misuse in crafting sophisticated spam emails that evade traditional spam filters. This work aims to evaluate the robustness and effectiveness of SpamAssassin against LLM-modified email content. We developed a pipeline to test this vulnerability. Our pipeline modifies spam emails using GPT-3.5 Turbo and assesses SpamAssassin's ability to classify these modified emails correctly. The results show that SpamAssassin misclassified up to 73.7% of LLM-modified spam emails as legitimate. In contrast, a simpler dictionary-replacement attack showed a maximum success rate of only 0.4%. These findings highlight the significant threat posed by LLM-modified spam, especially given the cost-efficiency of such attacks (0.17 cents per email). This paper provides crucial insights into the vulnerabilities of current spam filters and the need for continuous improvement in cybersecurity measures.

We consider Hidden Markov Models (HMMs) that admit unobservable epsilon-transitions (also called null transitions), allowing state changes of which the observer is unaware. Due to the presence of epsilon-loops this additional feature complicates the theory and requires to carefully set up the corresponding probability space and random variables. In particular we present an algorithm for determining the most probable explanation given an observation (a generalization of the Viterbi algorithm for HMMs) and a method for parameter learning that adapts the probabilities of a given model based on an observation (a generalization of the Baum-Welch algorithm). The latter algorithm guarantees that the given observation has a higher (or equal) probability after adjustment of the parameters and its correctness can be derived directly from the so-called EM algorithm. We also provide runtime results in order to evaluate the efficiency of the proposed algorithms.

The interplay between cause and effect of smart lighting devices on the perceived brightness in smart homes is an open research question. Solving it allows for outcome-oriented smart home control, for example choosing a target brightness in the living room and letting the smart home find out, what it should do to achieve this. We propose a novel method to learn the behaviour in smart homes with Symbolic Regression (SR), based on data gathered from sensors in the environment, and the actuator settings at the time of measurement. We use symbolic regression to find the dependency between the settings of smart home devices and the perceived brightness by their users. In this work we evaluate our method with a ray-traced room that includes a lamp, a window blind and the sun that assumes different positions relative to the room throughout the day. The results indicate that SR can be used for this problem, but more research and refinement needs to be performed to yield satisfying results.

A smart home control application allows users to interact with the Cyber-Physical System (CPS) in their living space. Conventional applications offer manual control over individual smart devices, disregarding compound effects from using multiple devices. Manual control is not user-friendly, as users must explore smart device settings until achieving their target states. In previous works, we proposed a symbolic regression approach to find settings of smart devices that achieve user-specified goals. We now demonstrate a novel application, SmartBright, an outcome-oriented light control for macOS. SmartBright allows users to set a target brightness and the current time of day; the application finds settings for the window blind and lamp to achieve this goal. We evaluated SmartBright by comparing suggested settings for a bright room against ray-traced brightness throughout the day. Additionally, we applied the user preference Do not use the lamp, if possible to minimize lamp use and evaluated these settings. Results show SmartBright suggests settings achieving user goals throughout the day, with average errors of 0.1175 P_l for the unrestricted case, and 0.1730 P_l for the restricted case.

Scripts, automation and interoperability of Smart Homes are brittle and break for all kinds of reasons. We propose and evaluate the Web of Things Proxy Layer (WoT-PL). This semantic middleware deploys digital twins for devices so that automation can work independently of the physical devices. We use Large Language Models (LLMs) to semantically translate various IoT device data into standardised Web of Things (WoT) Thing Descriptions; this approach unifies previously heterogeneous data structures and proprietary data types. Our experiments suggest that while LLMs seem promising in device data translation, consistent accuracy is challenging to achieve. The proposed WoT-PL framework aims to improve the adaptability and longevity of smart home systems.

The convergence of Cyber-Physical Systems (CPS) and autonomous systems presents numerous application areas and challenges. Many CPS use cases require autonomy, which necessitates robust monitoring, maintenance, and longevity of these systems. However, handling unmodeled data, environmental uncertainties, and emergent behaviour poses significant challenges. This paper proposes the use of a novel formalism, Extended Hidden Markov Models with ϵ -emissions ( ϵ -HMMT), to estimate the correctness of an autonomous system’s emergent behaviour in complex and changing environments. We discuss the advantages and disadvantages of different modeling approaches, emphasizing the need for a more adaptable and robust modeling technique.

Distributed machine learning (DML) may become an important component of IoT device fleets and smart homes in the future. However, it currently presents challenges where reliable (or frequent) internet connectivity is necessary, or trust is not handled. Since DML is generally decentralized and often relies on peer-to-peer networks, we argue that BitTorrent as a time -proven protocol in this space could aid in building a solution. This paper explores the possibilities of employing BitTorrent mechanisms for gossip -based DML. It provides initial evidence supporting the viability of this approach by analysing the behaviour of model training in a simulator representing 30 individual peers with distinct data sets.

Driving under stress negatively affects driving behaviour, increasing the risk of dangerous traffic situations and accidents. To effectively reduce driving stress, it is crucial to gain insights into stress triggers using ubiquitous stress detection methods that facilitate the collection of real-world driving data. We developed a system that localises stressors along a route using heart rate data collected from a smartwatch. Stress sources are identified by detecting and classifying heart rate anomalies, which are then correlated with GPS locations. Our system differentiates between common and individual stressors using a scoring system and maps detected stress indicators on a stressmap, allowing comparison across multiple rides. Crowdsourced deployment enhances the precision of stressor localization and enables their association with specific road characteristics. Our findings indicate that heart rate anomalies reliably predict the locations of stressors, which are consistently observed across multiple rides and different drivers. These stressors are linked to traffic facilities and road features, such as intersections and traffic lights. By employing ubiquitous stress detection methods, we enable the collection of crowdsourced data, providing new insights into real-world driving stress. Our stress detection and visualization system aims to improve route guidance, particularly benefiting stress-prone, stress-sensitive, and frequent drivers.

Multi-agent reinforcement learning solves optimization problems in sequential decision-making and enables controlling spatially distributed actuators. We consider a cooperative setting where agents can exchange information via a central controller, e.g. a cloud-based service. In real world applications however, communication channels are often error-prone and agents may become disconnected and can neither send its observation nor receive observations from other agents. We formalize this problem as a subclass of decentralized Markov decision processes and discuss the complexity of the problem. We then propose several solution concepts that involve breaking down the complexity by considering only a subset of failure scenarios, learning independent policies for each failure scenario, reconstructing missing information and learning policies that incorporate the state uncertainty in the training process.

We developed a tool to track the test person's gaze while conducting tasks in a web browser [1]. In the browser we can only track the mouse pointer, which is known as poor man's eye tracker. However, the mouse pointer does not necessarily reveal on which area of the web page the user is focusing and therefore it does not allow to conclude the attention of the test person. Therefore, we extended the mouse tracking by applying a fog to the web page. Only an area around the mouse pointer is without fog. Consequently, the mouse pointer must follow the gaze. In addition, we developed a visualisation tool to analyze the focus of the user, i.e. determine the areas where the user's eye had enough time to perceive the visible area. In summary, during our work, we developed two tools. One for conducting the crowd-sourced study (Pipe Maze Study), and one for visualizing and evaluating its findings (Pipe Maze Visualizer). You can find the source code for both tools in the following repository: https://github.com/vs-ude/poor-mans-eyetracker