Federated learning is well suited to edge environments but is often limited by the uplink cost of transmitting model updates. This Work-in-Progress paper presents MUFFLe, a communication-efficient update compression scheme that integrates generalized deduplication (GD) into the FedAvg pipeline. MUFFLe deduplicates repeated patterns across the update vector, yielding a fixed-rate, variable-count compression scheme. Preliminary experiments on IID MNIST with 20 clients show that MUFFLe reaches the target accuracy of 92.93% with 38 MB cumulative uplink communication, compared with 75 MB for 8-bit quantization, 86 MB for Top-k sparsification, and 310 MB for uncompressed FedAvg. These results demonstrate the feasibility of applying GD to communication-efficient federated learning.

Sensors provide a critical link between digital and physical systems in the Internet of Things (IoT). However, as they age, their accuracy degrades due to drift. This reduces data trustworthiness and requires significant maintenance investment to mitigate, especially in large-scale sensor deployments typical of IoT systems. Previous approaches to drift correction typically require large volumes of ground-truth data and do not consider measurement or prediction uncertainty. In this article, we propose a probabilistic sensor drift correction method that takes a fundamental approach to modeling the sensor response using Gaussian process regression (GPR). Tested using dissolved oxygen (DO) sensors, our method delivers mean-squared-error (mse) reductions of up to 90% and more than 20% on average. We also propose a novel uncertainty-driven calibration schedule optimization approach that builds on top of drift correction and further reduces mse by up to 15.7%.

Massive data streams from IoT and cyber-physical systems must be processed under strict bandwidth, latency, and resource constraints. Generalized Deduplication (GD) is a promising lossless compression framework, as it supports random access and direct analytics on compressed data. However, existing GD algorithms exhibit quadratic complexity 𝒪(nd^2), which limits their scalability for high-dimensional datasets. This paper proposes EntroGD, an entropy-guided GD framework that decouples analytical fidelity from compression efficiency to achieve linear complexity 𝒪(nd). EntroGD adopts a two-stage design, first constructing compact condensed samples to preserve information critical for analytics, and then applying entropy-based bit selection to maximize compression. Experiments on 18 IoT datasets show that EntroGD reduces configuration time by up to 53.5× compared to state-of-the-art GD compressors. Moreover, by enabling analytics with access to only 2.6% of the original data volume, EntroGD accelerates clustering by up to 31.6× with negligible loss in accuracy. Overall, EntroGD provides a scalable and system-efficient solution for direct analytics on compressed IoT data.

We introduce a novel event-trigger threshold (ETT) regulation mechanism based on the quantitative semantics of propositional logic (PL). We exploit the expressiveness of the PL vocabulary to deliver a precise and flexible specification of ETT regulation based on system requirements and properties. Additionally, we present a modified ETT regulation mechanism that provides formal guarantees for satisfaction/violation detection of arbitrary PL properties. To validate our proposed method, we consider a convoy of vehicles in an adaptive cruise control scenario. In this scenario, the PL operators are used to encode safety properties and the ETTs are regulated accordingly, e.g., if our safety metric is high there can be a higher ETT threshold, while a smaller threshold is used when the system is approaching unsafe conditions. Under ideal ETT regulation conditions in this safety scenario, we show that reductions between 41.8 - 96.3% in the number of triggered events is possible compared to using a constant ETT while maintaining similar safety conditions.

We are witnessing a plethora of computing and storage resources with various characteristics and technologies that operate in the edge, in the cloud and in high performance computing environments. In parallel, there is a strong move towards the creation of new services in support of diverse applications that span across these environments that interoperate in the form of a continuum. In this work, we present the SERRANO platform for the creation of an Edge-Cloud-HPC continuum in support of highly demanding, dynamic and security-critical applications. We present the platform’s architecture, main components and interactions. We also describe the way the various resources are integrated under the platform and orchestrated.

This paper presents a multimodal framework that integrates touch signals (contact points and surface normals) into 3D Gaussian Splatting (3DGS). Our approach enhances scene reconstruction, particularly under challenging conditions like low lighting, limited camera viewpoints, and occlusions. Different from the visual-only method, the proposed approach incorporates spatially selective touch measurements to refine both the geometry and appearance of the 3D Gaussian representation. To guide the touch exploration, we introduce a two-stage sampling scheme that initially probes sparse regions and then concentrates on high-uncertainty boundaries identified from the reconstructed mesh. A geometric loss is proposed to ensure surface smoothness, resulting in improved geometry. Experimental results across diverse scenarios show consistent improvements in geometric accuracy. In the most challenging case with severe occlusion, the Chamfer Distance is reduced by over 15x, demonstrating the effectiveness of integrating touch cues into 3D Gaussian Splatting. Furthermore, our approach maintains a fully online pipeline, underscoring its feasibility in visually degraded environments.

Reliability in distributed storage systems has typically focused on the design and deployment of data replication or erasure coding techniques. Although some scenarios have considered the use of replication for hot data and erasure coding for cold data in the same system, each is designed in isolation. We propose HyRES, a hybrid scheme incorporates the best characteristics of each scheme, thus, resulting in additional design flexibility and better potential performance for the system. We show that HyRES generalizes previously proposed hybrid schemes. We characterize the theoretical performance of HyRES as well as that of replication and erasure coding considering the effects of the size of the storage networks. We validate our theoretical results using simulations. These results show that HyRES can yield simultaneously lower storage costs than replication, lower probabilities of file loss than replication and erasure coding with similar worst case performance, and even lower effective repair traffic than replication when considering the network size.

Local differential privacy techniques for numerical data typically transform a dataset to ensure a bound on the likelihood that, given a query, a malicious user could infer information on the original samples. Queries are often solely based on users and their requirements, limiting the design of the perturbation to processes that, while privatizing the results, do not jeopardize their usefulness. In this article, we propose a privatization technique called Zeal, where perturbator and aggregator are designed as a unit, resulting in a locally differentially private mechanism that, by-design, improves the compressibility of the perturbed dataset compared to the original, saves on transmitted bits for data collection and protects against a privacy vulnerability due to floating point arithmetic that affects other state-of-the-art schemes. We prove that the utility error on querying the average and median is invariant to the bias introduced by Zeal in a wide range of conditions, and that under the same circumstances, Zeal also guarantees protection against the aforementioned vulnerability. Moreover, we show that in many scenarios Zeal can outperform other privatization techniques in terms of utility error, compression and data transmission efficiency. Our experiments show up to 94 % improvements in compression and up to 95 % more efficient data transmissions with respect to the original.

Smart cities and their associated applications integrate various cutting-edge technologies, such as machine learning (ML) and edge computing (EC). ML-driven services within smart cities are often computationally intensive and time-sensitive. While EC serves as a complementary infrastructure to traditional cloud-end users architecture, enhancing the quality-of-service (QoS) for end users, the advent of ML-dependent services necessitates a redefinition of QoS to incorporate relevant ML performance metrics. On the other side, the limited computational capabilities of EC servers make such QoS metrics conflicting. Therefore, there is a tradeoff between system cost for running ML models and ML model performance. In this article, we present an effective offloading scheme designed for a three-layer cloud-edge-users system, aiming to optimize the performance-QoS tradeoff in the long run, especially for ML-dependent services, considering the dynamic of the system. Leveraging Lyapunov optimization techniques, we transform the offloading problem from a long-term perspective into a frame-based problem. Recognizing that the new per-frame problem is NP-hard, we introduce a tabu search-based algorithm, termed TSMALE, to address this challenge. Analytically, we demonstrate that our proposed TSMALE algorithm achieves an approximate solution of roughly 0.923 compared to the optimal solution.

Despite rapid advancements, machine learning, particularly deep learning, is hindered by the need for large amounts of labeled data to learn meaningful patterns without overfitting and immense demands for computation and storage, which motivate research into architectures that can achieve good performance with fewer resources. This paper introduces dreaMLearning, a novel framework that enables learning from compressed data without decompression, built upon Entropy-based Generalized Deduplication (EntroGeDe), an entropy-driven lossless compression method that consolidates information into a compact set of representative samples. DreaMLearning accommodates a wide range of data types, tasks, and model architectures. Extensive experiments on regression and classification tasks with tabular and image data demonstrate that dreaMLearning accelerates training by up to 8.8x, reduces memory usage by 10x, and cuts storage by 42

IoT applications often require devices to continuously send huge amounts of sensor data to the cloud in order to detect anomalies. This paper proposes a novel preprocessing stage selecting small portions of the sensor data worth sending for further analysis, resulting in significant savings in transmission costs and processing time in the cloud, down to less than 1% of the complete stream, while achieving comparable detection results.

Resource constraints on the computing continuum require that we make smart decisions for serving AI-based services at the network edge. AI-based services typically have multiple implementations (e.g., image classification implementations include SqueezeNet, DenseNet, and others) with varying trade-offs (e.g., latency and accuracy). The question then is how should AI-based services be placed across Function-as-a-Service (FaaS) based edge computing systems in order to maximize total Quality-of-Service (QoS). To address this question, we propose a problem that jointly aims to solve (i) edge AI service placement and (ii) request scheduling. These are done across two time-scales (one for placement and one for scheduling). We first cast the problem as an integer linear program. We then decompose the problem into separate placement and scheduling subproblems and prove that both are NP-hard. We then propose a novel placement algorithm that places services while considering device-to-device communication across edge clouds to offload requests to one another. Our results show that the proposed placement algorithm is able to outperform a state-of-the-art placement algorithm for AI-based services, and other baseline heuristics, with regard to maximizing total QoS. Additionally, we present a federated learning-based framework, FLIES, to predict the future incoming service requests and their QoS requirements. Our results also show that our FLIES algorithm is able to outperform a standard decentralized learning baseline for predicting incoming requests and show comparable predictive performance when compared to centralized training.

The EU-funded EMPYREAN project (empyrean-horizon.eu) aims to establish a hyper-distributed computing paradigm, leveraging collaborative, heterogeneous IoT devices and federated resources. EMPYREAN focuses on developing technologies for efficient AI workload processing, secure distributed edge storage and cloud-native application development. It will offer open and standardised APIs and use open-source platforms. EMPYREAN's capabilities will be demonstrated through three use cases: advanced manufacturing, smart agriculture, and warehouse automation.

Erasure coding in distributed multi-cloud data storage increases availability, durability and security, but it also makes data analytics inefficient since the whole dataset must be reconstructed to answer a query, even if the result set is a small fraction of the complete file. Data compression has a similar trade-off as it can reduce storage costs while requiring the entire compressed data to be collected and decompressed in order to access even a few bytes. We propose TREAT, a novel method that combines erasure coding and compression to achieve efficient queries of time series datasets while keeping the benefits of both underlying techniques. Our evaluation of five real-life datasets shows that it can answer range queries up to 25 times faster with 100 times less data transfer than reconstructing the whole dataset.

The exponential growth of data generated by the Internet of Things presents significant challenges for data communication, storage, and analytics. Consequently, organizations often face high costs when attempting to leverage their own data. Novel techniques that holistically optimize data storage and analytics in IoT systems are therefore required. One promising approach is generalized deduplication (GD), which is a lossless compression technique that delivers high compression while also enabling low-cost random access directly on compressed data. In this article, we introduce GreedyGD, a novel GD data compression algorithm that offers reliable, efficient data analytics, along with more compression and faster runtime compared to previous GD compressors. Evaluating GreedyGD on 18 real-world datasets revealed excellent performance: a 11.2× speed-up, 1.6× more compression, and more accurate and reliable analytics while using 4× less data compared to previous GD compressors.

Exponential growth in data collection is creating significant challenges for data storage and analytics latency. Approximate Query Processing (AQP) has long been touted as a solution for accelerating analytics on large datasets, however, there is still room for improvement across all key performance criteria. In this paper, we propose a novel histogram-based data synopsis called PairwiseHist that uses recursive hypothesis testing to ensure accurate histograms and can be built on top of data compressed using Generalized Deduplication (GD). We thus show that GD data compression can contribute to AQP. Compared to state-of-the-art AQP approaches, Pairwise-Hist achieves better performance across all key metrics, including 2.6× higher accuracy, 3.5× lower latency, 24× smaller synopses and 1.5--4× faster construction time.

As the usage of the Internet of Things (IoT) has spread across various sectors, the amount of IoT data generated has increased dramatically. A large fraction of IoT applications focuses on monitoring significant signals and events, using anomaly detection methods to identify and react to unusual activity. This is typically done by continuously transferring massive amounts of sensor data to the Cloud, where algorithms are run, and typically results in large communication, computing and storage loads with large bills to pay for these Cloud resources. This work presents a unique preprocessing stage that can select small chunks of the datasets worth forwarding for further analysis, resulting in significant reductions in cloud processing time and transmission costs. We compare the results of the proposed technique with two other alternative methods, using three datasets and four anomaly detectors. We show that our method outperforms the others in almost all scenarios, it reduces transmission and Cloud processing costs as much as 100 times while achieving comparable detection performances.

We introduce RAGE, an image compression framework that achieves four generally conflicting objectives: 1) good compression for a wide variety of color images, 2) computationally efficient, fast decompression, 3) fast random access of images with pixel-level granularity without the need to decompress the entire image, 4) support for both lossless and lossy compression. To achieve these, we rely on the recent concept of generalized deduplication (GD), which is known to provide efficient lossless (de)compression and fast random access in time-series data, and deliver key expansions suitable for image compression, both lossless and lossy. Using nine different datasets, incl. graphics, logos, natural images, we show that RAGE has similar or better compression ratios to state-of-the-art lossless image compressors, while delivering pixel-level random access capabilities. Tests in an ARM Cortex-M33 platform show seek times between 9.9 and 40.6~ns and average decoding time per pixel between 274 and 1226~ns. Our measurements also show that RAGE's lossy variant, RAGE-Q, outperforms JPEG by several fold in terms of distortion in embedded graphics and has reasonable compression and distortion for natural images.

The Internet of Things (IoT) has brought about exponential growth in sensor data. This has led to increasing demands for efficient and novel data transmission, storage, and analytics solutions for sustainable IoT ecosystems. It has been shown that the generalized deduplication (GD) compression algorithm offers not only competitive compression ratio and throughput but also random access properties that enable direct analytics of compressed data. In this article, we thoroughly stress test existing methods for direct analytics of GD compressed data with a diverse collection of 103 data sets, identify the need to optimize GD for analytics, and develop a new version of GD to this end. We also propose the generalized deduplication-enabled approximate edge analytics (GLEAN) framework. This framework applies the aforementioned analytics techniques at the Edge server to deliver end-to-end lossless data compression and high-quality Edge analytics in the IoT, thereby addressing challenges related to data transmission, storage, and analytics. Impressive analytics performance was achieved using this framework, with a median increase in $k$ -means clustering error of just 2% relative to analytics performed on uncompressed data, while running $7.5\times $ faster and requiring $3.9\times $ less storage at the Edge server compared to universal compressors.

The number of IoT devices is expected to continue its dramatic growth in the coming years and, with it, a growth in the amount of data to be transmitted, processed and stored. Compression techniques that support analytics directly on the compressed data could pave the way for systems to scale efficiently to these growing demands. This paper proposes two novel methods for preprocessing a stream of floating point data to improve the compression capabilities of various IoT data compressors. In particular, these techniques are shown to be helpful with recent compressors that allow for random access and analytics while maintaining good compression. Our techniques improve compression with reductions up to 80% when allowing for at most 1% of recovery error.