Fraud detection is a critical task in finance and e-commerce, but fraudsters increasingly evade detection by distributing their activities across multiple institutions. While this makes collaborative fraud detection (CFD) essential, it is hindered by two fundamental barriers: stringent data privacy regulations and constrained communication networks. Existing systems are impractical, as they either incur prohibitive overheads or cannot attain the high detection accuracy required to manage financial risk. This paper introduces CFDGraph, the first privacy-preserving graph processing system designed to overcome these challenges by synergistically balancing accuracy, privacy, and communication efficiency. CFDGraph's novelty lies in three interconnected innovations. First, it employs differential privacy with adaptive quantization to simultaneously reduce both privacy-induced noise and communication overhead. Second, it introduces accuracy-aware noise mitigation techniques to preserve the high recall essential for fraud detection. Third, it implements privacy-aware message compression to dramatically reduce data transmission volume. Evaluations show CFDGraph achieves 100% recall in identifying top-k fraudulent accounts while reducing manual investigation costs by up to 99.5% compared to state-of-the-art approaches.

Modern HPC systems increasingly rely on tiered storage architectures with SSDs serving as a critical performance tier. However, the inherent asynchronous I/O characteristics of SSDs, including read/write bandwidth asymmetry and interference, pose significant challenges for traditional I/O schedulers. These challenges are exacerbated by the convergence of bursty HPC write workloads (e.g., checkpointing) and sustained AI read workloads (e.g., data streaming) on shared SSD infrastructure. Existing schedulers fail to adequately address these combined workloads, leading to suboptimal resource utilization. This article introduces HARMONI, a heterogeneity-aware reinforcement learning scheduler for mixed I/O in HPC storage systems. HARMONI leverages a graph neural network (GNN) to encode task-SSD dependencies and a hybrid interference predictor to adapt to hardware and I/O variations. Experimental results across diverse HPC and AI workloads demonstrate that HARMONI significantly reduces average makespan by up to 90% compared with state-of-the-art schedulers, effectively bridging the gap between evolving storage hardware and the dynamic I/O demands of modern HPC systems.

Erasure coding (EC) is increasingly used in distributed storage systems, such as CephFS, to achieve high data availability with minimal storage overhead. In CephFS, the Object Storage Daemons (OSDs) retrieve the EC chunks and reconstruct file blocks. However, this process performs poorly, especially in geo-distributed environments. While prefetching is commonly used to improve read throughput, in concurrent read scenarios it can create an imbalance in the data load across OSDs and in the data exchanged between Datacenters (DCs), leading to contention on a few network links and poor performance. In this paper, we present THEIA, a data access and retrieval framework that integrates prefetching for reads in geo-distributed, erasure-coded storage systems. First, THEIA mitigates the effects of imbalanced data loads across OSDs by implementing a distributed prefetching service that dynamically allocates and partitions caches across OSDs independently. Second, THEIA fetches chunks based on network load, thereby improving prefetching speed. Our evaluation shows that THEIA can deliver up to 2.27× the throughput of standard concurrent reads and improve the performance of data-intensive workflows by up to 2.8×, but it requires additional memory resources.

Erasure coding (EC) is a well-known redundancy technique used to prevent data loss in distributed systems. They typically reduce storage overhead significantly compared to replication. However, they are more complex to use than replication in terms of both deployment and data repair in the event of node failure, particularly in the context of decentralized distributed systems. In this paper, we introduce ALTOCUMULUS, an erasure-coded storage system that efficiently and effectively integrates Reed-Solomon codes on top of the InterPlanetary File System (IPFS). ALTOCUMULUS implements efficient parallel data writing and dynamic data retrieval that adapts its chunk selection according to nodes' performance at runtime. It also provides optimized data repair by enabling nodes to perform repair tasks evenly in parallel. Evaluation results show that ALTOCUMULUS significantly improves write performance compared to replication and alternative EC implementations in IPFS, reaching 89.7% and 92.9%, respectively. In addition, ALTOCUMULUS achieves comparable performance to replication for reads and data repair, while outperforming it in heterogeneous environments and when repairing large datasets.

Container orchestration systems, such as Kubernetes, streamline containerized application deployment. As more and more applications are being deployed in Kubernetes, there is an increasing need for rescheduling - relocating a running pod to different nodes - due to system upgrades, node failures, and load-balancing optimizations. Live migration, which transfers services from source nodes to target nodes with minimal downtime, is the ideal support for rescheduling. However, implementing live migration for pods that run stateful services is challenging, because Kubernetes manages pods as stateless. First, the current pod's network namespace initialization process causes a mismatch in the network state between the migrated pod and internal containers. Second, migrating the memory state results in extended downtime. Third, Kubernetes operations on pods do not consider preserving the state of the pods. Therefore, we propose KubeSPT to achieve live migration of stateful pods in rescheduling scenarios. First, we synchronize the network state of pods and internal containers by controlling packet flow and implement fast service redirection. Second, we introduce a Hot Data and Lazy-Restore method for memory restoration to reduce migration downtime. Finally, we decouple pod migration operations from other Kubernetes operations to ensure compatibility with live migration. Experimental results show that KubeSPT reduces downtime by 86% -93% compared to current rescheduling methods.

Containers, renowned for their lightweight nature and flexibility, have seen growing adoption for deploying edge services such as web applications. However, existing cloud-oriented container deployment frameworks fail to address the unique challenges of edge environments, including geographical distribution, device heterogeneity, and resource constraints. This oversight leads to suboptimal performance for latency-sensitive edge services like HPC/AI-powered autonomous driving and edge gaming, which demand rapid startup and immediate responsiveness. Our investigation demonstrates that current on-demand image solutions require excessive client-registry communication, resulting in prolonged Round-Trip Time (RTT) - a particularly severe limitation in geographically distributed edge platforms. Furthermore, we observe that the user-space file system (e.g., FUSE), typically employed to handle device heterogeneity, introduces substantial overhead to the native I/O stack. More critically, our findings reveal that on-demand image solutions exacerbate storage pressure on resource-constrained edge devices. To overcome these challenges, we introduce EDDE, an edge-optimized container deployment framework that redesigns the on-demand image pipeline. EDDE achieves up to 9.8x higher deployment efficiency than standard Docker. When compared to state-of-the-art on-demand solutions, EDDE delivers containers 147% faster on average, reduces native I/O latency by up to 28%, and decreases storage usage by an average of 34%.

Erasure Coding (EC) has recently been integrated and deployed in the Hadoop Distributed File System (HDFS) to provide the same fault tolerance guarantees as replication, but with significantly less storage overhead. When EC is used, data reads typically involve only data chunks. In this paper, we study the effect of data chunk distribution on the performance of reads and data-intensive applications, and present the design and evaluation of an erasure coding aware (EC-aware) block placement that balances the distribution of data chunks across nodes. Experimental results show that EC-aware block placement can reduce the execution time of Sort and WordCount applications by up to 25%.

Solid State Drives (SSDs) are widely used in data-intensive scenarios due to their high performance and decreasing cost. However, in shared environments, concurrent workloads can interfere with each other, leading to a violation of Quality of Service (QoS). While QoS mechanisms like fairness guarantees and latency constraints have been integrated into SSDs, existing transaction processing frameworks offer limited QoS guarantees and can significantly degrade overall performance in a shared environment. The reason is that the internal components of an SSD, originally designed to exploit parallelism, struggle to coordinate effectively when QoS mechanisms are applied to them. This article proposes a novel QoS -enhanced transaction pro cessing framework, called QoS-pro, which enhances QoS guarantees for concurrent workloads while maintaining high parallelism for SSDs. QoS-pro achieves this by redesigning transaction processing procedures to fully exploit the parallelism of shared SSDs and enhancing QoS-oriented transaction translation and scheduling with parallelism features in mind. In terms of fairness guarantees, QoS-pro outperforms state-of-the-art methods by achieving 96% fairness improvement and 64% maximum latency reduction. QoS-pro also shows almost no loss in throughput when compared with parallelism-oriented methods. Additionally, QoS-pro triggers the fewest Garbage Collection (GC) operations and minimally affects concurrently running workloads during GC operations.

The high-performance computing I/O stack has been complex due to multiple software layers, the inter-dependencies among these layers, and the different performance tuning options for each layer. In this complex stack, the definition of an “I/O access pattern” has been reappropriated to describe what an application is doing to write or read data from the perspective of different layers of the stack, often comprising a different set of features. It has become common to have to redefine what is meant when discussing a pattern in every new study, as no assumption can be made. This survey aims to propose a baseline taxonomy, harnessing the I/O community’s knowledge over the past 20 years. This definition can serve as a common ground for high-performance computing I/O researchers and developers to apply known I/O tuning strategies and design new strategies for improving I/O performance. We seek to summarize and bring a consensus to the multiple ways to describe a pattern based on common features already used by the community over the years.

The dynamic workload and latency sensitivity of DNN inference drive a trend toward exploiting serverless computing for scalable DNN inference serving. Usually, GPUs are spatially partitioned to serve multiple co-located functions. However, existing serverless inference systems isolate functions in separate monolithic GPU runtimes (e.g., CUDA context), which is too heavy for short-lived and fine-grained functions, leading to a high startup latency, a large memory footprint, and expensive inter-function communication. In this paper, we present StreamBox, a new lightweight GPU sandbox for serverless inference workflow. StreamBox unleashes the potential of streams and efficiently realizes them for serverless inference by implementing fine-grain and auto-scaling memory management, allowing transparent and efficient intra-GPU communication across functions, and enabling PCIe bandwidth sharing among concurrent streams. Our evaluations over real-world workloads show that StreamBox reduces the GPU memory footprint by up to 82% and improves throughput by 6.7X compared to state-of-the-art serverless inference systems.

This special issue covers emerging research and development challenges in serverless computing, ranging from the development of an allocation and scheduling simulator to evaluate various orchestration policies, to support for scalable and cost-effective deployment of serverless machine learning models in heterogeneous edge environments. The articles also demonstrate how the serverless computing paradigm has been deployed across different system architectures, from edge-only deployments to those that make use of a combination of edge and cloud environments. There is clear enthusiasm in the systems community in utilizing this paradigm for a wide range of applications.

Stream data processing is a widely used technology for analysing IoT-generated data shortly after being produced, and delivering timely insights about them. Executing such analysis in geo-distributed platforms enables shorter delays between data production and processing and fewer disturbances due to potential instability of long-distance networks, while retaining the ability to scale the processing capacity up and down according to the demand. However, current stream processing systems were designed for environments made of homogeneous servers connected together using high-speed network links. We experimentally study the performance of Apache Flink coupled with the Gesscale auto-scaler in conditions which resemble those of geo-distributed platforms. We demonstrate that Flink’s backpressure mechanism should not be used as the only trigger for rescaling operations in heterogeneous network conditions. Raw performance, as well as performance predictability, also degrade quickly in the presence of stateful data processing operators and/or high network latency between the processing nodes.

The continuous growth in data volume increases the interest in using peer-to-peer (P2P) systems not only to store static data (i.e., immutable data) but also to store and share mutable data -- data that are updated and modified by multiple users. Unfortunately, current P2P systems are mainly optimized to manage immutable data. Thus, each modification creates a new copy of the file, which leads to a high "useless" network usage. Conflict-free Replicated Data Types (CRDTs) are specific data types built in a way that mutable data can be managed without the need for consensus-based concurrency control. A few studies have demonstrated the potential benefits of integrating CRDTs in the InterPlanetary File System (IPFS), an open-source widely used P2P content sharing system. However, they have not been implemented and evaluated in a real IPFS deployment. This paper tries to fill the gap between theory and practice and provides a quantitative measurement of the performance of CRDTs in IPFS. Accordingly, we introduce IM-CRDT, an implementation of CRDTs in IPFS that focuses on the simple data type (i.e., Set); and carry out extensive experiments to verify whether CRDTs can efficiently be utilized in IPFS to handle mutable data. Experiments on Grid'5000 show that IM-CRDT reduces the data transfer of an update by up to 99.96% and the convergence time by 54.6%-62.6%. More importantly, we find that IM-CRDT can sustain low convergence time under concurrent updates.

Machine Learning (ML) workflows are increasingly deployed on serverless computing platforms to benefit from their elasticity and fine-grain pricing. Proper resource allocation is crucial to achieve fast and cost-efficient execution of serverless ML workflows (specially for hyperparameter tuning and model training). Unfortunately, existing resource allocation methods are static, treat functions equally, and rely on offline prediction, which limit their efficiency. In this paper, we introduce CE-scaling – a Cost-Efficient autoscaling framework for serverless ML work-flows. During the hyperparameter tuning, CE-scaling partitions resources across stages according to their exact usage to minimize resource waste. Moreover, it incorporates an online prediction method to dynamically adjust resources during model training. We implement and evaluate CE-scaling on AWS Lambda using various ML models. Evaluation results show that compared to state-of-the-art static resource allocation methods, CE-scaling can reduce the job completion time and the monetary cost by up to 63% and 41% for hyperparameter tuning, respectively; and by up to 58% and 38% for model training.

Graph processing is a popular computing model for big data analytics. Emerging big data applications are often maintained in multiple geographically distributed (geo-distributed) data centers (DCs) to provide low-latency services to global users. Graph processing in geo-distributed DCs suffers from costly inter-DC data communications. Furthermore, due to increasing privacy concerns, geo-distribution imposes diverse, strict, and often asymmetric privacy regulations that constrain geo-distributed graph processing. Existing graph processing systems fail to address these two challenges. In this paper, we design and implement PGPregel , which is an end-to-end system that provides privacy-preserving graph processing in geo-distributed DCs with low latency and high utility. To ensure privacy, PGPregel smartly integrates Differential Privacy into graph processing systems with the help of two core techniques, namely sampling and combiners , to reduce the amount of inter-DC data transfer while preserving good accuracy of graph processing results. We implement our design in Giraph and evaluate it in real cloud DCs. Results show that PGPregel can preserve the privacy of graph data with low overhead and good accuracy.

In many data-intensive applications, workflow is often used as an important model for organizing data processing tasks and resource provisioning is an important and challenging problem for improving the performance of workflows. Recently, system variations in the cloud and large-scale clusters, such as those in I/O and network performances and failure events, have been observed to greatly affect the performance of workflows. Traditional resource provisioning methods, which overlook these variations, can lead to suboptimal resource provisioning results. In this article, we provide a general solution for workflow performance optimizations considering system variations. Specifically, we model system dynamics as time-dependent random variables and take their probability distributions as optimization input. Despite its effectiveness, this solution involves heavy computation overhead. Thus, we propose three pruning techniques to simplify workflow structure and reduce the probability evaluation overhead. We implement our techniques in a runtime library, which allows users to incorporate efficient probabilistic optimization into existing resource provisioning methods. Experiments show that probabilistic solutions can improve the performance by up to 65 percent compared to state-of-the-art static solutions, and our pruning techniques can greatly reduce the overhead of our probabilistic approach.

Speculative execution can significantly improve the performance of Big Data applications by launching other copies of stragglers (slow tasks). Stragglers detection plays an important role in the effectiveness of speculative execution. The methods employed to detect stragglers use the information extracted from the last received heartbeats which may be outdated when triggering detection. This, in turn, can mislead Big Data analytic systems to make wrong detection with high inaccuracy. To shed the light on this issue, we carry out extensive simulations to identify how heartbeat arrival, task starting times, and detection methods impact the accuracy of stragglers detection in Big Data analytic systems. We reveal that the asynchrony in heartbeat arrivals not only lead to marking normal tasks as stragglers (false positives) but can also result in overlooking real stragglers (false negatives).

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Solid State Devices (SSDs) have been widely adopted in containerized cloud platforms as they provide parallel and highspeed data accesses for critical data-intensive applications. Unfortunately, the I/O stack of the physical host overlooks the layered and independent nature of containers, thus I/O operations require expensive file redirect (between the storage driver, Overlay2/EXT4, and the virtual file system, VFS) and are scheduled sequentially. Moreover, containers suffer from significant I/O contention as resources at the native file system are shared between them. This paper presents a Container-aware I/O stack (CAST). CAST is made up of Layer-aware VFS (LaVFS) and Container-aware Native File System (CaFS). LaVFS locates files based on layer information and enables simultaneous Copy-on-Write (CoW) operations and thus avoids the overhead of searching and modifying files. CaFS, on the other hand, provides contention-free access by designing fine-grain resource allocation at the native file system. Experimental results using a NVMe SSD with micro-benchmarks and real-world applications show that CAST achieves 216%-219% (38%-98%, respectively) improvement over the original I/O stack.