Edge servers usually store collected data in cloud servers and use deduplication techniques to remove redundancy. However, edge servers can also perform deduplication during data collection. This requires transferring fingerprints from the cloud servers to the edge servers for assistance. Since the large volume of fingerprint data on the cloud server, for example, 1 PB of data corresponds to 8 TB of fingerprints, transferring all fingerprints to the edge servers is impractical. Therefore, we propose a fingerprint offloading strategy. Only a small amount of fingerprints and data chunks needed for edge deduplication are offloaded from the cloud server to the edge server, enabling cloud-edge collaborative deduplication. The general process is as follows: First, the edge server collect a large amount of data from various devices, divides the data into chunks, and calculates fingerprints to identify unique data chunks and fingerprints. Then, the edge server upload the fingerprints to the cloud server. The cloud server check the fingerprints and offload the data chunks corresponding to existing fingerprints back to the edge server. Upon receiving these data chunks, the edge server performs thorough deduplication. Finally, the deduplicated data chunks are uploaded to the cloud server, ensuring that only unique data is transmitted to the cloud server. Experiments used chunks ranging from 1 KB to 16 KB, with an average size of 4 KB, and employed three real backup datasets. The results showed that the edge server computation time was reduced by 48.1%, metadata storage was reduced by 98.1%, and the upload volume from the edge server to the cloud server decreased by 87.1%. The size of fingerprints and data chunks offloaded to the edge server ranged from 9.4 MB to 1624.2 MB.
With advantages of scalability and low cost, cloud object storage has become the preferred storage solution for massive data. However, object storage has its own defects, such as poor performance of small I/Os, which restricts object storage from being widely used in some scenarios like Big Data. In this paper, we propose GooseFS, a distributed high-performance cache service built on top of object storage, which accelerates the access to object storage under the storage-compute separation architecture. GooseFS makes great enhancements on small I/Os performance as well as metadata performance for data stored in object storage. GooseFS introduces three key designs: (1) Compute-Side Cache, which reduces the data access latency under random small I/O with multi-granularity cache management and short circuit read mechauism. (2) Storage-Side Cache, which improves the throughput with a high-performance SSD cache pool. (3) Metadata Acceleration, which significantly improves the performance of metadata operations through special metadata organization and lock-free strong consistency cache. Experiment show that compared with native object storage, GooseFS increases the throughput of small random I/O by 5x -9 x and improves metadata operation performance by at least 3.9 x, meeting the performance requirements of various workloads.
Solid State Drive cache, implemented as on-board shared DRAM memory, can significantly enhance I/O performance by caching frequently accessed data. Although SSD caching strategies for single I/O data flows have been extensively explored, studies on cache partitioning to optimize parallel I/O in an SSD are scarce. In this paper, we present a novel dynamic cache partitioning approach designed to improve overall performance of multi-parallel I/O data flows by minimizing performance degradation of cache pollution and resource contention. By dynamically adjusting cache partition sizes for each data flow by considering cache sensitivity on performance, our strategy seeks to determine the optimal cache partition sizes to maximize overall I/O throughput. We implemented the strategy in the SSD simulator MQSim and evaluated its performance using various synthetic and real-world workloads. Our experimental results indicate that our dynamic cache partitioning strategy achieves an overall throughput increase of up to 33.22% compared to shared cache methods and outperforms static cache partitioning strategies by up to 21.19%.
As a complementary technique to data deduplication, delta compression calculates and stores differences between highly similar but non-duplicate chunks to reduce storage costs in storage systems. A key aspect of efficient post-deduplication delta compression is resemblance detection, which identifies similar reference chunks to achieve significant data reduction. However, our evaluation of existing methods indicates that the throughput of resemblance detection is the system bottleneck due to the computationally intensive nature of feature extraction, which is carried out sequentially. Additionally, these methods suffer from low similarity between detected chunks and miss some potential similar chunks, reducing the redundancy elimination effect. This paper introduces ERD, a technique aimed at optimizing resemblance detection, featuring two key advancements. First, it employs AVX-512 instructions to generate rolling hashes in parallel based on a novel overlapping sub-chunk partition strategy, and optimizes the linear transformation process, thereby reducing computational overhead while eliminating computational errors introduced by parallelization. Second, it introduces a locality-aware feature matching technique to further detect potentially similar chunks and enhance the detection capabilities. Experimental results show that ERD accelerates feature extraction for resemblance detection by 2.1x similar to 2.18x (the state-of-the-art method Finesse) and 5.66x similar to 6.21x (N-trans) while increasing the delta compression ratio by up to 32.5% (18.2% on average) (Finesse).
Node failures in decentralized distributed storage systems are common. To ensure data availability, these systems employ data redundancy mechanisms, typically relying on repli-cas. This paper proposes a decentralized data redundancy scheme based on erasure coding, with Reed-Solomon and IPFS as exam-ples. Compared with replication, the proposed scheme reduces storage space and enhances fault tolerance. Files are sharded and encoded across multiple nodes, avoiding the high redundancy of replicas. Users can retrieve any K shards from N nodes to reconstruct the original files. This erasure coding optimization combines efficient data exchange among decentralized nodes with erasure coding technology, significantly reducing storage space compared with the replica mechanism. The implementation involves truncating and sharding the blocks within the Merkle DAG generated by files, enabling flexible adjustments to the code rate of erasure codes and the allocation of storage nodes based on user needs and available resources. This method achieves a balance between storage efficiency and data availability.
Cloud object storage systems provide massive data storage capabilities where data is stored in different storage clusters. Storing data according to access characteristics efficiently in different clusters is a challenging task. Methods considering past data access frequency bring the problem of low storage utilization and load imbalance. We propose a Cost-Aware Migration Scheme for cloud object storage systems(CAMS) based on object hotness and life cycle to improve the utilization of cloud object storage systems and reduce the Total Cost of Ownership(TCO). CAMS establishes an accurate object hotness standard, it uses object hotness and lifecycle prediction to guide data migration. CAMS was tested using real-world datasets from production cloud object storage system, the results show that CAMS strategies outperform Cold, CoinFlip and RejectX strategies with gains of up to 19.79% on the estimated TCO.
WriteBooster is widely adopted as a non-volatile write buffer to enhance user experience for smartphones. How-ever, the host suffers sub-optimal write performance when using WriteBooster due to the lack of utilization of rich semantic infor-mation. With the zoned storage being included in smartphones, WriteBooster design presents new opportunities. In this paper, we propose Zoned-WB, a WriteBooster management scheme based on zoned storage to utilize the rich semantic information on the host to improve user experience. Specifically, Zoned- Wbincludes two parts, zoned storage-based WB and foreground request-aware WB. First, the zoned storage-based WB is aimed at designing WriteBooster management scheme based on zoned storage. Second, foreground request-aware WB is designed to adaptively adjust the capacity quota for different types of requests in Writebooster based on rich semantic information on the host. We evaluate Zoned-WB on a zoned storage emulator with workloads collected from smartphones. Evaluation results show that Zoned- Wbcan effectively improve user experience.
With SSDs gradually replacing HDDs as the main-stream storage media in modern large-scale data centers, SSD failure analysis has become increasingly important. We conducted an in-depth data-driven analysis of the failure characteristics of an SSD-based data center in Alibaba based on the failure datasets in 2018 and 2019 and SMART logs on December 31, 2019. Our objectives focus on 3 “W”s, illustrated as follows. What factors influence the occurrence of failures? Why do these factors influence the occurrence of failures? How can companies reduce the occurrence of SSD failures? We expect that our findings and analysis can benefit future SSD-based storage system designs.
Data I/O poses a significant bottleneck for distributed deep learning applications. Utilizing computing-node attached storage as a cache has become a prevalent solution to this problem. Given the large size of training datasets and the limited capacity of a single-node local storage, training samples are loaded in shards across computing nodes, requiring a deep learning job on one node to access samples from neighboring nodes. Compared to local node access, these neighboring accesses are highly inefficient for small samples. To mitigate this issue, we propose a node-aware prefetch algorithm that greedily fetches samples from neighboring nodes. Evaluation results show that our approach improves neighboring access performance by 60x for small samples of 64B size. For large samples of 1MB size, our approach still exhibits a 26% improvement.
Garbage collection (GC) is critical to improving 3D NAND flash memory space utilization. However, GC is very time-consuming for migrating valid data for error correction, which leads to a sharp decline in system performance. Copyback is an advanced command that can be used to accelerate data migration in GC. However, existing copyback-based GC scheme can not guarantee data reliability for various types of flash pages and cause a large decoding latency in the error correction process. In this paper, we first evaluate the copyback error characteristics that measure the quantitative relationships between the number of copybacks and the raw bit error rate (RBER) of upper, middle, and lower pages of triple level cell NAND flash through practical testing, respectively. Then we propose a multi-threshold copyback-based GC scheme (MCBGC). Under the specific RBER limit, the copyback thresholds of upper, middle, and lower pages are determined respectively based on the copyback error characteristics. Experimental results show that compared with the existing copyback-based GC scheme, the write latency of the proposed scheme can be improved by up to 17.7%.
It is essential to utilize a large number of I/O workloads to analyze commodity system performance or simulate scientific phenomena in high-performance scientific computing. I/O traces are often unavailable at scale due to trace storage overhead, privacy concerns, and the performance impact of trace instrumentation. We study how to generate sufficiently representative I/O workloads using Generative Adversarial Networks (GANs). The best GAN architecture can generate I/O workloads with maximum mean discrepancy (MMD) as low as 0.015-0.05, which implies the synthetic I/O workloads have successfully learned the potential distribution of real I/O traces. We demonstrate that the performance similarity between the original I/O trace and the generated I/O workload through trace replay can be 90.36%-97.32%.
With the increasing demand for artificial intelligence (AI), cluster jobs require high-performance GPU instances and often face stringent deadline constraints. Previous studies have proposed various scheduling strategies to minimize job completion time and instance costs, assuming accurate prediction of job execution time across different instances. However, such assumptions are often unrealistic due to the inherent unpredictability of job execution time. Additionally, jobs typically involve large data sets and incur substantial cold-start time, leading to increased job completion latency and degradation of user quality of service. To tackle these challenges, we introduce an algorithm named Exponential Weighting Scheduling (EWS) for GPU clusters, which employs an online learning approach based on the combinatorial multi-armed bandit (CMAB) framework. EWS dynamically updates the probability distribution of decision spaces using real-time information on job performance across different instances and employs randomized decision-making for scheduling. Furthermore, we provide theoretical proof that this online strategy guarantees sublinear regret in terms of performance. Extensive experiments validate that our algorithm significantly enhances user quality of service and reduces instance utilization costs compared to other state-of-the-art baselines.
With vertical stacking, 3D NAND flash memory can achieve continuous capacity growth. However, as the number of stacked layers in a flash block increases, the endurance variation between the stacked layers becomes more and more significant due to process variation, which will seriously affect the lifetime of 3D NAND flash memory. We investigated the endurance variation characteristics between layers and divided the stacked layers into top, middle, and bottom layers according to the endurance characteristics. We found that the endurance of the bottom layer pages is much weaker than that of the other two layers, in response to this endurance variation feature, we proposed a new layer-aware write strategy, called LA-Write. First of all, the write-skip unit in LA-Write will reduce the wear pressure of the pages through write-skip operations. Secondly, LA-Write maintains a layer-aware table, which stores the probability of pages in different layers performing write-skip operation. Setting the probability of the bottom pages to the highest will result in more write-skip operations on the bottom layers, mitigating endurance variations between layers. Experimental results show that LA-Write can increase SSD lifetime by an average of 31%.
Scheduling and placement for deep learning (DL) jobs on GPU clusters is essential to improve quality of service and reduce operational cost. Scheduling involves determining the execution order of waiting jobs, and placement entails selecting appropriate computing nodes to execute the jobs. In this paper, we propose a RL-based scheduling and placement method for DL jobs on large-scale GPU clusters. The key idea is to employ two RL agents with adaptable policy networks. These networks are capable of reducing computational complexity and supporting a flexible action space. First, we regard the scheduling and placement as unified sequence selection processes, i.e., selecting an element from the candidate sequence. And they are tackled by two similar RL agents. Second, we develop an encoder-only Transformer-based policy network. This network can manage variable-length sequences and offer a flexible action space. Third, we propose a novel sequence filtering approach. This approach can exclude those job sequences offering limited learning value, so as to enhance the agents' training efficiency. We evaluate our method's performance using a real production-level job trace, comparing it with several heuristic and RL methods. Our method achieves an average improvement of up to 1.30 x in average job completion time and up to 1.11 x in energy consumption.
Turbulence is a three-dimensional fluid motion state with multiple scales and mutual coupling in time and space. Studying turbulence phenomena is crucial for the design and drag reduction of aircraft and engines. In recent years, with the powerful computing power of computers, data-based turbulence research methods have become a hot topic, such as the Direct Numerical Simulation (DNS) method, which is receiving attention. However, compared to Large Eddy Simulation (LES), the amount of data generated by direct numerical simulation methods far exceeds that of traditional LES, which brings difficulties in data storage. Abroad, multiple research institutions in Europe, the United States, and Japan have established multiple turbulence data sharing platforms, but the construction of turbulence data sharing platforms in China is still in a blank stage. This article first analyzes the characteristics of turbulence science big data, and then starts from the practical problems faced by domestic research institutions when sharing and using data, designs and implements the first independently controllable turbulence data sharing platform in China. Currently, the platform has integrated 180TB of data from five domestic universities and research institutions, providing an important resource sharing platform and data research tool for turbulence researchers in China.
When running deep learning training jobs, in order to prevent training loss due to software/hardware failures, a checkpointing mechanism is usually used to periodically store snapshots of the training model in non-volatile memory to facilitate recovery from failures. However, the recovery effect of this method is closely related to the checkpointing period and the failure time, and in the worst case, even nearly one period of training accuracy will be lost. With increasing model sizes, the need to develop more fine-grained model fault recovery methods to reduce model fault loss is more urgent for deep learning training tasks that use checkpointing mechanisms as fault recovery guarantees. We present TranLogs, a checkpoint failure recovery method combined with training logs for lossless failure recovery when training deep learning models. TranLogs proposes to use logging to supplement the checkpoint file and track the iterative process of model training parameters. For the situation where the checkpoint file is not updated in time and encounters model failures, resulting in a large loss during recovery, TranLogs can handle it effectively. The best-case scenario achieves about 24% and 6% improvement in the loss and accuracy of the recovered training state, respectively, which is closer to the state already achieved by the model training at the moment of failure.
In distributed storage systems, erasure coding is a well-proven and effective fault-tolerant technique that is extensively employed. The escalating demands for storage performance from enterprise users have markedly intensified the bandwidth requirements associated with erasure coding. ARM-based CPUs exhibit limitations in their performance regarding erasure coding. The use of FPGA to offload erasure coding has been covered in earlier studies. However, FPGA-based offloading schemes for erasure coding is not suitable for every storage scenario. Both performance and cost become significant challenges. To address these challenges, we propose an FPGA-based storage system acceleration architecture that enables both high performance and low cost. By constructing an EC evaluation model, a quantitative analysis mechanism is used to determine the most useful scenario. We implement a co-designed prototype of hardware and software that consists of an FPGA for erasure coding offloading and a unique software acceleration layer installed in the IEIT storage cluster. It could increase throughput by up to 2.67 times.
It is efficient to accelerate SQL queries by utilizing Computational Storage Device (CSD). Since current CSDs have limited processing capacity and cannot accelerate all SQL queries, existing schemes typically offload filter operators to significantly reduce data movement to the computational storage device. However, leaving all filter operators in SQL queries to the computational storage device directly leads to poor performance. To address these problems, we propose the Self-Adaptive Query Offloading (SAQO) strategy for offloading filtering operators. The SAQO strategy dynamically chooses the best execution location for filtering operators by considering the selectivity of the operator, the computational storage device's load, and the task characteristics, which ultimately accelerates the speed of executing data query tasks. The real-world applications Spark SQL and TPC-H are used to evaluate the SAQO strategy. The SAQO strategy reduces the execution time of data query tasks by 58.7% compared to the host execution strategy and 9.4% compared to the state-of-the-art offloading strategy.
Thread scheduling is a core functionality of operating systems, optimizing CPU resource utilization through dedicated algorithms. However, modifying kernel-level scheduling in Linux is complex and error-prone. User-space scheduling offers a promising alternative, yet existing implementations struggle with speed, flexibility, and robustness. This paper introduces F4, a user-space thread scheduling framework that shifts scheduling algorithm deployment from the kernel to user space. By allowing the kernel to delegate scheduling decisions through interfaces like system calls and eBPF hooks, F4 enables user space to develop and implement its own scheduling strategies. This separation of scheduling mechanisms and policy definitions enhances efficiency and adaptability. F4 improves communication through message queues and the shoot system, accelerates execution with inter-processor interrupts (IPI), and increases interference resistance through dynamic priority adjustments. These innovations allow F4 to achieve fast, flexible, and stable-fortified thread scheduling in user space, addressing the limitations of current user-space scheduling solutions.
Locally repairable codes (LRC) have dominated the coding design of erasure coding in wide stripes due to their excellent local property and low bandwidth requirement. In distributed storage systems, repair degree usually refers to the number of helper nodes that need to be contacted in data repair process. As the stripe becomes wider, the probability of multiple nodes failing simultaneously in the LRC increases, which leads to a significant increase in the repair degree. So how to improve the existing schemes to improve code reliability and reduce the repair degree in case of multiple failures remains to be further explored. In this paper, we present a new family of locally repairable codes, TFR-LRC (Locally Repairable Codes for seeking the Trade-off between Fault tolerance and Repair degree), which can achieve different trade-offs between fault tolerance and repair cost. TFR-LRC can sacrifice a little bit of storage cost for lower repair degree, and can sacrifice a little bit of repair cost for higher fault tolerance. In order to verify the feasibility of TFR-LRC, we conducted simulation experiments. The results show that, compared to the existing LRC schemes, TFR-LRC can greatly reduce the repair degree in case of multiple failures by sacrificing a little bit of storage cost without changing the traditional LRC structures, and meanwhile TFR-LRC can make higher fault tolerance under the same parameter settings by sacrificing a little bit of repair cost.
