Modern applications span multiple clouds to reduce costs, avoid vendor lock-in, and leverage low-availability resources in another cloud. However, standard object stores operate within a single cloud, forcing users to manually manage data placement across clouds, i.e., navigate their diverse APIs and handle heterogeneous costs for network and storage. This is often a complex choice: users must either pay to store objects in a remote cloud, or pay to transfer them over the network based on application access patterns and cloud provider cost offerings. To address this, we present SkyStore, a unified object store that addresses cost-optimal data management across regions and clouds. SkyStore introduces a virtual object and bucket API to hide the complexity of interacting with multiple clouds. At its core, SkyStore has a novel TTL-based data placement policy that dynamically replicates and evicts objects according to application access patterns while optimizing for lower cost. Our evaluation shows that across various workloads, SkyStore reduces the overall cost by up to 6x over academic baselines and commercial alternatives like AWS multi-region buckets. SkyStore also has comparable latency, and its availability and fault tolerance are on par with standard cloud offerings.

Regulated enterprises often seek to extend their workloads into the cloud, but are impeded by integration concerns relating to security, governance and compliance. Further, enterprises running mission-critical applications, face throughput and latency challenges due to cloud integration overheads. We present Hybrid Cloud Connector to accelerate on-prem to cloud integration by handling non-functional aspects in lieu of the application, reducing complexity, and centralizing administration via a policy-driven control point.

This work tackles an inherent conflict between two important trends. The first is the integration of data compression capabilities into many storage systems supporting random I/O on the compressed data. The second is encrypting data at the host, before data is written to the storage, in order to address regulatory and enterprise requirements. This provides end-to-end protection for the data, but since the data arrives encrypted, it prevents the storage from compressing the data. Can compression savings be achieved together with host side encryption without changing the storage protocols or storage backend? In this paper we show that they can. We present Length Preserving Compression (LPC), which combines compression with encryption, to provide the benefits of both. The challenge is to achieve compression savings without overloading the host side with complex data management tasks which come with the fact that compression changes the data layout. We do this by keeping the data layout management capabilities on the storage while compressing and encrypting on the host. Equally important is that LPC works without changes to the compressing storage system or to the standard storage protocols involved. We implemented LPC in Linux dm-crypt and in the Xen blktap encryption and extensively evaluated its performance and impact on security and compression ratios.

Distributed storage systems often triplicate data to reduce the risk of permanent data loss, thereby tolerating at least two simultaneous disk failures at the price of 2/3 of the capacity. To reduce this price, some systems utilize erasure coding. But this optimization is usually only applied to cold data, because erasure coding might hinder performance for warm data. We propose RAIDP---a new point in the distributed storage design space between replication and erasure coding. RAIDP maintains only two replicas, rather than three or more. It increases durability by utilizing small disk "add-ons" for storing intra-disk erasure codes that are local to the server but fail independently from the disk. By carefully laying out the data, the add-ons allow RAIDP to recover from simultaneous disk failures (add-ons can be stacked to withstand an arbitrary number of failures). RAIDP retains much of the benefits of replication, trading off some performance and availability for substantially reduced storage requirements, networking overheads, and their related costs. We implement RAIDP in HDFS, which triplicates by default. We show that baseline RAIDP achieves performance close to that of HDFS with only two replicas, and performs within 21% of the default triplicating HDFS with an update-oriented variant, while halving the storage and networking overheads and providing similar durability.

Until now object storage has not been a first-class citizen of the Apache Hadoop ecosystem including Apache Spark. Hadoop connectors to object storage have been based on file semantics, an impedance mismatch, which leads to low performance and the need for an additional consistent storage system to achieve fault tolerance. In particular, Hadoop depends on its underlying storage system and its associated connector for fault tolerance and allowing speculative execution. However, these characteristics are obtained through file operations that are not native for object storage, and are both costly and not atomic. As a result these connectors are not efficient and more importantly they cannot help with fault tolerance for object storage. We introduce Stocator, whose novel algorithm achieves both high performance and fault tolerance by taking advantage of object storage semantics. This greatly decreases the number of operations on object storage as well as enabling a much simpler approach to dealing with the eventually consistent semantics typical of object storage. We have implemented Stocator and shared it in open source. Performance testing with Apache Spark shows that it can be 18 times faster for write intensive workloads and can perform 30 times fewer operations on object storage than the legacy Hadoop connectors, reducing costs both for the client and the object storage service provider.

1 SYSTEM DESCRIPTION Introduction. We present a design of the Shared Cloud Object Store, in which the data access control plane is governed by Hyperledger Fabric (HLF) [2]. Our system facilitates decentralized management of cloud storage, shared between mutually distrusting business parties. Finally, we leverage the cloud object store (COS) to store the objects o -chain, anchoring the proof-of-retrievability inside the blockchain, preventing ledger to scale to unmanageable size.

Data is the natural resource of the 21st century. It is being produced at dizzying rates, e.g., for genomics, for media and entertainment, and for Internet of Things. Object storage systems such as Amazon S3, Azure Blob storage, and IBM Cloud Object Storage, are highly scalable distributed storage systems that offer high capacity, cost effective storage. But it is not enough just to store data; we also need to derive value from it. Apache Spark is the leading big data analytics processing engine combining MapReduce, SQL, streaming, and complex analytics. We present Stocator, a high performance storage connector, enabling Spark to work directly on data stored in object storage systems, while providing the same correctness guarantees as Hadoop's original storage system, HDFS. Current object storage connectors from the Hadoop community, e.g., for the S3 and Swift APIs, do not deal well with eventual consistency, which can lead to failure. These connectors assume file system semantics, which is natural given that their model of operation is based on interaction with HDFS. In particular, Spark and Hadoop achieve fault tolerance and enable speculative execution by creating temporary files, listing directories to identify these files, and then renaming them. This paradigm avoids interference between tasks doing the same work and thus writing output with the same name. However, with eventually consistent object storage, a container listing may not yet include a recently created object, and thus an object may not be renamed, leading to incomplete or incorrect results. Solutions such as EMRFS [1] from Amazon, S3mper [4] from Netflix, and S3Guard [2], attempt to overcome eventual consistency by requiring additional strongly consistent data storage. These solutions require multiple storage systems, are costly, and can introduce issues of consistency between the stores. Current object storage connectors from the Hadoop community are also notorious for their poor performance for write workloads. This, too, stems from their use of the rename operation, which is not a native object storage operation; not only is it not atomic, but it must be implemented using a costly copy operation, followed by delete. Others have tried to improve the performance object storage connectors by eliminating rename, e.g., the Direct-ParquetOutputCommitter [5] for S3a introduced by Databricks, but have failed to preserve fault tolerance and speculation. Stocator takes advantage of object storage semantics to achieve both high performance and fault tolerance. It eliminates the rename paradigm by writing each output object to its final name. The name includes both the part number and the attempt number, so that multiple attempts to write the same part use different objects. Stocator proposes to extend an already existing success indicator object written at the end of a Spark job, to include a manifest with the names of all the objects that compose the final output; this ensures that a subsequent job will correctly read the output, without resorting to a list operation whose results may not be consistent. By leveraging the inherent atomicity of object creation and using a manifest we obtain fault tolerance and enable speculative execution; by avoiding the rename paradigm we greatly decrease the complexity of the connector and the number of operations on the object storage. We have implemented our connector and shared it in open source [3]. We have compared its performance with the S3a and Hadoop Swift connectors over a range of workloads and found that it executes many fewer operations on the object storage, in some cases as few as one thirtieth. Since the price for an object storage service typically includes charges based on the number of operations executed, this reduction in operations lowers the costs for clients in addition to reducing the load on client software. It also reduces costs and load for the object storage provider since it can serve more clients with the same amount of processing power. Stocator also substantially increases performance for Spark workloads running over object storage, especially for write intensive workloads, where it is as much as 18 times faster.

We present Stocator, a high performance object store connector for Apache Spark, that takes advantage of object store semantics. Previous connectors have assumed file system semantics, in particular, achieving fault tolerance and allowing speculative execution by creating temporary files to avoid interference between worker threads executing the same task and then renaming these files. Rename is not a native object store operation; not only is it not atomic, but it is implemented using a costly copy operation and a delete. Instead our connector leverages the inherent atomicity of object creation, and by avoiding the rename paradigm it greatly decreases the number of operations on the object store as well as enabling a much simpler approach to dealing with the eventually consistent semantics typical of object stores. We have implemented Stocator and shared it in open source. Performance testing shows that it is as much as 18 times faster for write intensive workloads and performs as much as 30 times fewer operations on the object store than the legacy Hadoop connectors, reducing costs both for the client and the object storage service provider.

Data is the natural resource of the 21st century. It is being produced at dizzying rates, e.g., for genomics by sequencers, for Media and Entertainment with very high resolution formats, and for Internet of Things (IoT) by multitudes of sensors. Object Stores such as AWS S3, Azure Blob storage, and IBM Cloud Object Storage, are highly scalable distributed storage systems that offer high capacity, cost effective storage for this data. But it is not enough just to store data; we also need to derive value from it. Apache Spark is the leading big data analytics processing engine. It runs up to one hundred times faster than Hadoop MapReduce and combines SQL, streaming and complex analytics. In this poster we present Stocator, a high performance storage connector, that enables Spark to work directly on data stored in object storage systems.

Storage class memory (SCM) is an emerging class of memory devices that are both byte addressable, and persistent. There are many different technologies that can be considered SCM, at different stages of maturity. Examples of such technologies include NVDIMM-N, PCM, SttRAM, Racetrack, FeRAM, and others. Currently, applications rely on storage technologies such as Flash memory and hard disks as the physical media for persistency. SCM behaves differently and has significantly different characteristics than these existing technologies. That means there is a need for a fundamental change in the way we program data persistency in applications to fully realize the potential of this new class of device. Previous work such as [1] focuses on designing a filesystem optimized to run over SCM memory. Other projects such as Mnemosyne [4] provide a general purpose API for applications to use SCM memory. Mnemosyne however, does not provide a transaction mechanism flexible enough to allow different SCM updates spanning different parts of the code to be considered one transaction. Our work is focused on employing minimal changes needed to retrofit an existing key-value store to take advantage of SCM technology. We demonstrate these changes on Redis (REmote DIctionary Server) [3], which is a popular key-value store. We show how Redis can be modified to take advantage of these new abilities by allowing the application to manage its own storage in a unique way. Our approach is to use two types of memory technologies (DRAM and SCM) for different purposes in a single application. To optimize the system data capacity, we keep a minimal dataset in persistent memory, while keeping metadata (such as indexing) in DRAM, which can be rebuilt upon failures. Persistency in Redis is currently performed by logging all transactions to an append-only log file (AOF). These transactions can then be replayed to recover from a failure. The transactions are not made persistent until the AOF file is flushed to disk, which is very slow. Flushing after every transaction has a performance impact, but flushing periodically creates a risk of lost data. By using SCM instead of a disk, we can effectively flush every transaction without impacting performance. In order to change Redis to store data objects on the SCM, we must ensure consistency of persistent data even after an unexpected shutdown. To ensure consistency, a modified version of dlmalloc [2] is used for all allocations done on the SCM, and mfence commands are used to overcome unexpected reordering. We model the SCM using a memory mapped file backed on a ramdisk. We compared our changes to Redis using an AOF backed on a ramdisk. Although we don't take into account the latency overheads of accessing the SCM, this comparison gives us a good upper bound of performance benefits of using SCM memory. Our results demonstrate an average latency reduction of 43% and average throughput increase of 75%.