In this study, we first analyze the characteristics of a content addressable storage (CAS) system, and discuss the key problems of keyword search in a CAS system, including data collection, index structure, index storage and result ranking. Then, we choose inter-planetary file system (IPFS), a CAS system, as the experimental platform. On IPFS, we propose a CASearch system, which is a keyword search engine for a CAS system. In the design, we try to ignore the characteristics of IPFS and focus on the common problems of a CAS system. The CASearch node can collect keywords proactively. CASearch binds the keywords by the variable index files, which is a feature of CAS. CASearch also ranks search results by the distance between nodes in the distributed hash table network. Using these solutions, we solve the four basic problems mentioned above. Finally, we evaluate CASearch in terms of data collection, time overhead, storage overhead and result ranking to prove its feasibility and advantages.

针对发布/订阅系统中缓存副本一致性维护问题,首先,对原有基于谣言的一致性维护算法进行改进,提出一种基于轨迹标签的谣言一致性维护算法.该算法通过在更新消息报文中添加节点轨迹信息,避免向已更新节点发送冗余更新消息.其次,为提高系统消息传播的可靠性,设计一种基于轨迹标签的层次化反馈恢复机制,结合发布/订阅系统推拉传输模式,减少反馈消息数目,避免反馈爆炸.模拟实验结果证明,改进的一致性维护算法降低了一致性维护消息开销和时间开销,提高了系统的可靠性和可扩展性.

Although flash storage has largely replaced hard disks in consumer class devices, enterprise workloads pose unique challenges that have slowed adoption of flash in ``performance tier'' storage appliances. In this paper, we describe Purity, the foundation of Pure Storage's Flash Arrays, the first all-flash enterprise storage system to support compression, deduplication, and high-availability. Purity borrows techniques from modern database and key-value storage architectures, and introduces novel storage primitives that have wide applicability to data management systems. For instance, all writes in Purity are monotonic, and deletions are handled using an atomic predicate-based tuple elision primitive. Purity's redundancy mechanisms are optimized for SSD failure modes and performance characteristics, allowing for fast recovery from component failures and lower space overhead than the best hard disk systems. We built deduplication and data compression schemes atop these primitives. Flash changes storage capacity/performance tradeoffs: unlike disk-based systems, flash deployments are rarely performance bound. A single Purity appliance can provide over 7GiB/s of throughput on 32KiB random I/Os, even through multiple device failures, and while providing asynchronous off-site replication. Typical installations have 99.9% latencies under 1ms, and production arrays average 5.4x data reduction and 99.999% availability. Purity takes advantage of storage performance increasing more rapidly than computational performance to build a simpler (with respect to engineering, installation, and management) scale-up storage appliance that supports hundreds of terabytes of highly-available, high-performance storage. The resulting performance and capacity supports many customer deployments of multiple applications, including scale-out and parallel systems, such as MongoDB and Oracle RAC, on a single Purity appliance.

分析和讨论了现有的主要远程复制方案的特点,设计并实现了一种基于LVM的异步实时远程复制系统.它是在现有IP网络的基础上通过纯软件的方式实现的异步在线远程复制系统,无需任何昂贵的硬件支持,为远程数据容灾技术提供了一个简单经济的解决方案.

Driven by the requirements for extremely high bandwidth and large capacity, storage subsystem architectures are undergoing fundamental changes. The object-based storage model, which repartitions the file system functionalities and offloads the storage management functions to intelligent storage devices, is a promising new model. In this model, Object-based Storage Devices (OSDs) manage their own storage space, provide persistent storage, and export an object interface to the data. With abundant on-board computing power, OSDs are able to provide complex services to facilitate high level system designs. This thesis focuses on the design, performance and functionality of an individual OSD in a large distributed object-based storage system, currently being developed in the Storage Systems Research Center at the University of California, Santa Cruz. Based on the file system workload analysis and the expected object workload studies, I extract unique features of object workloads and design an efficient storage manager, named OBFS, for individual OSDs. OBFS employs variable-sized blocks to optimize disk layouts and improve object throughput. Object metadata and attributes are also laid contiguously together with data, which further improves disk bandwidth utilization. The physical storage space is partitioned into fixed-size regions to organize blocks with different sizes together, which effectively reduces file system fragmentation in the long run. A series of experiments was conducted to evaluate OBFS performance compared with two Linux file systems, Ext2 and XFS. The results show that OBFS successfully limits the file system fragmentation even after long term aging. OBFS demonstrates very good synchronous write performance, exceeding those of Ext2 and XFS by up to 80% on both fresh systems and aged systems. Its asynchronous write performance is about 5% to 10% lower than that of Ext2, but 20% to 30% higher than that of XFS on a lightly used disk. On a heavily used disk, OBFS beats both Ext2 and XFS by 20%. For read operations, the performance of OBFS almost doubles that of Ext2 and is only slightly slower than that of XFS. Overall, OBFS achieves 30% to 40% performance improvements over Ext2 and XFS under expected object workloads, and forms a fundamental building block for larger distributed storage systems.

As an emerging nonvolatile secondary storage technology, MEMS-based storage exhibits several desirable properties including high performance, high storage volumic density, low power consumption, low entry cost, and small form factor. However, MEMS-based storage provides a limited amount of storage per device and is likely to be more expensive than magnetic disk. Systems designers will therefore need to make trade-offs to achieve well-balanced designs. We present an architecture in which MEMS devices are organized into MEMS storage enclosures with online spares. Such enclosures are proven to be highly reliable storage building bricks with no maintenance during their economic lifetimes. We also demonstrate the effectiveness of using MEMS as another layer in the storage hierarchy, bridging the cost and performance gap between MEMS storage and disk. We show that using MEMS as a disk cache can significantly improve system performance and cost-performance ratio.

The data storage needs of large high-performance and general-purpose computing environments are generally best served by distributed storage systems. Traditional so lutions, exemplified by NFS, provide a simple distributed storage system model, but cannot meet the demands of high-performance computing environments where a single server may become a bottleneck, nor do they scale well due to the need to manually partition (or repartition) the data among the servers. Object-based storage promises to address these needs through a simple networked data storage unit, the Object Storage Device (OSD) that manages all local storage issues and exports a simple read/write data interface. Despite this simple concept, many challenges remain, including efficient object storage, centralized metadata management, data and metadata replication, and data and metadata reliability. We describe Ceph, a distributed object-based storage system that meets these challenges, providing highperformance file storage that scales directly with the number of OSDs and Metadata servers.

Non-volatile storage technologies such as flash memory, Magnetic RAM (MRAM), and MEMS-based storage are emerging as serious alternatives to disk drives. Among these, MEMS storage is predicted to be the least expensive and highest density, and at about 1 ms access times still considerably faster than hard disk drives. Like the other emerging non-volatile storage technologies, it will be highly suitable for small mobile devices but will, at least initially, be too expensive to replace hard drives entirely. Its non-volatility, dense storage, and high performance still makes it an ideal candidate for the secondary storage subsystem. We examine the use of MEMS storage in the storage hierarchy and show that using a technique called MEMS Caching Disk, we can achieve 30-49% of the pure MEMS storage performance by using only a small amount (3% of the disk capacity) of MEMS storage in conjunction with a standard hard drive. The resulting system is ideally suited for commercial packaging with a small MEMS device included as part of a standard disk controller or paired with a disk.

Parallel scientific applications require high-performanc e I/O support from underlying file systems. A comprehensive understanding of the expected workload is t herefore essential for the design of high-performance parallel file systems. We re-examine the w orkload characteristics in parallel computing environments in the light of recent technology ad vances and new applications.