Large-scale distributed systems often require scalable and fault-tolerant mechanisms to coordinate exclusive access to shared resources such as files, replicas or the primary role. The best known algorithms to implement distributed mutual exclusion with leases, such as Multipaxos, are complex, difficult to implement, and rely on stable storage to persist lease information. In this paper we present {\bf F}LEASE, an algorithm for fault-tolerant lease coordination in distributed systems that is simpler than Multipaxos and does not rely on stable storage. The evaluation shows that {\bf F}LEASE can be used to implement scalable, decentralized lease coordination that outperforms a central lock service implementation by an order of magnitude.

Today's distributed file system architectures scale well to large amounts of data. Their performance, however, is often limited by their metadata server. In this paper, we reconsider the database backend of the metadata server and propose a design that simplifies implementation and enhances performance.In particular, we argue that the concept of log-structured merge (LSM) trees is a better foundation for the storage layer of a metadata server than the traditionally used B-trees. We present BabuDB, a database that relies on LSM-tree-like index structures, and describe how it stores file system metadata.We show that our solution offers better scalability and performance than equivalent ext4 and Berkeley DB-based metadata server implementations. Our experiments include real-world metadata traces from a Linux kernel build and an IMAP mail server. Results show that BabuDB is up to twice as fast as the ext4-based backend and outperforms a Berkeley DB setup by an order of magnitude.

A lease is a token which grants its owner exclusive access to a resource for a defined span of time. In order to be able to tolerate failures, leases need to be coordinated by distributed processes. We present FaTLease, an algorithm for fault-tolerant lease negotiation in distributed systems. It is built on the Paxos algorithm for distributed consensus, but avoids Paxos' main performance bottleneck of requiring persistent state. This property makes our algorithm particularly useful for applications that can not dispense any disk bandwidth. Our experiments show that FaTLease scales up to tens of thousands of concurrent leases and can negotiate thousands of leases per second in both LAN and WAN environments.

In today's Grids, files are usually managed by Grid data management systems that are superimposed on existing file and storage systems. In this paper, we analyze this predominant approach and argue that object-based file systems can be an alternative when adapted to the characteristics of a Grid environment. We describe how we are solving the challenge of extending the object-based storage architecture for the Grid in XtreemFS, an object-based file system for federated infrastructures. Copyright © 2008 John Wiley & Sons, Ltd.

Striping is a technique that distributes (cid:28)le content over multiple storage servers and thereby enables parallel access. In order to be able to provide a consistent view across (cid:28)le data and metadata operations, the (cid:28)le system has to track the layout of the (cid:28)le and know where the (cid:28)le ends and where it contains gaps. In this paper, we present a light-weight protocol for maintaining a consistent notion of a (cid:28)le’s layout that provides POSIX semantics without restricting concurrent access to the (cid:28)le. In an evaluation, we show that the protocol scales and elicit its corner cases.

Striping is a technique that distributes file content over multiple storage servers and thereby enables parallel access. In order to be able to provide a consistent view across file data and metadata operations, the file system has to track the layout of the file and know where the file ends and where it contains gaps. In this paper, we present a light-weight protocol for maintaining a consistent notion of a file's layout that provides POSIX semantics without restricting concurrent access to the file. In an evaluation, we show that the protocol scales and elicit its corner cases.

This paper describes the architecture of XtreemFS, a federated and globally distributed file system. XtreemFS has to cope with scalability problems, a huge number of clients connected over WANs, the related high latency and security problems, as well as distributed file data and metadata. The XtreemFS architecture solves performance, scalability and grid-related problems by a novel design combining techniques used in grid file systems with state-of-the-art cluster file system methods. This article is published as part of the LinuxTag 2007 programme. © 2007 by the authors and the XtreemOS consortium Licensed under the Creative Commons NoDerivs-NonCommercial 1.0 license. XtreemFS is part of the XtreemOS project. XtreemOS is an Integrated Project supported by the European Commission's IST program #FP6-033576.

PP Restricted to other programme participants (including the Commission Services) RE Restricted to a group specified by the consortium (including the Commission Services) CO Confidential, only for members of the consortium (including the Commission Services)

In today's Grids, les are usually managed by Grid data management systems that are superimposed on existing le and storage systems. In this position paper, we analyze this predominant approach and argue that object-based le systems can be an alternative when adapted to the characteristics of a Grid environment. We describe how we are solving the challenge of extending the object-based storage architecture for the Grid in XtreemFS, an object-based le system for federated infrastructures.

Optimistic replication allows mobile workspaces to remain accessible during interrupted network access, but require consecutive handling of any conflicts. When conflict handling requires manual intervention by the user, the system must not require that conflicts are resolved immediatley after their detection. We present a log-based algorithm for reconciling changes to replicated data that supports deferred manual conflict resolution, while sharing the favorable scalability properties of log-based approaches of small storage and communication overhead. Simulation results validate our design and show that it compares favorably in the relevant metrics with version vector-based designs.

This document describes the APIs for WP8's high level data access services and libraries. These prototypes have been deployed on the GridLab testbed.

Optimistic replication accepts changes to replicated data sets without immediate coordination, with the assumption that conflicts can later be resolved by a separate protocol. This protocol will subsequently reconcile changes between replicas, and detect and resolve any conflicts. We present a log-based reconciliation architecture that is designed to record and reconcile changes to data efficiently in terms of communication and storage overhead. Redundancy is eliminated through the use of a log-based storage mechanism. A general data model accommodates a large variety of data types. Because of its storage efficiency, the architecture is especially suited to small data such as database records.

While attribute-value pairs are a popular method to name objects, information retrieval from those attribute-based namespaces is not an easy task. The user has to recall correct attribute names and values and master the syntax and semantics of query formulation. This paper describes hierarchical structures in attribute-based namespaces, shows how to extract them efficiently and evaluates the quality of these structures in an user experiment. It proposes an user interface for browsing attribute-named object sets which makes this task resemble todays file-system browsers and compares the usability of this interface to normal form-based methods in an user study.

In this paper, we propose a new paradigm for local communication between devices in Ubiquitous Computing environments, assuming a multitude of computerized everyday appliances communicating with each other to solve tasks. This paradigm is based on the concept that the location of devices is central for the communication in such a scenario. Devices define their communication scope by spatial criteria. In our paradigm no explicit addressing or identification of communication partners is used. In comparison to traditional communication methods the approach eases routing and discovery problems and can be deployed in a highly dynamic environment without centralized services. We use the term local communication as inter-device communication in a physically restricted local area. This is well distinguish from the terms telecommunication as communication over distance where location information is explicitly hidden. The communication model (RAUM) introduced is based on the observation that humans structure their environment primarily spatially. We show that spatially aware communication, is an efficient method communication in ubiquitous computing environments. We relate the communication architecture of the OSI/ISO reference architecture. An exemplary implementation that realizes a context information system is described. Based on this system several applications (Smart Doorplate, Communication with peripheral devices) have been implemented and evaluated.

This document provides an evaluation of data management application prototypes discussed in D8.9, and reports on the results for WP8's data management and visualization software and services.