Atomic Broadcast, an important abstraction in dependable distributed computing, is usually implemented by solving infinitely many instances of the well-known consensus problem. Some asynchronous consensus algorithms achieve the optimal latency of two (message) steps but cannot guarantee this latency even in good runs, those with timely message delivery and no crashes. This is due to collisions, a result of concurrent proposals. Collision-fast consensus algorithms, which decide within two steps in good runs, exist under certain conditions. Their direct application to solving atomic broadcast, though, does not guarantee delivery in two steps for all messages unless a single failure is tolerated. We show a simple way to build a fault-tolerant collision-fast Atomic Broadcast algorithm based on a variation of the consensus problem we call M-Consensus. Our solution to M-Consensus extends the Paxos protocol to allow multiple processes, instead of the single leader, to have their proposals learned in two steps.

Facebook recently deployed Facebook Messages, its first ever user-facing application built on the Apache Hadoop platform. Apache HBase is a database-like layer built on Hadoop designed to support billions of messages per day. This paper describes the reasons why Facebook chose Hadoop and HBase over other systems such as Apache Cassandra and Voldemort and discusses the application's requirements for consistency, availability, partition tolerance, data model and scalability. We explore the enhancements made to Hadoop to make it a more effective realtime system, the tradeoffs we made while configuring the system, and how this solution has significant advantages over the sharded MySQL database scheme used in other applications at Facebook and many other web-scale companies. We discuss the motivations behind our design choices, the challenges that we face in day-to-day operations, and future capabilities and improvements still under development. We offer these observations on the deployment as a model for other companies who are contemplating a Hadoop-based solution over traditional sharded RDBMS deployments.

Agents in agreement protocols play well-distinct roles. Proposers propose values to the acceptors , which will accept proposals and inform the learners so they detect that an agreement has been reached. A fourth role is that of the coordinator , who filters the proposals from proposers to acceptors. While proposers, learners, and coordinators are easily replaced, substituting an acceptor is prohibitive. Protocols that do not employ a coordinator are less resilient to acceptor failures. Protocols that use one coordinator are more resilient to acceptor failures, at the expense of one extra communication step even in the absence of failures. Moreover, they require replacing the coordinator as soon as it fails, a reconfiguration that, although relatively inexpensive, diminishes the protocol availability. Hence, either option, i.e., one or zero coordinator, has its drawbacks. In previous works, we have presented an alternative: multicoordinated agreement protocols. Such protocols are as resilient as single-coordinated protocols but require less reconfiguration to cope with coordinator failures. In fact, most reconfiguration can be done in parallel to the execution of the protocol’s normal steps. Multicoordination can be applied to several problems. In this paper we exemplify its use in solving consensus and then introduce a fast multicoordinated agreement protocol for agents organized in groups, an abstraction for fast local area networks interconnected by slower links.

Hadoop is becoming the standard framework to process large amounts of data. It takes advantage of distributed computing to do it fast and efficiently. However, this is only possible if data is supplied with high availability and consistency. Those goals are fulfilled by a core piece of Hadoop called the Hadoop Distributed Filesystem (HDFS). HDFS is a very scalable distributed filesystem capable of storing petabytes and providing high throughput data access. It makes intensive use of replication and checksums to protect the system from data loss and corruption. Despite of all those qualities, HDFS has a central component whose maintenance demands the entire system to be shut down. Furthermore, that component is also a single point of failure. Those limitations make HDFS unsuitable for 24x7 applications. In this technical report we are going to make a high-level introduction to HDFS and discuss attempts to solve the mentioned problem. ∗aoriani@comp.ufscar.br †islene@ic.unicamp.br ‡rschmidt@facebook.com

Adaptability and graceful degradation are important features in distributed systems. Yet, consensus and other agreement protocols, basic building blocks of reliable distributed systems, lack these features and must perform expensive reconfiguration even in face of single failures. In this paper we describe multicoordinated mode of execution for agreement protocols that has improved availability and tolerates failures in a graceful manner. We exemplify our approach by presenting a generic broadcast algorithm. Our protocol can adapt to environment changes by switching to different execution modes. Finally, we show how our algorithm can solve the generalized consensus and its many instances (e.g., consensus and atomic broadcast).

In the deferred update technique for database replication, a number of database replicas are used to implement a single serializable database interface. Its main idea consists in executing all operations of a transaction initially on a single database replica. Transactions that do not change the database state can commit locally to the replica they executed, but other transactions must be globally certified and, if committed, have their update operations submitted to all replicas. Despite its wide use, we are not aware of any work that explored the inherent limitations and characteristics of deferred update database replication, ours being the first attempt in this direction.

Classic Paxos is a round-based distributed consensus algorithm. Normally, proposals are sent to the round leader and decided in two communication steps more. Fast Paxos extends Classic Paxos by allowing also fast rounds, in which a decision can be learned in two communication steps without relying on a leader but requiring bigger quorums. We extend Fast Paxos and allow yet another sort of round with multiple quorums of coordinators. Such new rounds have the same expected latency and quorum size requirements as classic rounds but do not rely on a single leader, providing better availability and allowing load balance. Fast rounds and ours have a different liveness requirement: the absence of collisions—which may happen if different values are proposed concurrently. However, collisions are inherently more expensive in fast rounds. We apply our algorithm to Generalized Consensus, a generalization of agreement problems that can use the application semantics to avoid

AbstractThispaperintroducesWoodfrog,alibrarythatprovidespersistencepropertiestoSmartfrogcom-ponents. As arctic wood frogs do to survive cold winters, our library “freezes” a component whenthereisasystemfailuresothatitsinternalstatecanberecoveredandtheexecutionresumedafterthefault is overcome. Moreover, as part of the “thawing” process, references to the recovered compo-nentinothersystemobjectsaretransparentlyrestored. Thelibraryisimplementedinamodularwaywith three very distinct tasks: stable state maintenance, component recovery and reference update.All these tasks can be overwritten and implemented using different algorithms, which allows theuser to choose mechanisms more suitable to the application being deployed. 1 Introduction In a system built out of commodity hardware and software, different failures may happen during theexecution. The simplest one (and probably the least frequent as well) is a server crash, when the wholeserver goes down and processes running on it are killed. Most failures, however, are due to softwareproblems and their results might be interestingly complex. If the problem is deterministic and persistent(caused by program mistakes known as Bohrbugs [3]), there is not much that can be done withoutcorrecting or reconfiguring the piece of software which is causing the problem. On the other hand,if the problem is transient or intermittent (caused by Heisenbugs [3]), it can be temporarily solved byrestarting the application, restarting the system, or resorting to a different replica that did not fail. Someproblems, specially Heisenbugs, do not necessarily crash the system or application. They may onlymake the system malfunction by stopping minor services, degrading performance or wasting resources,problems that might only be identified by an external agent responsible for system recovery. A simpleaction this agent can perform is to force a server or group of servers to reboot. If the system toleratesserver crashes, when it resumes normal execution, the transient problems will have been overcome.Smartfrog is an application management framework developed at HP Laboratories and aimed athelping configuration, deployment and maintenance of complex systems and applications [12]. Briefly,the whole system is defined in terms of cooperating components organized in a tree-like structure, with

1 Motivation High-performance data management systems have traditionally relied upon specialized hardware and proprietary software, both invariably expensive. Even though powerful hardware infrastructures, built out of commodity components, have become affordable in recent years, software still remains an obstacle to open high-performance and high-availability data management systems. The Sprint project aims to overcome this obstacles.

The paper presents a primary-backup protocol to manage replicated in-memory database systems (IMDBs). The protocol exploits two features of IMDBs: coarse-grain concurrency control and deferred disk writes. Primary crashes are quickly detected by backups and a new primary is elected whenever the current one is suspected to have failed. False failure suspicions are tolerated and never lead to incorrect behavior. The protocol uses a consensus-like algorithm tailor-made for our replication environment. Under normal circumstances (i.e., no failures or false suspicions), transactions can be committed after two communication steps, as seen by the applications. Performance experiments have shown that the protocol has very low overhead and scales linearly with the number of replicas

In this paper, we study the problem of partial database replication. Numerous previous works have investigated database replication, however, most of them focus on full replication. We are here interested in genuine partial replication protocols, which require replicas to permanently store only information about data items they replicate. We define two properties to characterize partial replication. The first one, Quasi-Genuine Partial Replication, captures the above idea; the second one, Non-Trivial Certification, rules out solutions that would abort transactions unnecessarily in an attempt to ensure the first property. We also present two algorithms that extend the Database State Machine [8] to partial replication and guarantee the two aforementioned properties. Our algorithms compare favorably to existing solutions both in terms of number of messages and communication steps.

Current cluster architectures provide the ideal environment to run federations of main-memory database systems (FMMDBs). In FMMDBs, data resides in the main memory of the federation servers, significantly improving performance by avoiding I/O during the execution of read operations. To maximize the performance of update transactions as well, some applications recur to deferred disk writes. This means that update transactions commit before their modifications are written on stable storage and durability must be ensured outside the database. While deferred disk writes in centralized MMDBs relax the durability property of transactions only, in FMMDBs transaction atomicity may be also violated in case of failures. We address this issue from the perspective of log-based rollback-recovery in distributed systems and provide an efficient solution to the problem.

Communication-induced checkpointing protocols that ensure rollback-dependency trackability (RDT) guarantee important properties to the recovery system without explicit coordination. However, there was no garbage collection algorithm for them which did not use some type of process synchronization, like time assumptions or reliable control message exchanges. This paper addresses the problem of garbage collection for RDT checkpointing protocols and presents an optimal solution for the case where coordination is done only by means of timestamps piggybacked in application messages. The algorithm uses the same timestamps as off-the-shelf RDT protocols and ensures the tight upper bound on the number of uncollected checkpoints for each process during all the system execution

Carlos Aguilar-Melchor Luciana Arantes Filipe Araujo Mark Astley Lasaro Camargos Julien Cartigny Yanninck Chevalier Neha Dhar Richard Ekwall Tirthankar Ghosh Yongsheng He Jung-Rim Hyun Z. Jerzak Andrew Jiang Anxiao Jiang Marc-Olivier Killijian Turgay Korkmaz Andrea Matsunaga Peter Musial Vincent Nicomette Fabio Picconi Francesco Quaglia Miroslava Raspopvic Paolo Romano Sania Salahuddin Rodrigo Schmidt Robert Sheahan Alexander Sprintson Peter Sobe Martin Suesskraut Krishnan Sugavanam Caimu Tang Ali Saman Tosun Goce Trajcevski Mauricio Tsugawa Radek Vingralek Ute Wappler Marcin Wieloch Ning Wu Jing Xu Vaide Zuikeviciute

Communication-induced checkpointing protocols that ensure rollback-dependency trackability (RDT) guarantee important properties to the recovery system without explicit coordination. However, to the best of our knowledge, there was no garbage collection algorithm for them which did not use some type of process synchronization, like time assumptions or reliable control message exchanges. This paper addresses the problem of garbage collection for RDT checkpointing protocols and presents an optimal solution for the case where coordination is done only by means of timestamps piggybacked in application messages. Our algorithm uses the same timestamps as off-the-shelf RDT protocols and ensures the tight upper bound on the number of uncollected checkpoints for each process during all the execution.

With the emergence of a mobile and large-scale Internet, highly-dynamic distributed systems are becoming increasingly important. Examples of this growing importance can be found in recent researches in largescale peer-to-peer protocols [1, 3], as well as in ad hoc network technologies [4]. Intuitively, ahighly-dynamicdistributed system can be defined as a system whose configuration changes very often. What may vary in a configuration depends on the considered model. In a probabilistic crashrecovery model, for instance, the configuration might consist of a probability associated with each node, representing the average uptime of that node.

In this paper, we propose a novel approach for solving the reliable broadcast problem in a probabilistic unreliable model. Our approach consists in first defining the optimality of probabilistic reliable broadcast algorithms and the adaptiveness of algorithms that aim at converging toward such optimality. Then, we propose an algorithm that precisely converges toward the optimal behavior, thanks to an adaptive strategy based on Bayesian statistical inference. We compare the performance of our algorithm with that of a typical gossip algorithm through simulation. Our results show, for example, that our adaptive algorithm quickly converges toward such exact knowledge.

In this paper, we propose a novel approach for solving the reliable broadcast problem in a probabilistic model, i.e., where links lose messages and where processes crash and recover probabilistically. Our approach consists in flrst deflning the optimality of probabilistic reliable broadcast algorithms and the adaptiveness of algorithms that aim at converging toward such optimality. Then, we propose an algorithm that precisely converges toward the optimal behavior, thanks to an adaptive strategy based on Bayesian statistical inference. Our adaptive algorithm is modular and consists of two activities. The flrst activity is responsible for solving the reliable broadcast, given information about the failure probability of each link and of each process. This activity relies on the notion of Maximum Reliability Tree, which we derive from the notion of Maximum Spanning Tree. The other activity is responsible for approximating failure probabilities of links and processes, using Bayesian networks. We compare the performance of our algorithm with that of a typical gossip algorithm through simulation. Our results show, for example, that our adaptive algorithm quickly converges toward such exact knowledge.

checkpoints que se tornam obsoletos à medida que a computação progride e novos checkpoints são gravados pelos processos.