Serverless computing, or Function-as-a-Service (FaaS), is emerging as a popular alternative model to on-demand cloud computing. Function services are executed by a FaaS provider; a client no longer uses cloud infrastructure directly as in traditional cloud consumption. Is serverless computing a feasible and beneficial approach to big data processing, regarding performance, scalability, and cost effectiveness? In this paper, we explore this research question using matrix multiplication as example. We define requirements for the design of serverless big data applications, present a prototype for matrix multiplication using FaaS, and discuss and synthesize insights from results of extensive experimentation. We show that serverless big data processing can lower operational and infrastructure costs without compromising system qualities; serverless computing can even outperform cluster-based distributed compute frameworks regarding performance and scalability.
Traditional ACID transactions, typically supported by relational database management systems, emphasize database consistency. BASE provides a model that trades some consistency for availability, and is typically favored by cloud systems and NoSQL data stores. With the increasing popularity of blockchain technology, another alternative to both ACID and BASE is introduced: SALT. In this keynote paper, we present SALT as a model to explain blockchains and their use in application architecture. We take both, a transaction and a transaction processing systems perspective on the SALT model. From a transactions perspective, SALT is about Sequential, Agreed-on, Ledgered, and Tamper-resistant transaction processing. From a systems perspective, SALT is about decentralized transaction processing systems being Symmetric, Admin-free, Ledgered and Time-consensual. We discuss the importance of these dual perspectives, both, when comparing SALT with ACID and BASE, and when engineering blockchain-based applications. We expect the next-generation of decentralized transactional applications to leverage combinations of all three transaction models.
Cloud providers offer a range of fully managed infrastructure services that enable a "serverless" architecture and development paradigm. Following this paradigm, software services can be built on compositions of cloud infrastructure services that offer fine-granular pay-per-use pricing models. While this development and deployment approach simplifies service development and management, it remains an open challenge to make use of fine-granular pricing models for improving cost transparency and reducing cost of service operations. As a solution, we present Costradamus, a cost-tracing system that implements a generic cost model and three different tracing approaches. With Costradamus, we can derive cost and performance information per API operation. We evaluate our approach and system in a smart grid context and discuss unexpected performance and deployment cost tradeoffs.
Service marketplaces promise an open platform for sellers and buyers of IT services. The marketplace design usually assumes that market functions, such as match-making, transaction settlement, and dispute resolution are performed by intermediaries in a centralized system. We propose the concept of trustless intermediation to enable new forms of decentralized service marketplaces. By leveraging blockchain-enabled smart contracts we eliminate the need for trust in marketplace intermediaries and reduce barriers of entry, lock-in, and transaction costs, by removing now obsolete trust-establishing mechanisms. Desema, our decentralized service marketplace prototype, is a first implementation of this concept that is based on the Ethereum blockchain in combination with IPFS, a peer-to-peer distributed file system.
Distributed database system performance benchmarks are an important source of information for decision makers who must select the right technology for their data management problems. Since important decisions rely on trustworthy experimental data, it is necessary to reproduce experiments and verify the results. We reproduce performance and scalability benchmarking experiments of HBase and Cassandra that have been conducted by previous research and compare the results. The scope of our reproduced experiments is extended with a performance evaluation of Cassandra on different Amazon EC2 infrastructure configurations, and an evaluation of Cassandra and HBase elasticity by measuring scaling speed and performance impact while scaling.
Applications often have consistency requirements beyond those guaranteed by the underlying eventually consistent storage system. In this work, we present an approach that guarantees monotonic read consistency and read your writes consistency by running a special middleware component on the same server as the application. We evaluate our approach using both simulation and real world experiments on Cloud storage systems.
In this work, the author focuses on a specific type of software that is delivered as a service: distributed database software, which delivers cloud database services.
Creating system setups for controlled performance evaluation experiments of distributed systems is time-consuming and expensive. Re-creating experiment setups and reproducing experimental results that have been published by other researchers is even more challenging. In this paper, we present an experiment automation approach for evaluating distributed systems in compute cloud environments. We propose three concepts which should guide the design of experiment automation tools: (1) capture experiment plans in software modules, (2) run experiments in a publicly accessible cloud-based Elastic Lab, and (3) collaborate on experiments in an open, distributed collaboration system. We developed two tools which implement these basic concepts and discuss challenges and lessons learned during our implementation. An initial exemplary use case with Apache Cassandra on top of Amazon EC2 provides a first insight into the types of performance and scalability experiments enabled by our tools.
Creating system setups for controlled performance evaluation experiments of distributed systems is time-consuming and expensive. Re-creating experiment setups and reproducing experimental results that have been published by other researchers is even more challenging. In this paper, we present an experiment automation approach for evaluating distributed systems in compute cloud environments. We propose three concepts which should guide the design of experiment automation tools: (1) capture experiment plans in software modules, (2) run experiments in a publicly accessible cloud-based Elastic Lab, and (3) collaborate on experiments in an open, distributed collaboration system. We developed two tools which implement these basic concepts and discuss challenges and lessons learned during our implementation. An initial exemplary use case with Apache Cassandra on top of Amazon EC2 provides a first insight into the types of performance and scalability experiments enabled by our tools.
Compute clouds are pools of virtual machines that are shared in a multi-tenant environment by multiple users. The virtual machine images are stored in one or more repositories and are pre-configured with an operating system. Users of the compute cloud can upload their own images or install and configure additional software on top of existing basic virtual machines. Today, the Amazon Elastic Compute Cloud (EC2) counts more than 35,000 publicly available virtual machine images. We observe, however, that the meta-data that describes the virtual machine images is of poor quality and does not cover vital information such as operating system configurations or software package installations. The sprawl of poorly documented virtual machine images poses a hurdle to sharing and re-use among members of the compute cloud community. We present a method that allows collecting software-related meta-data in compute clouds through appliance introspection. Moreover, we show how applications in the domains of selection and configuration management benefit from rich meta-data and interact with the method. The method has been implemented as an automated tool, the crawler, that collects configuration data of virtual machine images in public compute clouds and evaluated our approach by crawling Amazon EC2.
Cloud database services promise high performance, high availability, and elastic scalability. The system that provides cloud database services must, hence, be designed and managed in a way to achieve these high quality objectives. There are two technology trends that facilitate the design and management of cloud database service systems. First, the development of distributed replicated database software that is optimally designed for highly available and scalable Web applications and offered as open source software. Second, the possibility to deploy the system on cloud computing infrastructure to facilitate availability and scalability via on-demand provisioning of geo-located servers. We argue that a runtime quality measurement and analysis framework is necessary for the successful runtime management of cloud database service systems. Our framework offers three contributions over the state of the art: (i) the analysis of scaling strategies, (ii) the analysis of conflicts between contradictory objectives, and (iii) the analysis of system configuration changes on runtime performance and availability.
Sherpa is a large-scale distributed and globally replicated multi-tenant cloud data storage system. Sherpa scales by horizontally partitioning data into tablets and distributing these tablets across multiple servers. While Sherpa scales for increasing workload sizes by adding servers, it is vulnerable to load imbalance among tablets that cause hotspots to develop on just a few servers. In this paper we describe Yak, the Sherpa load balancer. Yak detects hotspots and then automatically balances load by migrating tablets from the overloaded servers, and also by splitting data into new tablets. We describe Yak's design principles, algorithms and architecture. We then evaluate Yak on workloads based on Sherpa production scenarios.
This paper suggests a definition of the term Cloud Federation, a concept of service aggregation characterized by interoperability features, which addresses the economic problems of vendor lock-in and provider integration. Furthermore, it approaches challenges like performance and disaster-recovery through methods such as co-location and geographic distribution. The concept of Cloud Federation enables further reduction of costs due to partial outsourcing to more cost-efficient regions, may satisfy security requirements through techniques like fragmentation and provides new prospects in terms of legal aspects. Based on this concept, we discuss a reference architecture that enables new service models by horizontal and vertical integration. The definition along with the reference architecture serves as a common vocabulary for discussions and suggests a template for creating value-added software solutions.
Cost and scalability benefits of Cloud storage services are apparent. However, selecting a single storage service provider limits availability and scalability to the selected provider and may further cause a vendor lock-in effect. In this paper, we present MetaStorage, a federated Cloud storage system that can integrate diverse Cloud storage providers. MetaStorage is a highly available and scalable distributed hash table that replicates data on top of diverse storage services. MetaStorage reuses mechanisms from Amazon's Dynamo for cross-provider replication and hence introduces a novel approach to manage consistency-latency tradeoffs by extending the traditional quorum (N,R,W) configurations to an (N_P,R,W) scheme that includes different providers as an additional dimension. With MetaStorage, new means to control consistency-latency tradeoffs are introduced.
The advent of advanced virtualized IT infrastructures that can be provisioned as on-demand services, known as Infrastructure-as-a-Service (IaaS) Cloud Computing, has created new research challenges and opportunities. The capability to rapidly allocate and deallocate seemingly infinite amounts of system resources is a defining characteristic of this technological trend. Operating multi-tier applications on a continuously changing environment is one of the big challenges in IaaS Cloud Computing. This challenge is even more daring, if applications are not running at one specific Cloud site but on multiple different sites of various providers. This work is motivated by the need for new deployment description approaches that target application run-time aspects in federated Clouds. We propose six key requirements for IaaS deployment description languages that facilitates continuous application deployment on a permanently changing infrastructure across multiple Cloud sites: software deployment on dynamic virtual machine resource pools, continuous system supervision and change management, a generic model for federated Clouds, automated software configuration management, multi-tier dependency management, and use of a machine-readable language.
1 Introduction In recent years, Cloud Computing has become an emerging technology that gains wide influence on IT systems. Cloud Computing is a distributed computing model for enabling service-oriented, on-demand network access to rapidly scalable resources [9]. Such resources include infrastructure as a service (IaaS), development and runtime platforms as a service (PaaS), and software and business applications as a service (SaaS). Clients do not own the resources, yet applications and data are guaranteed to be available and ubiquitously accessible by means of Web services and Web APIs " in the Cloud ". The main value proposition of Cloud Computing is to provide the clients a cost-effective, convenient means to consume the amount of IT resources that is actually needed; for the service provider, better resource utilization of existing infrastructure is achieved through a multi-tenant architecture. From a business perspective, Cloud Computing is about improving organizational efficiency and reducing cost, often coupled with the objective of achieving a faster time-to-market. Centrally hosted services with self-service interfaces can help to reduce lead times between organizational units who use the cloud as a collaborative IT environment. Re-usable components, packaged on virtual machines, provide a way to exchange working IT solutions. Capabilities to allocate and de-allocate shared resources on demand can significantly decrease overall IT spending. Low-cost access to data centres in different geographical regions may further reduce market entry barriers and enable new business models. From a technology and engineering perspective, Cloud Computing can help to realize or improve scalability, availability, and other non-functional properties of application architectures. In this paper, we focus on the technology perspective, and in particular on challenges and opportunities of Cloud Computing research related to quality-driven software service architectures. These include aspects of availability, runtime performance and power management, as well as privacy and distributed data usage. Not all desired architectural properties can be achieved at the same time. Trade-off decisions have to be made between several (sometimes contradictory) goals, such as: • increase availability & reliability • increase performance (latency, throughput) • increase security and ensure privacy 2
IT Service Continuity Management (ITSCM) delivers the recovery of IT services in the event of a disaster. ITSCM is widely perceived as an expensive challenge for enterprise-class IT operations. Cloud computing offers a model for dynamic, scalable infrastructure resource allocation on a pay-per-use basis. These attributes promise to bring cost-efficiency to ITSCM invocation and operation processes that only in the rare event of a rehearsal or an actual disaster need to allocate infrastructure resources. We propose to use the Web Service Business Process Execution Language (BPEL) in combination with Virtual Appliances to implement standardized, testable and executable ITSCM processes. The suggested solution is described and evaluated against collected data from manual recovery processes.