An effective isolation among workloads within a shared and possibly contended compute environment is a crucial aspect for industry and academia alike to ensure optimal performance and resource utilization. Modern ecosystems offer a wide range of approaches and solutions to ensure isolation for a multitude of different compute resources. Past experiments have verified the effectiveness of this resource isolation with micro benchmarks. The effectiveness of Quality of Service (QoS) isolation for intricate workloads beyond micro benchmarks however, remains an open question. This paper addresses this gap by introducing a specific example involving a database workload isolated using Cgroups from a disruptor contending for CPU resources. Despite the even distribution of CPU isolation limits among the workloads, our findings reveal a significant impact of the disruptor on the QoS of the database workload. To illustrate this, we present a methodology for quantifying this isolation, accompanied by an implementation incorporating essential instrumentation through Extended Berkeley Packet Filter (eBPF). This not only highlights the practical challenges in achieving robust QoS isolation but also emphasizes the need for additional instrumentation and realistic scenarios to comprehensively evaluate and address these challenges.

Industry and academia have strong incentives to adopt virtualization technologies. Such technologies can reduce the total cost of ownership or facilitate business models like cloud computing. These options have recently grown significantly with the rise of Kubernetes and the OCI runtime specification. Both enabled virtualization technology vendors to easily integrate their solution into existing infrastructures, leading to increased adoption. Making a detailed decision on a technology selection based on objective characteristics is a complex task. This specifically includes the instrumentation of performance characteristics that are an important aspect for a fair comparison. Moreover, a subsequent quantification of the isolation capability based on performance metrics is not readily available. In this paper, we instrument and determine the OCI runtime isolation capability by measuring virtualized system resources. We hereby build on two previous contributions, a proven isolation measurement workflow engine, and meaningful isolation metrics. The existing workflow engine is extended to integrate OCI runtime instrumentation as well as the novel isolation metrics. We indicate a quantifiable distinction between the isolation capabilities of these technologies. Researchers and industry alike can use the results to make decisions on the adoption of virtualization technology based on their isolation characteristics. Furthermore, our extended measurement workflow engine can be leveraged to conduct further experiments with new technologies, metrics, and scenarios.

Monitoring infrastructures is a key cornerstone for both security and resilience mechanisms, as well as, infrastructure management helping to ensure availability and reliability. Its importance is increasing in the light of Internet-of-Things (IoT) infrastructures growing both in size and complexity, hence, widening the attack surface and decreasing the possibilities of manual management. There is a multitude of different monitoring concepts providing solutions to those problems. However, in the context of IoT those traditional concepts are limited in their adoption capabilities due to fundamental differences in architecture and structure. In this paper, we create a systematization of knowledge about the impact of IoT infrastructure characteristics on requirements for and architecture of monitoring. In particular, we first discuss definitions of monitoring resulting in an own definition, before elaborating on the terminology encompassing monitoring. We further discuss requirements imposed by IoT systems combined with an analysis of monitoring properties relevant for IoT. As part of this paper, we also focus on highlighting current and future directions of IoT architecture and infrastructure.

The research field of data analytics has grown significantly with the increase of gathered and available data. Accordingly, a large number of tools, metrics, and best practices have been proposed to make sense of this vast amount of data. To this end, benchmarking and standardization are needed to understand the proposed approaches better and continuously improve them. For this purpose, numerous associations and committees exist. One of them is SPEC (Standard Performance Evaluation Corporation), a non-profit corporation for the standardization and benchmarking of performance and energy evaluations. This paper gives an overview of the recently established SPEC RG Predictive Data Analytics Working Group. The mission of this group is to foster interaction between industry and academia by contributing research to the standardization and benchmarking of various aspects of data analytics.

Byzantine fault tolerance (BFT) can preserve the availability and integrity of IoT systems where single components may suffer from random data corruption or attacks that can expose them to malicious behavior. While state-of-the-art BFT state-machine replication (SMR) libraries are often tailored to fit a standard request-response interaction model with dedicated client-server roles, in our design, we employ an IoT-fit interaction model that assumes a loosly-coupled, event-driven interaction between arbitrarily wired IoT components. In this paper, we explore the possibility of automating and streamlining the complete process of integrating BFT SMR into a component-based IoT execution environment. Our main goal is providing simplicity for the developer: We strive to decouple the specification of a logical application architecture from the difficulty of incorporating BFT replication mechanisms into it. Thus, our contributions address the automated configuration, re-wiring and deployment of IoT components, and their replicas, within a component-based, event-driven IoT platform.

Collection and analysis of distributed (cloud) computing workloads allows for a deeper understanding of user and system behavior and is necessary for efficient operation of infrastructures and applications. The availability of such workload data is however often limited as most cloud infrastructures are commercially operated and monitoring data is considered proprietary or falls under GPDR regulations. This work investigates the generation of synthetic workloads using Generative Adversarial Networks and addresses a current need for more data and better tools for workload generation. Resource utilization measurements such as the utilization rates of Content Delivery Network (CDN) caches are generated and a comparative evaluation pipeline using descriptive statistics and time-series analysis is developed to assess the statistical similarity of generated and measured workloads. We use CDN data open sourced by us in a data generation pipeline as well as back-end ISP workload data to demonstrate the multivariate synthesis capability of our approach. The work contributes a generation method for multivariate time series workload generation that can provide arbitrary amounts of statistically similar data sets based on small subsets of real data. The presented technique shows promising results, in particular for heterogeneous workloads not too irregular in temporal behavior.

Correctly configuring a distributed database management system (DBMS) deployed in a cloud environment for maximizing performance poses many challenges to operators. Even if the entire configuration spectrum could be measured directly, which is often infeasible due to the multitude of parameters, single measurements are subject to random variations and need to be repeated multiple times. In this work, we propose Baloo, a framework for systematically measuring and modeling different performance-relevant configurations of distributed DBMS in cloud environments. Baloo dynamically estimates the required number of measurement configurations, as well as the number of required measurement repetitions per configuration based on a desired target accuracy. We evaluate Baloo based on a data set consisting of 900 DBMS configuration measurements conducted in our private cloud setup. Our evaluation shows that the highly configurable framework is able to achieve a prediction error of up to 12 %, while saving over 80 % of the measurement effort. We also publish all code and the acquired data set to foster future research.

Cloud resources have become a preferred operational model distributed Database Management Systems (DBMS) by offering the elasticity and virtually unlimited scalability, but increase the risk of failures with increasing cluster sizes. While distributed DBMS provide high-availability mechanisms, it is currently an open research question to what extent they are able to provide availability and performance guarantees in case of cloud resource failures. Especially as existing DBMS benchmarks do not consider availability. We present a comprehensive methodology for evaluating the availability of distributed DBMS in case of cloud resource failures. Based on this methodology, we introduce a novel framework that automates the full evaluation process, including the failure injection, and emphasizes reproducibility. The framework is validated by 16 diverse availability evaluations. The results show that distributed DBMS are not necessary available even if sufficient replicas are available and clients can experience significant downtimes.

Auto-scaling is able to change the scale of an application at runtime. Understanding the application characteristics, scaling impact as well as the workload, an auto-scaler aligns the acquired resources to match the current workload. For distributed Database Management Systems (DBMS) forming the backend of many large-scale cloud applications, it is currently an open question to what extent they support scaling at run-time. In particular, elasticity properties of existing distributed DBMS are widely unknown and difficult to evaluate and compare. This paper presents a comprehensive methodology for the evaluation of the elasticity of distributed DBMS. On the basis of this methodology, we introduce a framework that automates the full evaluation process. We validate the framework by defining significant elasticity scenarios for a case study that comprises two DBMS for write-heavy and read-heavy workloads of different intensities. The results show that scalable distributed DBMS are not necessarily elastic and that adding more instances to a cluster at run-time may even decrease the experienced performance.

Big Data and IoT applications require highly-scalable database management system (DBMS), preferably operated in the cloud to ensure scalability also on the resource level. As the number of existing distributed DBMS is extensive, the selection and operation of a distributed DBMS in the cloud is a challenging task. While DBMS benchmarking is a supportive approach, existing frameworks do not cope with the runtime constraints of distributed DBMS and the volatility of cloud environments. Hence, DBMS evaluation frameworks need to consider DBMS runtime and cloud resource constraints to enable portable and reproducible results. In this paper we present Mowgli, a novel evaluation framework that enables the evaluation of non-functional DBMS features in correlation with DBMS runtime and cloud resource constraints. Mowgli fully automates the execution of cloud and DBMS agnostic evaluation scenarios, including DBMS cluster adaptations. The evaluation of Mowgli is based on two IoT-driven scenarios, comprising the DBMSs Apache Cassandra and Couchbase, nine DBMS runtime configurations, two cloud providers with two different storage backends. Mowgli automates the execution of the resulting 102 evaluation scenarios, verifying its support for portable and reproducible DBMS evaluations. The results provide extensive insights into the DBMS scalability and the impact of different cloud resources. The significance of the results is validated by the correlation with existing DBMS evaluation results.

Virtualisation first and cloud computing later has led to a consolidation of workload in data centres that also comprises latency-sensitive application domains such as High Performance Computing and telecommunication. These types of applications require strict latency guarantees to maintain their Quality of Service. In virtualised environments with their churn, this demands for adaptability and flexibility to satisfy. At the same time, the mere scale of the infrastructures favours commodity (Ethernet) over specialised (Infiniband) hardware. For that purpose, this paper introduces a novel traffic management algorithm that combines Rate-limited Strict Priority and Deficit round-robin for latency-aware and fair scheduling respectively. In addition, we present an implementation of this algorithm on the bmv2 P4 software switch by evaluating it against standard priority-based and best-effort scheduling.

Cloud data centres share physical resources at the same time with multiple users, which can lead to resource interferences. Especially with resource intensive computations like HPC or big data processing jobs, neighbouring applications in a cloud data centre may experience less performance of their assigned virtual resources. This work evaluates the predictability of such resource intensive jobs in principle. The assumption is, that the execution behaviour of such computations depends on the computation and the environment parameters. From these two influencing factors, the predictability is the outcome of removing the hardware dependent environment parameters from the observed execution behaviour, in order to compute any other execution behaviour for computations with similar computation parameters but on a different environment. The assumptions are analysed and evaluated with the HPC application Molpro.

Infrastructure as a Service (IaaS) Cloud services allow users to deploy distributed applications in a virtualized environment without having to customize their applications to a specific Platform as a Service (PaaS) stack. It is common practice to host multiple Virtual Machines (VMs) on the same server to save resources. Traditionally, IaaS data center management required manual effort for optimization, e.g. by consolidating VM placement based on changes in usage patterns. Many resource management algorithms and frameworks have been developed to automate this process. Resource management algorithms are typically tested via experimentation or using simulation. The main drawback of both approaches is the high effort required to conduct the testing. Existing Cloud or IaaS simulators require the algorithm engineer to reimplement their algorithm against the simulator's API. Furthermore, the engineer manually needs to define the workload model used for algorithm testing. We propose an approach for the simulative analysis of IaaS Cloud infrastructure that allows algorithm engineers and data center operators to eval- uate optimization algorithms without investing additional effort to reimplement them in a simulation environment. By leveraging runtime monitoring data, we automatically construct the simula- tion models used to test the algorithms. Our validation shows that algorithm tests conducted using our IaaS Cloud simulator match the measured behavior on actual hardware.

The H2020 research CloudSocket project enacts the business IT-alignment by implementing Business process as a Service (BPaaS).

With the rapid rise of the cloud computing paradigm, the manual maintenance and provisioning of the technological layers behind it, both in their hardware and virtualized form, became cumbersome and error-prone. This has opened up the need for automated capacity planning strategies in heterogeneous cloud computing environments. However, even with mechanisms to fully accommodate customers and fulfill service-level agreements, providers often tend to over-provision their hardware and virtual resources. A proliferation of unused capacity leads to higher energy costs, and correspondingly, the price for cloud technology services. Capacity planning algorithms rely on data collected from the utilized resources. Yet, the amount of data aggregated through the monitoring of hardware and virtual instances does not allow for a manual supervision, much less data analysis or a correlation and anomaly detection. Current data science advancements enable the assistance of efficient automation, scheduling and provisioning of cloud computing resources based on supervised and unsupervised machine learning techniques. In this work, we present the current state of the art in monitoring, storage, analysis and adaptation approaches for the data produced by cloud computing environments, to enable proactive, dynamic resource provisioning.

Cloud computing and its computing as an utility paradigm provides on-demand resources allowing the seamless adaptation of applications to fluctuating demands. While the Cloud's ongoing commercialisation has lead to a vast provider landscape, vendor lock-in is still a major hindrance. Recent outages demonstrate that relying exclusively on one provider is not sufficient. While existing cloud orchestration tools promise to solve the problems by supporting deployments across multiple cloud providers, they typically rely on provider dependent models forcing prior knowledge of offers and obstructing flexibility in case of errors. We propose a cloud provider-agnostic application and resource description using a constraint language. It allows users to express resource requirements of an application without prior knowledge of existing offers. Additionally, we propose a discovery service automatically collecting available offers. We combine this with a matchmaking algorithm representing the discovery model and the user-given constraints in a constraint satisfaction problem (CSP) that is then solved. Finally, we manipulate this discovery model during runtime to react on errors. Our evaluation shows that using a constraint-based language is a feasible approach to the provider selection problem, and that it helps to overcome vendor lock-in.

CACTOS offers Cloud developers, operators, and consultants a context-aware optimisation for private Clouds. It leads to better and more reliable user experience, by optimising the mapping of virtual to physical resources, considering application requirements and heterogeneity. The optimisation and simulation requires monitoring, and an integration for controlling and intercepting client requests.

Cloud computing is the promise to provide flexible IT solutions. This correlates with an increasing demand in flexibility of business processes in companies. However, there is still a huge gap between business and IT management. The evolution of cloud service models tries to bridge this by bringing up fine grained and multi-dimensional service models. One of the new service models is Business Process as a Service (BPaaS), which promises to bridge the gap from business process to cloud computing. Yet, the BPaaS paradigm is not thoroughly classified with respect to the cloud computing characteristics. In this paper we introduce a first classification of the BPaaS paradigm with the focus on the common cloud characteristics. Therefore, we analyze the traditional path from a business process model to its execution via on-demand resources and derive a leveled model for BPaaS. For each level, we introduce the entities on that level in terms of (i) correlation to cloud characteristics, (ii) concepts and (iii) tools, and evaluate its cloudification options, i.e. the ability to support the provision of a business process as a service. The presented work enables the categorisation of items in the BPaaS paradigm and outlines how traditional business processes can be enabled for cloud delivery. This classification and analysis will be extended, once the BPaaS paradigm reached wider acceptance in academia and industry, and more standards evolved.

Cloud computing proved to offer flexible IT solutions. Although large enterprises may benefit from this technology by educating their IT departments, SMEs face the risk to dramatically falling behind in cloud usage and hence lose the ability to efficiently adapt their IT to their business needs. This chapter presents the vision and the outcome of the H2020 project CloudSocket. The foundation lays the idea of Business Processes as a Service, where concept models and semantics are applied to align business processes with Multi-Cloud deployed workflows. The proposed CloudSocket platform consist of four architectural building blocks: (i) design, (ii) allocation, (iii) execution, and (iv) evaluation. These are organised as environments that cope with specific tasks and research questions. An overview of each environment is given along with main prototypes that were developed to push the state-of-the-art in the respective field. We show the success of the achievements in current research endeavours and how we will pursue the open questions.