In times of increasing bandwidth demands, network operators strive for increased visibility of their network's utilization as well as an indication of the legitimacy of traffic processed across network nodes. Additionally, the detection and mitigation of illegitimate or malicious traffic such as denial of service attacks remains a current and persistently active field of research. Flow-based network monitoring can provide this information live from any network interface. This paper introduces a flow processing platform meant to receive flow information from border interfaces and distribute the acquired information to specialized applications. Transit providers deploying our platform can use this information directly, but also provide all interested customers or network entities with the specific subset concerning them. Between collecting and redistributing the flow information, our platform offers different methods of enrichment using a variety of sources allowing for views superior to plain Netflow records. However, a provided tool can reencode and reexport standard Netflow to ensure compatibility and allow for seamless integration of customer-specific streams into preexisting setups. This platform's components allow the enrichment, division and anonymization of flow data to a number of highly customized streams for any type of application, either on a customer-specific or a network-wide provider level. Applications include the conversion of flow data for time-series databases and the accompanying dashboards, the detection of DDoS attacks or other high-traffic situation on any network level, the identification of faulty network routing policies, or any other use case conceivable on regular flow data, but within an arbitrary network scope.

Cloud data centres share physical resources at the same time with multiple users, which can lead to resource interferences. Especially with resource intensive computations like HPC or big data processing jobs, neighbouring applications in a cloud data centre may experience less performance of their assigned virtual resources. This work evaluates the predictability of such resource intensive jobs in principle. The assumption is, that the execution behaviour of such computations depends on the computation and the environment parameters. From these two influencing factors, the predictability is the outcome of removing the hardware dependent environment parameters from the observed execution behaviour, in order to compute any other execution behaviour for computations with similar computation parameters but on a different environment. The assumptions are analysed and evaluated with the HPC application Molpro.

Cloud data centres with providers on physical level and customers on virtual level both monitor their hard-and software infrastructure to understand load patterns and to detect malfunctions and bottlenecks. The motivation for cloud monitoring on both the virtual and physical level can be summarized to the three occasions alerting, resource allocation, and visualization. Typical cloud monitoring solutions transfer and store all metrics of all systems under observation in central stores like time series databases. Applications then query, aggregate and compute their result out of these monitoring data. In large data centres, the amount of data scales up and leads to a reasonable overhead. In addition monitoring on virtual and physical level duplicates the overhead. We present an approach for monitoring resource statistics on the physical level only, and provide resource utilisation profiles to cloud middleware and customers, instead of storing the raw time series data. The approach first revisits the necessary metrics for hardware independent resource profiles, considering overbooked physical servers as well. A profile consists of a static (e.g. CPU cores) and dynamic (e.g. changing utilisation) part, and is based on statistical computations like histograms and Markov chains.

Containers emerged as cloud resource offerings. While the advantages of containers, such as easing the application deployment, orchestration and adaptation, work well for stateless applications, the feasibility of containerization of stateful applications, such as database management system (DBMS), still remains unclear due to potential performance overhead. The myriad of container operation models and storage backends even raises the complexity of operating a containerized DBMS. Here, we present an extensible evaluation methodology to identify performance overhead of a containerized DBMS by combining three operational models and two storage backends. For each combination a memory-bound and disk-bound workload is applied. The results show a clear performance overhead for containerized DBMS on top of virtual machines (VMs) compared to physical resources. Further, a containerized DBMS on top of VMs with different storage backends results in a tolerable performance overhead. Building upon these baseline results, we derive a set of open evaluation challenges for containerized DBMSs.

CACTOS offers Cloud developers, operators, and consultants a context-aware optimisation for private Clouds. It leads to better and more reliable user experience, by optimising the mapping of virtual to physical resources, considering application requirements and heterogeneity. The optimisation and simulation requires monitoring, and an integration for controlling and intercepting client requests.

The presented dynamic network scheduler improves the fairness and efficiency of network utilization in a Cloud data centre. The proposed design utilizes a directed graph which represents the network comprising routers, switches, physical hypervisors, and virtual machines (VMs) as graph nodes, and represents the physical network connections as weighted edges. The edges have a guaranteed transmission rate derived from the number of devices sharing an outgoing link with a defined bandwidth. Moreover, to maximize utilization of the resources, each node gets a deserved rate dynamically depending on the measured utilization metrics. A VM throttles up or down its traffic up to its deserved rate. The conceptual design of a dynamic network scheduler is further prototypically implemented and evaluated. The implementation uses Software Defined Networking (SDN) with OpenFlow, Ryu SDN controller, and Open vSwitch as software switch on the hypervisor level. The presented dynamic network scheduler uses OpenFlow for monitoring and applying flows to control the link bandwidth. The evaluation shows that the dynamic network scheduler maximizes fairness in resource sharing while minimizing the unutilized resources.

IaaS Cloud systems enable the Cloud provider to overbook his data centre by selling more virtual resources than physical resources available. This approach works if on average the resource utilisation of a virtual machine is lower than the virtual machine boundaries. If this assumption is violated only locally, Cloud users will experience performance degradation and poor quality of service. This paper proposes the introduction of dynamic overbooking in the sense that the overbooking factors are not equal for all physical resources, but vary dynamically depending on the resource demands of the virtual resources they host. It allows new pricing models that are dependent on the overbooking a Cloud customer is willing to accept. Additionally, we discuss prerequisites for supporting its realisation in an OpenStack private Cloud, including a monitoring system, dedicated metrics to be monitored, as well as performance models that predict the performance degradation depending on the overbooking.

The paper presents a concept and an implementation for an image registry for virtual collaborative environments (ViCE). This cross-platform and cross-organizational image registry bridges gaps between execution environment platforms and user communities. The presented concept consists of a conceptual architecture and a sophisticated set of metadata fields to describe images as virtual environments. The main challenge is the wide spread definition of an image. The terminology defines execution environments, which consist of runtime technologies (virtual machines, containers, applications) and a management layer (basic management, cloud computing, container clusters, job schedulers). An execution environment runs a deployable implicit or declarative image to build a virtual environment. With this abstraction the image registry can share virtual environment across Cloud computing, HPC, classroom setups, with any of KVM, Docker, Singularity, etc. in use. The open source implementation is written in Go and presented with a scalable microservice architecture, using Couchbase as metadata store and RabbitMQ as communication hub between software components.

Driven by new application domains, the database management systems (DBMSs) landscape has significantly evolved from single node DBMS to distributed database management systems (DDBMSs). In parallel, cloud computing became the preferred solution to run distributed applications. Hence, modern DDBMSs are designed to run in the cloud. Yet, in distributed systems the probability of failures is the higher the more entities are involved and by using cloud resources the probability of failures increases even more. Therefore, DDBMSs apply data replication across multiple nodes to provide high availability. Yet, high availability limits consistency or partition tolerance as stated by the CAP theorem. As the decision for two of the three attributes in not binary, the heterogeneous landscape of DDBMSs gets even more complex when it comes to their high availability mechanisms. Hence, the selection of a high available DDBMS to run in the cloud becomes a very challenging task, as supportive evaluation frameworks are not yet available. In order to ease the selection and increase the trust in running DDBMSs in the cloud, we present the Gibbon framework, a novel availability evaluation framework for DDBMSs. Gibbon defines quantifiable availability metrics, a customisable evaluation methodology and a novel evaluation framework architecture. Gibbon is discussed by an availability evaluation of MongoDB, analysing the take over and recovery time.

Model-drivenengineering promotes models and modeltrans- formations as the primary assets in software development. The models@run-time approach provides an abstract rep- resentation of a system at run-time, whereby changes in the model and the system are constantly reflected on each other. In this paper, we report on more than three years of experience with realising models@run-time in scalable cloud scenarios using a technology stack consisting of the Eclipse Modelling Framework (EMF) and Connected Data Objects(CDO).We establish requirements for the three roles domain-specific language (DSL) designer, developer, and operator, and compare them against the capabilities of EM- F/CDO. It turns out that this technology stack is well-suited for DSL designers, but less recommendable for developers and even less suited for operators. For these roles, we experi- enced a steep learning curve and several lacking features that hinder the implementation of models@run-time in scalable cloud scenarios. Performance experiences show limitations for write heavy scenarios with an increasing amount of total elements. While we do not discourage the use of EMF/CDO for such scenarios, we recommend that its adoption for sim- ilar use cases is carefully evaluated until this technology stack has realised our wish list of advanced features.

Even though the cloud era has begun almost one decade ago, many problems of the first hour are still around. Vendor lock-in and poor tool support hinder users from taking full advantage of main cloud features: dynamic and scale. This has given rise to tools that target the seamless management and orchestration of cloud applications. All these tools promise similar capabilities and are barely distinguishable what makes it hard to select the right tool. In this paper, we objectively investigate required and desired features of such tools and give a definition of them. We then select three open-source tools (Brooklyn, Cloudify, Stratos) and compare them according to the features they support using our experience gained from deploying and operating a standard three-tier application. This exercise leads to a fine-grained feature list that enables the comparison of such tools based on objective criteria as well as a rating of three popular cloud orchestration tools. In addition, it leads to the insight that the tools are on the right track, but that further development and particularly research is necessary to satisfy all demands.

Distributed database systems represent an essential component of modern enterprise application architectures. If the overall application needs to provide reliability and availability, the database has to guarantee these properties as well. Entailing non-functional database features such as replication, consistency, conflict management, and partitioning represent subsequent challenges for successfully designing and operating an available and reliable database system. In this document, we identify why these concepts are important for databases and classify their design options. Moreover, we survey how eleven modern database systems implement these reliability and availability properties.

Recent advances in hardware development coupled with the rapid adoption and broad applicability of cloud computing have introduced widespread heterogeneity in data centers, significantly complicating the management of cloud applications and data center resources. This paper presents the CACTOS approach to cloud infrastructure automation and optimization, which addresses heterogeneity through a combination of in-depth analysis of application behavior with insights from commercial cloud providers. The aim of the approach is threefold: to model applications and data center resources, to simulate applications and resources for planning and operation, and to optimize application deployment and resource use in an autonomic manner. The approach is based on case studies from the areas of business analytics, enterprise applications, and scientific computing.