Simulation can be a powerful technique for evaluating the performance of large-scale cloud computing services in a relatively low cost, low risk and time-sensitive manner. Large-scale data indexing, distribution and management is complex to analyse in a timely manner. In this paper, we extend the CloudSim cloud simulation framework to model and simulate a distributed search engine architecture and its workload characteristics. To test the simulation framework, we develop a model based on a real-world ElasticSearch deployment on Linknovate.com. An experimental evaluation of the framework, comparing simulated and actual query response time, precision and resource utilisation, suggests that the proposed framework is capable of predicting performance at different scales in a precise, accurate and efficient manner. The results can assist ElasticSearch users to manage their scalability and infrastructure requirements.

Increased complexity in IT, big data, and advanced analytical techniques are some of the trends driving demand for more sophisticated and scalable search technology. Despite Quality of Service (QoS) being a critical success factor in most enterprise software service offerings, it is often not a generic component of the enterprise search software stack. In this paper, we explore enterprise search engine dependability and performance using a real-world company architecture and associated data sourced from an ElasticSearch implementation on Linknovate.com. We propose a Fault Tree model to assess the availability and reliability of the Linknovate.com architecture. The results of the Fault Tree model are fed into a Stochastic Petri Net (SPN) model to analyze how failures and redundancy impact application performance of the use case system. Availability and MTTF were used to evaluate the reliability and throughput was used to evaluate the performance of the target system. The best results for all three metrics were returned in scenarios with high levels of redundancy.

The objective of the work package "Data Collection, Visualization and Analysis" of RECAP is to provide the necessary tools for managing and refining the data needed for the rest of the work packages. This includes the collection as well as the generation of data. Within this work package, the task of Artificial Workload Generation is responsible for the generation of a collection of datasets with artificial workloads, that complement the real data traces collected from industrial partners. Moreover, because publicly available workload data is scarce we provide the data as public data sets. This document is a companion report to deliverable which is of type “dataset”. The aim of the report is to describe the collection of datasets that constitute D5.3 and the mathematical techniques (structural time series models, generative adversarial networks, and workload based on traffic propagation) by which one can artificially generate and/or augment such datasets. The datasets described include real data traces collected by industrial partners and artificial data traces generated by the use of statistical models and neural networks. Each published data set can be used by the scientific and industrial community as a starting point for the modelling and experimental validation of distributed edge and cloud applications, facilitating the repeatability of the results.

The REliable CApacity Provisioning and enhanced remediation for distributed cloud applications (RECAP) project aims to advance cloud and edge computing technology, to develop mechanisms for reliable capacity provisioning, and to make application placement, infrastructure management, and capacity provisioning autonomous, predictable and optimized. This paper presents the RECAP vision for an integrated edge-cloud architecture, discusses the scientific foundation of the project, and outlines plans for toolsets for continuous data collection, application performance modeling, application and component auto-scaling and remediation, and deployment optimization. The paper also presents four use cases from complementing fields that will be used to showcase the advancements of RECAP.