The National Institute of Standards and Technology defines the fundamental characteristics of cloud computing as: on-demand computing, offered via the network, using pooled resources, with rapid elastic scaling and metered charging. The rapid dynamic allocation and release of resources on demand to meet heterogeneous computing needs is particularly challenging for data centres, which process a huge amount of data characterised by its high volume, velocity, variety and veracity (4Vs model). Data centres seek to regulate this by monitoring and adaptation, typically reacting to service failures after the fact. We present a real cloud test bed with the capabilities of proactively monitoring and gathering cloud resource information for making predictions and forecasts. This contrasts with the state-of-the-art reactive monitoring of cloud data centres. We argue that the behavioural patterns and Key Performance Indicators (KPIs) characterizing virtualized servers, networks, and database applications can best be studied and analysed with predictive models. Specifically, we applied the Boosted Decision Tree machine learning algorithm in making future predictions on the KPIs of a cloud server and virtual infrastructure network, yielding an R-Square of 0.9991 at a 0.2 learning rate. This predictive framework is beneficial for making short- and long-term predictions for cloud resources.

The African Orphan Crops Consortium promotes the strategic, genome-enabled improvement of under-researched crops for biodiversity-based, nutritious food solutions in Africa. The African Plant Breeding Academy empowers the continent’s plant breeders to apply advanced genetic approaches and shared genetic solutions to the task of tailoring the immense diversity of underutilized crops to the needs of Africa’s producers, processors and consumers.

GPUs have been demonstrated to be highly effective at improving the performance of Multi-Agent Systems (MAS). One of the major limitations of further performance improvements is in the memory bandwidth required to move agent data through the GPU's memory hierarchy. This paper presents a formal model for data aware simulation and an empirical study into the impact of minimising data movement on performance. This study proposes a method that can be applied to the simulation of complex systems on GPUs to extract required data from agent behaviour during simulation time and how this information can be used to reduce data movement. The FLAME GPU software has been extended to demonstrate this technique. Three benchmark experiments have been applied to evaluate the overall reduction in simulation execution time under specific criteria. The results of the comparison between the current and new system show that reducing data movement within a simulation improves overall performance with up to 4.8x speedup reported.

Agent Based Modelling (ABM) systems have become a popular technique for describing complex and dynamic systems. ABM is the simulation of intelligent agents and how these agents communicate with each other within the model. The growing number of agent-based applications in the simulation and AI fields led to an increase in the number of studies that focused on evaluating modelling capabilities of these applications. Observing system performance and how applications behave during increases in population size is the main factor for benchmarking in most of these studies. System scalability is not the only issue that may affect the overall performance, but there are some issues that need to be dealt with to create a standard benchmark model that meets all ABM criteria. This paper presents a new benchmark model and benchmarks the performance characteristics of the FLAME GPU simulator as an example of a parallel framework for ABM. The aim of this model is to provide parameters to easily measure the following elements: system scalability, system homogeneity, and the ability to handle increases in the level of agent communications and model complexity. Results show that FLAME GPU demonstrates near linear scalability when increasing population size and when reducing homogeneity. The benchmark also shows a negative correlation between increasing the communication complexity between agents and execution time. The results create a baseline for improving the performance of FLAME GPU and allow the simulator to be contrasted with other multi-agent simulators.

In speech pathology, new assistive technologies using ASR and machine learning approaches are being developed for detecting speech disorder events. Classically-trained ASR model tends to remove disfluencies from spoken utterances, due to its focus on producing clean and readable text output. However, diagnostic systems need to be able to track speech disfluencies, such as stuttering events, in order to determine the severity level of stuttering. To achieve this, ASR systems must be adapted to recognise full verbatim utterances, including pseudo-words and non-meaningful part-words. This work proposes a training regime to address this problem, and preserve a full verbatim output of stuttering speech. We use a lightly-supervised approach using task-oriented lattices to recognise the stuttering speech of children performing a standard reading task. This approach improved the WER by 27.8% relative to a baseline that uses word-lattices generated from the original prompt. The improved results preserved 63% of stuttering events (including sound, word, part-word and phrase repetition, and revision). This work also proposes a separate correction layer on top of the ASR that detects prolongation events (which are poorly recognised by the ASR). This increases the percentage of preserved stuttering events to 70%.

In the future multi-cloud ecosystem, many cloud providers and consumers will interact to create, discover, negotiate and use software services. Cloud service brokers will play a central role in bringing providers and consumers together, assisting with software service creation (from abstract models to platform-specific deployments), multi-cloud translation (model-driven adaptation and deployment of services) quality assurance (governance; functional testing and monitoring), service continuity (failure prevention and recovery) and market competition (arbitrage; service optimization; service customization). The emerging ecosystem will be supported by common standards, service models, methods and mechanisms that will operate across a wide variety of platforms and infrastructure, and across disparate service protocols.

In this article we present a tool designed for cloud service testing, able to generate test cases from a formal specification of the service, in form of a deterministic stream X-machine (DSXM) model. The paper summarizes the theoretical foundations of X-machine based testing and illustrates the usage of the developed tool on some examples. It shows in detail how the specification should be written, which are the design for test conditions it should satisfy, in order to assure the generation of high quality test suites for the cloud service.

Cloud application platforms gain popularity and have the potential to alter the way service-based cloud applications are developed involving utilisation of platform basic services. A platform basic service provides certain functionality and is usually offered via a web API. However, the diversification of the services and the available providers increase the challenge for the application developers to integrate them and deal with the heterogeneous providers' web APIs. Therefore, a new approach of developing applications should be adopted in which developers leverage multiple platform basic services independently from the target application platforms. To this end, the authors present a development framework assisting the design of service-based cloud applications. The objective of the framework is to enable the consistent integration of the services, and to allow the seamless use of the concrete providers. The optimal service provider each time can vary depending on criteria such as pricing, quality of service and can be determined based upon Big Data analysis approaches.

Cloud brokerage is an enabling technology allowing various services to be merged together for providing optimum quality of service for the end-users. Within this collection of composed services, testing is a challenging task which brokers have to take on to ensure quality of service. Most Software-as-a-Service (SaaS) testing has focused on high-level test generation from the functional specification of individual services, with little research into how to achieve sufficient test coverage of composite services. This paper explores the use of model-based testing to achieve testing of composite services, when two individual web services are tested and combined. Two example web services – a login service and a simple shopping service – are combined to give a more realistic shopping cart service. This paper focuses on the test coverage required for testing the component services individually and their composition. The paper highlights the problems of service composition testing, requiring a reworking of the combined specification and regeneration of the tests, rather than a simple composition of the test suites; and concludes by arguing that more work needs to be done in this area.

Cloud application platforms gain popularity and have the potential to change the way applications are developed, involving composition of platform basic services. In order to enhance the developer's experience and reduce the barriers in the software development, a new paradigm of cloud application creation should be adopted. According to that developers are enabled to design their applications, leveraging multiple platform basic services, independently from the target application platforms. To this end, this paper proposes a development framework for the design of service-based cloud applications comprising two main components: the meta-model and the Platform Service Manager. The meta-model describes the building blocks which enable the construction of Platform Service Connectors in a uniform way while the Platform Service Manager coordinates the interaction of the application with the concrete service providers and further facilitates the administration of the deployed platform basic services.

In future Cloud ecosystems, brokers will mediate between service providers and consumers, playing an increased role in quality assurance, checking services for functional compliance to agreed standards, among other aspects. To date, most Software-as-a-Service (SaaS) testing has been performed manually, requiring duplicated effort at the development, certification and deployment stages of the service lifecycle. This paper presents a strategy for achieving automated testing for certification and re-certification of SaaS applications, based on the adoption of simple state-based and functional specifications. High-level test suites are generated from specifications, by algorithms that provide the necessary and sufficient coverage. The high-level tests must be grounded for each implementation technology, whether SOAP, REST or rich-client. Two examples of grounding are presented, one into SOAP for a traditional web service and the other into Selenium for a SAP HANA rich-client application. The results demonstrate good test coverage. Further work is required to fully automate the grounding.

As the number of cloud service providers grows and the requirements of cloud service consumers become more complex, the latter will come to depend more and more on the intermediation services of cloud service brokers. Continuous quality assurance and optimisation of services is becoming a mission-critical objective that many consumers will find difficult to address without help from cloud service intermediaries. The Broker@Cloud project envisages a software framework that will make it easier for cloud service intermediaries to address this need, and this paper provides an analysis of key requirements for this framework. We discuss the methodology that we followed to capture these requirements, which involved defining a conceptual service lifecycle model, carrying out a series of Design Thinking workshops, and formalising requirements based on an agile requirements information model. Then, we present the key requirements identified through this process in the form of summarised results.

Market analysts have foreseen the emergence of cloud brokers in the mediation of cloud services. But rather than focus on current kinds of intermediary role, it is more constructive to consider the kinds of brokerage capability that could be offered in the future, which go far beyond the integration, aggregation and customization services available today. This paper identifies advanced capabilities for cloud service governance, quality assurance and optimization that will be critical in catalyzing the emergence of cloud service ecosystems, environments in which all parties will find their symbiotic niches. It shows the path whereby a platform provider could evolve to become the hub of a cloud service ecosystem, through gradually taking on more of these advanced brokerage capabilities. The paper provides an overview of work conducted by the EU FP7 Broker@Cloud project towards realizing these advanced brokerage capabilities.

In the future, the Cloud will evolve into a rich ecosystem of service providers and consumers, each building upon the offerings of others. Cloud service brokers will play an important role, mediating between providers and consumers. As well as providing vertical integration and value-added aggregation of services, brokers will play an increased role in continuous quality assurance and optimization. This may range from setting common standards for service specification, providing mechanisms for lifecycle governance and service certification, to automatic arbitrage respecting consumer preferences, continuous optimization of service delivery, failure prevention and recovery at runtime. This workshop introduces some of these anticipated methods and investigates some of the mechanisms envisaged in future Cloud service brokerage.

Almost all organisations, irrespective of their size, rely on IT systems. Individuals are making increasing use of web applications for personal and social activities. Furthermore, as systems in general become more technologically advanced, an increasing proportion of the implementation is in software rather than in hardware. As a result, the quality and reliability of systems and the data they contain is increasingly dependent on the correctness, robustness and security of the associated software. The trend is clear: software is becoming pervasive and, as a result, software quality assurance, in the widest sense, is becoming a crucial everyday concern for all of us. Software testing is, by far, the most important means of assuring software quality. The Testing: Academic and Industrial Conference-Practice and Research Techniques 2010 (TAIC-PART 2010) was the fifth conference in a series of highly successful events. The conference brought together industrialists and academics to promote collaboration on problems in software testing. Among the wide range of topics in computer science and software engineering, software testing is an ideal candidate for academic and industrial collaboration because advances in research can have such wide-ranging and far-reaching implications for industry. Conversely, the advances in computing and communications technology and the growth of the associated software engineering activity are producing new research challenges at an increasing rate. TAIC-PART 2010 received forty paper submissions. After a rigorous reviewing process in which each paper was subjected to at least three independent reviews followed by programme committee discussion, 15 full papers and seven abstracts were accepted. One of these submissions was from industry, four were from collaborations between industry and academia and the remainder came from academia. The papers originated from 13 countries in Europe, North and South America, and Asia. The two best papers selected by the editors for publication in this special section are significantly extended versions of those TAIC-PART papers that received the strongest support from the referees. Each of the extended papers has been refereed by at least three expert reviewers and has undergone revisions as a result. They not only address important issues in security testing and fault localisation but also use novel and interesting techniques that could be deployed more widely. The first paper by Ben Kam and Thomas Dean, Linguistic security testing for text communication protocols considers the problem of protocol security testing. Protocols rely heavily on syntactic structures, and the authors adopt a syntax-based approach to security testing. The basic idea is to capture existing input to the application under test. The input is then parsed into an abstract syntax tree. Using a set of rewrite rules, the input is transformed to create test cases. A key contribution of their work is that they use a relatively simple context-free grammar together with XML mark-up rules to specify lexical, syntactic and context sensitive constraints. From this augmented grammar, they automatically generate a program that inserts XML tags into the appropriate parts of the captured test input. Multiple test generators can then use the tagged test input to produce different test cases, which are used to validate the security protocols of the software under test. In a case study, the authors tested an application that uses the iCalendar protocol for the communication of calendar information. Just over a thousand test cases were generated from a sample input; and upon execution of those test cases, an error was found, corresponding to a weakness in the iCalendar security protocol. Overall, the Syntax-based Security Testing framework is a relatively lightweight way of producing a protocol independent testing tool. New testing schemes can be accommodated by augmenting the grammar to add additional mark-up and creating the appropriate test case generators. The second paper by Mihai Nica, Simona Nica and Franz Wotawa, On the use of mutations and testing for debugging, is concerned with the problem of debugging but from a testing viewpoint. The main goal is to use testing as a way of localising faults within software. The underlying concept is to treat the statements in a program as a set of equations on the variables that appear in those statements. Each test case constrains the values of input and output variables. A failing test produces an inconsistency in the system of equations. The fault localisation process works by attempting to restore consistency. By removing individual statements until the system of equations is consistent, it is possible to identify candidate faulty statements, of which there are more than the actual number of faults. These candidates are subject to mutation, as a way of attempting to effect a repair. Candidates are retained if their repaired mutation causes an increase in consistency, whereas other candidates are discarded. Further test cases may be introduced to compare alternative surviving candidates and alternative repairs, which may need some input from the tester, or an oracle to help localise the fault. The approach is largely automated, but there are limitations as to the program constructs that can be represented as a set of equations. In addition, the constraint solver does not scale to large programs. In spite of these limitations, however, the approach is applied to a real world program with encouraging results. Both papers have a strong technical and also an empirical content. They capture the interplay of research and practice that is central to the aim of the conference and are fine examples of research in software testing. Finally, we would like to thank all the authors and reviewers for their effort and time. Furthermore, we would also like to thank the editors-in-chief of Software: Practice and Experience, Nigel Horspool and Andy Wellings, and the editorial staff for their support.

In previous work we have described how refinements can be checked using a temporal logic based model-checker, and how we have built a model-checker for Z by providing a translation of Z into the SAL input language. In this paper we draw these two strands of work together and discuss how we have implemented refinement checking in our Z2SAL toolset.The net effect of this work is that the SAL toolset can be used to check refinements between Z specifications supplied as input files written in the LATEXmark-up. Two examples are used to illustrate the approach and compare it with a manual translation and refinement check.

A comparative study was conducted into three Java unit-testing tools that support automatic test-case generation or test-case evaluation: Randoop, JWalk and μJava. These tools are shown to adopt quite different testing methods, based on different testing assumptions. The comparative study illustrates their respective strengths and weaknesses when applied to realistically complex Java software. Different trade-offs were found between the testing effort required, the test coverage offered and the maintainability of the tests. The conclusion evaluates how effective these tools were as alternatives to writing carefully hand-crafted tests for testing with JUnit.

Formal methods and testing are two important approaches that assist in the development of high-quality software. While traditionally these approaches have been seen as rivals, in recent years a new consensus has developed in which they are seen as complementary. This article reviews the state of the art regarding ways in which the presence of a formal specification can be used to assist testing.

Recently there has been a focus on test first and test driven development; several empirical studies have tried to assess the advantage that these methods give over testing after development. The results have been mixed. In this paper we investigate nine teams who tested during coding to examine the effect it had on the external quality of their code. Of the top three performing teams two used a documented testing strategy and the other an ad-hoc approach to testing. We conclude that their success appears to be related to a testing culture where the teams proactively test rather than carry out only what is required in a mechanical fashion.

We propose a benchmark for object-oriented unit testing, called the behavioural response. This is a normative set of state- and equivalence partition-based test cases. Metrics are then defined to measure the adequacy and effectiveness of a test set (with respect to the benchmark) and the efficiency of the testing method (with respect to the time invested). The metrics are applied to expert manual testing using JUnit, and semi-automated testing using JWalk, testing a standard suite of classes that mimic component evolution.