Machine Learning (ML) is the discipline that studies methods for automatically inferring models from data. Machine learning has been successfully applied in many areas of software engineering including: behaviour extraction, testing and bug fixing. Many more applications are yet to be defined. Therefore, a better fundamental understanding of ML methods, their assumptions and guarantees can help to identify and adopt appropriate ML technology for new applications. In this chapter, we present an introductory survey of ML applications in software engineering, classified in terms of the models they produce and the learning methods they use. We argue that the optimal choice of an ML method for a particular application should be guided by the type of models one seeks to infer. We describe some important principles of ML, give an overview of some key methods, and present examples of areas of software engineering benefiting from ML. We also discuss the open challenges for reaching the full potential of ML for software engineering and how ML can benefit from software engineering methods.
Automata learning is an established class of techniques for inferring automata models by observing how they respond to a sample of input words. Recently, approaches have been presented that extend these techniques to infer extended finite state machines (EFSMs) by dynamic black-box analysis. EFSMs model both data flow and control behavior, and their mutual interaction. Different dialects of EFSMs are widely used in tools for model-based software development, verification, and testing. This survey paper presents general principles behind some of these recent extensions. The goal is to elucidate how the principles behind classic automata learning can be maintained and guide extensions to more general automata models, and to situate some extensions with respect to these principles.
In recent years, research efforts have been directed towards the use of Machine Learning (ML) techniques to support and automate activities such as specification mining, risk assessment, program analysis, and program repair. The focus has largely been on the use of machine learning black box methods whose inference mechanisms are not easily interpretable and whose outputs are not declarative and guaranteed to be correct. Hence, they cannot readily be used to inform the elaboration and revision of declarative software models identified to be incorrect or incomplete. On the other hand, recent advances in ML have witnessed the emergence of new logic-based machine learning approaches that overcome such limitations and which have been proven to be well-suited for many software engineering tasks. In this chapter, we present a survey of the state-of-the-art of logic-based machine learning techniques, highlight their expressivity, define their different underlying semantics, and discuss their efficiency and the heuristics they adopt to guide the search for solutions. We then demonstrate the application of this type of machine learning to (declarative) specification refinement and revision as a complementary task to program analysis.
We present a survey of recent progress in the area of learning-based testing (LBT). The emphasis is primarily on fundamental concepts and theoretical principles, rather than applications and case studies. After surveying the basic principles and a concrete implementation of the approach, we describe recent directions in research such as: quantifying the hardness of learning problems, over-approximation methods for learning, and quantifying the power of model checker generated test cases. The common theme underlying these research directions is seen to be metrics for model convergence. Such metrics enable a precise, general and quantitative approach to both speed of learning and test coverage. Moreover, quantitative approaches to black-box test coverage serve to distinguish LBT from alternative approaches such as random and search-based testing. We conclude by outlining some prospects for future research.
In this chapter we examine the problem of testing functional black-box programs that do not require sequential inputs. We specifically focus on the case where there is no existing specification from which to derive tests. Research into this problem dates back over three decades, and has produced a variety of techniques, all of which employ various types of data mining and machine learning algorithms to examine test executions and to inform the selection of new tests. Here we provide an overview of these techniques and examine their limitations and opportunities for future research.
Active automata learning is slowly becoming a standard tool in the toolbox of the software engineer. As systems become ever more complex and development becomes more distributed, inferred models of system behavior become an increasingly valuable asset for understanding and analyzing a system's behavior. Five years ago (in 2011) we have surveyed the then current state of active automata learning research and applications of active automata learning in practice. We predicted four major topics to be addressed in the then near future: efficiency, expressivity of models, bridging the semantic gap between formal languages and analyzed components, and solutions to the inherent problem of incompleteness of active learning in black-box scenarios. In this paper we review the progress that has been made over the past five years, assess the status of active automata learning techniques with respect to applications in the field of software engineering, and present an updated agenda for future research.
Any complex software system exhibits a tension between the technical perspective required for its realization and the user-level perspective. We term this the "how-what gap" , represented by the questions "how is a system implemented" vs. "what is its functionality/usage". The normative, anticipated behavior of a software system as envisaged during its development and the de facto, observed behavior emerging after its continued operation tends to drift apart, resulting in behavioral inconsistency. We discuss how behavioral consistency in software systems can be captured in technical and formal terms, we sketch a possible tool chain that could support it, and we describe some of the research challenges that must be solved. Our main idea is to combine software analysis approaches represented by various forms of static analysis and formal verification with runtime verification, monitoring, and automata learning in order to optimally leverage the de facto observed behaviour of the deployed systems.
Active inference algorithms that are used to extract behavioural models of software systems usually assume that the System Under Inference (SUI) can be reset. Two approaches have been proposed to infer systems that cannot be reset. Rivest and Schapire proposed an adaptation of the L* algorithm that relies on having a homing sequence for the SUI. We detail here another approach that is based on characterization sequences. More precisely, we assume classical testing hypotheses, namely that we are given a bound n on the number of states and a set W of characterizing sequences to distinguish states. Contrary to L*, it does not require an external oracle to decide on equivalence. The length of the test sequence is polynomial in n and the exponent depends on the cardinality vertical bar W vertical bar of the characterization set. For systems where resetting is impossible or expensive, this approach can be a viable alternative to classical learning methods.
We present a survey of the recent research efforts in integrating model learning with model-based testing. We distinguished two strands of work in this domain, namely test-based learning (also called test-based modeling) and learning-based testing. We classify the results in terms of their underlying models, their test purpose and techniques, and their target domains.
Health is an elementary foundation of prosperous human life. Average human life expectancy has never been as long as it is today and medical advances have greatly improved overall population health. However, modern societies are burdened by new complications in the form of lifestyle diseases which arise due to various aspects of modern life, such as sedentary behavior. The pressure on public health systems is ever increasing with the emergence of further complex and expensive treatment options, and due to the complications resulting from demographic change. The technological advancements of the industrial and information age, the computational revolution in general, and video games for entertainment specifically contribute to the prevalence of some prevalent lifestyle-related health issues. At the same time, computing devices and interactive applications also play an important role in improving all areas of individual and public health. Recent research and early commercial releases deliver convincing evidence that playful applications and games for health in particular offer approaches that can help overcome the motivational barriers which often restrain successful health treatments or preventive actions and behavior. This chapter provides an overview of the arguments that motivate the application of play and game techniques for personal and public health. It summarizes the basic promises and challenges of games for health research and development, provides starting points regarding their design and implementation, illustrates selected aspects along the lines of exemplary applications, and hints at pressing open challenges as well as promising avenues for further research and developments. A selection of quality references for further reading is included in the last section.
Content is a key component for successful computer games and it is also one of the most labour and time intensive tasks a game developer can face. As the scale of contemporary games increases, players come to expect higher standards of fidelity and immersion, thus increasing the need to create large amounts of content. This chapter focuses on the creation of content for serious games, particularly research that can aid designers and game developers in generating large amounts of content quickly and effectively. While game developers have had many tools and methods for creating content for standard computer games, this chapter will concentrate on the types of content specific to serious games.
A serious game needs to combine a number of different aspects to help the end user in reaching the desired effects. This requires incorporating a broad range of different aspects in the design, stemming from a broad range of different fields of expertise. For designers, developers, researchers, and other stakeholders it is not straightforward how to organize the design and development process, to make sure that these aspects are properly addressed. In this chapter we will discuss a number of ways of organizing the design and development process and various models that support specific design decisions during this process, concluding with a discussion of design patterns for serious games.
While it might seem desirable to "program" games "close to a high-level specification", the pragmatic reality has not, until very recently, allowed this. In this chapter we discuss the fundamental aspects that define a game and show how these aspects are captured by means of the so-called game development tools. In particular, we show: (i) how the various historical tools have always been intrinsically inspired by the dominant programming languages/paradigms that were the most popular at the time when the tool was developed; and (ii) we discuss these tools by comparing their pro and cons (typically imposed by the chosen language/paradigm or the available hardware) in order to understand the most important requirements for their next generations.
Current crowdsourcing platforms typically concentrate on simple microtasks and do not meet the needs of academic research well, where more complex, time consuming studies are required. This has lead to the development of specialised software tools to support academic research on such platforms. However, the loose coupling of the software with the crowdsourcing site means that there is only limited access to the features of the platform. In addition, the specialised nature of the software tools means that technical knowledge is needed to operate them. Hence there is great potential to enrich the features of crowdsourcing platforms from an academic perspective. In this chapter we discuss the possibilities for practical improvement of academic crowdsourced studies through adaption of technological solutions.
Crowdsourcing solutions are increasingly being adopted across a variety of domains these days. An important consequence of the flourishing crowdsourcing markets is that experiments which were traditionally carried out in laboratories on a much smaller scale can now tap into the immense potential of online labor. Researchers in different fields have shown considerable interest in attempting to carry out priorly constrained lab experiments in the crowd. In this chapter, we reflect on the key factors to consider while transitioning from controlled laboratory experiments to large scale experiments in the crowd.
Human-centred empirical evaluations play important roles in the fields of human-computer interaction, visualisation, and graphics. The advent of crowdsourcing platforms, such as Amazon Mechanical Turk, has provided a revolutionary methodology to conduct human-centred experiments. Through such platforms, experiments can now collect data from hundreds, even thousands, of participants from a diverse user community over a matter of weeks, greatly increasing the ease with which we can collect data as well as the power and generalisability of experimental results. However, such an experimental platform does not come without its problems: ensuring participant investment in the task, defining experimental controls, and understanding the ethics behind deploying such experiments en masse. This book is intended to be a primer for computer science researchers who intend to use crowdsourcing technology for human centred experiments. It focuses on methodological considerations when using crowdsourcing platforms to run human-centred experiments, particularly in the areas of visualisation and of quality of experience (QoE) for online video delivery. We hope that this book can act as a primer to researchers in our fields that intend to run experiments on crowdsourcing for the purposes of human-centred experimentation.
This chapter introduces information processing perspectives from cognitive psychology, providing historical background content where it might prove useful. The hope is that this will provide readers enough of an understanding of psychology perspectives, theories, and methods that they can better apply crowdsourcing methods to understand the cognitive outcomes of interaction within visualization environments and other computer interfaces.
The driving force behind digital crowdsourcing are its workers: working, hidden behind the scenes, churning out data in experiments, participating in research studies, completing little tasks to accomplish HITs online. Understanding workers and crowdwork better is therefore key to develop a more effective and fair use of crowdsourcing for research. This chapter attempts to help develop an understanding of the various aspects of the crowd by drawing parallels between workers of different platforms (AMT, Microworkers and Crowdee) through quantitative and qualitative analysis of current and newly collected data. A picture of the crowd is drawn by uncovering their motivations, workplaces, skills and infrastructure, issues and perspectives about the design of microtasks, the employers and the microtask-based platforms. Legal and ethical perspectives on crowdwork are also discussed, and online resources are reviewed that researchers can use as a primer to employ crowdworkers in an ethical and fair way. The chapter provides information, a review of internationally recognised ethical principles and practical advice to those who would like to use crowdsourcing for experiments and to carry out research studies as an informed researcher and crowd employer.
Crowdsourcing offers great potential to overcome the limitations of controlled lab studies. To guide future designs of crowdsourcing-based studies for visualization, we review visualization research that has attempted to leverage crowdsourcing for empirical evaluations of visualizations. We discuss six core aspects for successful employment of crowdsourcing in empirical studies for visualization – participants, study design, study procedure, data, tasks, and metrics & measures. We then present four case studies, discussing potential mechanisms to overcome common pitfalls. This chapter will help the visualization community understand how to effectively and efficiently take advantage of the exciting potential crowdsourcing has to offer to support empirical visualization research.
This chapter contains an introduction to this book which aims at providing guidance to people who are interested in conducting or dealing with research in the area of entertainment computing and serious games. The chapter starts with defining key terminology. It then illustrates benefits and challenges in this area of research by discussing the development of educational games, one of the most recognized subsets of serious games. Describing and characterizing the current state of the research communities involved in entertainment computing and serious games, an overview of the research landscape is presented. Finally, an overview of the structure of this book is given and the individual chapters of this book are briefly summarized.
