Graphical User Interfaces (GUIs) are amongst the most common user interfaces, enabling interactions with applications through mouse movements and key presses. Tools for automated testing of programs through their GUI exist, however they usually rely on operating system or framework specific knowledge to interact with an application. Due to frequent operating system updates, which can remove required information, and a large variety of different GUI frameworks using unique underlying data structures, such tools rapidly become obsolete, Consequently, for an automated GUI test generation tool, supporting many frameworks and operating systems is impractical. We propose a technique for improving GUI testing by automatically identifying GUI widgets in screen shots using machine learning techniques. As training data, we generate randomized GUIs to automatically extract widget information. The resulting model provides guidance to GUI testing tools in environments not currently supported by deriving GUI widget information from screen shots only. In our experiments, we found that identifying GUI widgets in screen shots and using this information to guide random testing achieved a significantly higher branch coverage in 18 of 20 applications, with an average increase of 42.5% when compared to conventional random testing.

Programs that use a Natural User Interface (NUI) are not controlled with a mouse and keyboard, but through input devices that monitor the user's body movements. Manually testing applications through such interfaces is time-consuming. Generating realistic test data automatically is also challenging, because the input is a complex data structure that represents real body structures and movements. Previously, it has been shown that models learned from user interactions can be used to generate tests for NUI applications controlled by the Microsoft Kinect. In this paper, we study the case of the Leap Motion input device, which allows applications to be controlled with hand movements and finger positions, resulting in substantially more complex input data structures. We present a framework to model human hand data interacting with applications, and generate test data automatically from these models. We also evaluate the influence of the training data, as well as the influence of using a single model of the complete user data vs. multiple models for the different aspects of hand movement (e.g., finger positions, hand positions, hand rotations). Experiments on five applications controlled by the Leap Motion demonstrate that our approach generates effective test data. The quality and quantity of the training data used to derive the models is the main factor that determines their effectiveness. On the other hand, the effects of using multiple (as opposed to single) models are minor and application specific.

Writing good software tests is difficult and not every developer's favorite occupation. Mutation testing aims to help by seeding artificial faults (mutants) that good tests should identify, and test generation tools help by providing automatically generated tests. However, mutation tools tend to produce huge numbers of mutants, many of which are trivial, redundant, or semantically equivalent to the original program, automated test generation tools tend to produce tests that achieve good code coverage, but are otherwise weak and have no clear purpose. In this paper, we present an approach based on gamification and crowdsourcing to produce better software tests and mutants: The Code Defenders web-based game lets teams of players compete over a program, where attackers try to create subtle mutants, which the defenders try to counter by writing strong tests. Experiments in controlled and crowdsourced scenarios reveal that writing tests as part of the game is more enjoyable, and that playing Code Defenders results in stronger test suites and mutants than those produced by automated tools.

Search-based testing supports developers by automatically generating test suites with high coverage, but the effectiveness of a search-based test generator depends on numerous parameters. It is unreasonable to expect developers to understand search algorithms well enough to find the optimal parameter settings for a problem at hand, and even if they did, a static value for a parameter can be suboptimal at any given point during the search. To counter this problem, parameter control methods have been devised to automatically determine and adapt parameter values throughout the search. To investigate whether parameter control methods can also improve search-based generation of test suites, we have implemented and evaluated different methods to control the crossover and mutation rate in the EvoSuite unit test generation tool. Evaluation on a selection of open source Java classes reveals that while parameter control improves the values of mutation and crossover rate successfully during runtime, the positive effects of this improvement are often countered by increased costs of fitness evaluation.