Testing Real-Time Embedded Systems (RTES) is a non-trivial task - time adds a new dimension to the complexity of the testing process. In previous research, we introduced a ‘priority-based’ approach which tested the logical and timing behaviour of an RTES modelled formally as Uppaal timed automata. In this paper, we develop a novel Analytical Hierarchy Process (AHP) decision-making framework for our priority-based testing approach that provides testers with a systematic approach through which they can prioritize the available testing sets that best fulfils their testing requirements.
Unit testing in the presence of side effects requires the construction of a suitable test fixture before each test run. We consider the problem of providing test fixtures for unit testing of client-side JavaScript code that manipulates its underlying web page. We propose using techniques from software transactional memory to restore the test fixture after each test run.
We describe a method and algorithm for model-based construction of an on-line reactive planning tester (RPT) for black-box testing of state based systems specified by non-deterministic extended finite state machine (EFSM) models. The key idea of RPT lies in off-line preprocessing of the System Under Test (SUT) model to prepare the data for efficient on-line reactive test planning. A test purpose is attributed to the transitions of the SUT model by a set of Boolean conditions called traps. The result of the off-line analysis is a set of constraints used in on-line testing for guiding the SUT towards taking the moves represented by trap-labelled transitions in SUT model and generating required data for inputs. We demonstrate the results on a simple example and discuss the practical experiences of using the proposed method.
Inductive inference is the process of hypothesizing a model from a set of examples. It can be considered to be the inverse of program testing, which is the process of generating a finite set of tests that are intended to fully exercise a software system. This relationship has been acknowledged for almost 30 years, and has led to the emergence of several induction-based techniques that aim either to generate suitable test sets or assess the adequacy of existing test sets. Unfortunately these techniques are usually deemed to be too impractical, because they are based on exact inference, requiring a vast set of examples or tests. In practice a test set can still be adequate if the inferred model contains minor errors. This paper shows how the Probably Approximately Correct (PAC) framework, a well-established approach in the field of inductive inference, can be applied to inductive testing techniques. This facilitates a more pragmatic assessment of these techniques by allowing for a degree of error. This evaluation framework gives rise to a challenge: To identify the best combination of testing and inference techniques that produce practical and (approximately) adequate test sets.
Key aspects of software development, in particular testing and debugging, still commonly rely on manual techniques. If the programs contain enough built-in correctness information, in the form of contracts, it is possible to automate some of the most tedious and error-prone aspects of testing, and even to perform corrections automatically. At both ETH and Eiffel Software we have developed such techniques and integrated them into Eiffel programming environments, in the form of tools for automated program testing and automated program fixing: the AutoTest and AutoFix frameworks. Some of the work is still research in progress, but many results have already been integrated in the production version of the EiffelStudio environment.
