Back-to-Back Testing Framework Using a Machine Learning Method

In back-to-back testing of software, expected outputs (test oracles) are generated from software that is similar to SUT (software under test), and are compared with test outputs from the SUT in order to reveal faults. The advantages of back-to-back testing are that one can automatically perform the creation of expected outputs that is one of the most costly processes in software testing, and one can obtain detailed expected outputs that are not limited to a specific aspect, such as state transitions. However, it is not easy to automatically classify the differences between the test outputs and the expected outputs into two groups, that is, one resulting from failures of the SUT and another resulting from intended functional differences between the SUT and the similar software. The manual classification is too costly and back-to-back testing can hardly be applied unless the functions of the similar software are exactly equal to the intended functions of the SUT. To solve this costly classification problem, this paper proposes a novel back-to-back testing framework in which a SVM (support vector machine) classifies them automatically.