Black-Box Test Case Prioritization Using Log Analysis and Test Case Diversity

Regression testing is a software testing type that examines whether updates made in the software impact the existing functionality of the application. Depressingly, the long testing time and high testing costs make regression testing very expensive. Test case prioritization (TCP) stands out as one of the extensively researched regression testing techniques. It prioritizes test cases to optimize their execution order, aiming to maximize the prioritization goals and reveal faults earlier to provide feedback to testers. The TCP technique based on log analysis (LogTCP) designs the prioritization strategy using logs generated during test case execution. However, LogTCP’s performance is limited by its inability to incorporate the diversity of test cases for sorting. To overcome these concerns, we propose a method to implement TCP using k-means clustering and log analysis(called KL-TCP), that takes into account both log information and test case diversity. We examine the effectiveness of this strategy in ten open source Java projects on GitHub. The experimental results show that our proposed method outperforms LogTCP method by detecting a higher average percentage of faults. The best average performance of the ten project experiments reached 0.77(APFD).