Existing research in mobile app compatibility analysis faces two critical limitations: the absence of large-scale real-world datasets and the lack of systematic taxonomies for tool evaluation. To address these gaps, this study proposes a novel two-dimensional classification framework that categorizes compatibility issues through two analytical dimensions: causes (including API-induced issues such as deprecated methods, system-induced issues like permission changes, and hardware-induced issues such as sensor variations) and consequences (encompassing crashes, functional failure, UI inconsistencies, and performance degradation). Based on this classification framework, we developed CompatBench, the first manually curated dataset containing 78 real-world cases collected from GitHub submissions spanning 2017 to 2024. The dataset comprises 45 researcher-reproduced issues and 33 user-reported issues. Key findings reveal significant discrepancies between laboratory reproduction scenarios and real-world cases. API-related causes dominate reproduction cases at 91.1 %, but account for only 42.4 % of user-reported issues. Regarding manifestation visibility, 35.6 % of reproduction cases lack clear symptoms, whereas 96.9 % of real cases exhibit observable failures such as crashes or UI inconsistencies. Repair dynamics analysis shows 88.8 % of reproducible issues resolved within 50 days, contrasting with a 55.7 % fix rate for real issues. These results highlight critical methodological gaps in current research paradigms, particularly in causal inference based on observable consequences and consequence prediction based on potential causes. The study establishes three core contributions for validating compatibility management tools: a standardized taxonomy for issue classification, an openaccess dataset with real-world cases, and empirical evidence demonstrating discrepancies between laboratory and field observations. The findings emphasize the necessity of incorporating real-world datasets to improve evaluation validity in compatibility research.

Open-source software (OSS) greatly facilitates program development for developers. However, the high number of vulnerabilities in open-source software is a major concern, including in Golang, a relatively new programming language. In contrast to other commonly used OSS package managers, Golang presents a distinctive feature whereby commits are prevalently used as dependency versions prior to their integration into official releases. This attribute can prove advantageous to users, as patch commits can be implemented in a timely manner before the releases. However, Golang employs a decentralized mechanism for managing dependencies, whereby dependencies are upheld and distributed in separate repositories. This approach can result in delays in the dissemination of patches and unresolved vulnerabilities. To tackle the aforementioned concern, a comprehensive investigation was undertaken to examine the life cycle of vulnerability in Golang, commencing from its introduction and culminating with its rectification. To this end, a framework was established by gathering data from diverse sources and systematically amalgamating them with an algorithm to compute the lags in vulnerability patching. It turned out that 66.10% of modules in the Golang ecosystem were affected by vulnerabilities. Within the vulnerability life cycle, we found two kinds of lag impeding the propagation of vulnerability fixing. By analyzing reasons behind non-lagged and lagged vulnerabilities, timely releasing and indexing patch versions could significantly enhance ecosystem security.

The compatibility issues caused by Android fragmentation have become a vital task in the development of Android applications. To locate those issues, thousand of crowd testers run apps on different devices with different configurations to achieve the largest coverage, which might be costly and time-consuming. Since existing approaches to selecting optimal devices are device-side analysis without the information of the internal structures of apps, app-side analysis that flags the essential devices for testers has remained elusive. To mitigate this gap of compatibility crowdtesting, this paper proposes an app constraint analysis approach named CompatDroid to generate the optimal device set to guide crowd testers. By evaluating 46 benchmark apps on 14 SDK versions, the optimal device sets are successfully generated, and CompatDroid only needs no more than 7 Android versions to achieve almost the same code coverage (i.e., 33.13%) testing on all 14 android versions (i.e., 34.65%) in 36 of 46 apps, which indicates that it can drastically reduce the consumption of test resources while losing little test coverage. On a larger dataset, CompatDroid successfully analyzes 98.3% of 645 apps, in which the median number of the optimal SDK versions set is 2.5 versions, and 68.92% of those apps contain the constraints of SDK version (i.e., SDK version) while 84.86% of them do not have the constraints of hardware information (i.e., model name and manufacture name).

Android API-related compatibility issues have be-come a severe problem and significant challenge for app devel-opers due to the well-known Android fragmentation issues. To address this problem, many effective approaches such as app-based and API lifetime-based methods have been proposed to identify incompatible API usages. However, due to the various implementations of API usages and different API invoking paths, there is still a significant weakness of existing approaches, i.e., introducing a massive number of false positives (FP) and false negatives (FN). To this end, in this paper, we propose PSDroid, an automated compatibility detection approach for Android apps, which aims to reduce FPs and FNs by overcoming several technical bottlenecks. Firstly, we make substantial efforts to carry out a preliminary study to summarize a set of novel API usages with diverse checking implementations. Secondly, we construct a refined API lifetime database by leveraging a semantic resolving analysis on all existing Android SDK frameworks. Based on the above two key phases, we design and implement a novel path-sensitive semantic approach to effectively and automatically detect incompatibility issues. To demonstrate the performance, we compared with five existing approaches (i.e., FicFinder, ACRYL, CIDER, IctAPIFinder, and CID) and the results show that PSDroid outperforms existing tools. We also conducted an in-depth root cause analysis to comprehensively explain the ability of PSDroid in reducing FPs and FNs. Finally, 18/30 reported issues have been confirmed and further fixed by app developers.

The autonomous system is an extremely complex and multidisciplinary system, and the overall goal of this paper is to study safety testing methods for autonomous systems to enhance the robustness of autonomous systems. In order to realize an autonomous system with high reliability and robustness, this paper summarizes the security defect detection methods for autonomous systems from the two dimensions of static analysis and dynamic testing and expounds on the syntax-level, semantic, and system-level testing methods for autonomous systems.

Most of metamorphic testing (MT) research works focused on the generation and application of metamorphic relations (MRs). There is no clear conclusion about the relationship between test case generation methods and performance of MT. In this article, we introduce a novel method based on adaptive random testing (ART) and MR for MT test case generation. It proposes a family of algorithms for MT test cases generation, named as MT based ART (MT-ART). Three distances are measured to generate the next MT test case. In order to verify the performance of this method, series of experiments on four programs with different numbers of inputs are introduced. The results show that MT-ART performs better than other ART algorithms not only in test effectiveness, but also in test efficiency and test coverage. Based on this article, the following conclusions can be drawn: first, considering the effectiveness of MRs and test cases in MT may lead to better results. In this way, most of the existing research can be improved by this method. This is the most important contribute of our research. Second, not only the source test cases, but also the follow-up test cases can improve the performance of MT. Therefore, they should be considered together during the process of the next test case generation. Third, the average distance performs better than the max distance and the minimum distance in metamorphic test case selection.

With the prosperity of Mobile APPs, developers need to dynamically find the compatibility issues by testing APP on all Android versions and devices, which is costly. This paper finds a systematic test method based on source codes of apps to be tested by identify compatibility-related characteristic codes. We propose an automated tool named “Periph” to search the source code of each app that can eventually generate compact versions and devices set to guide the testing. Through testing 21 apps on 13 Android versions, we find that our proposed tool can greatly reduce the consumption of test resources while only losing little test coverage.

The number of mobile applications has increased geometrically nowadays, but how to ensure their quality and conduct adequate and effective testing is still a challenge for developers. On the one hand, the number of mobile apps is increasing, and the update speed is faster and faster. Many small and medium-sized companies can hardly test the app adequately before each release. On the other hand, mobile apps play more and more important roles in peoples life, such as financial payment. For the sake of company security and user privacy, most companies will encrypt key codes in their APP. Even third-party testers cannot get source code, which also leads to many researchers cannot carry out further research and effective testing for these widely used mainstream APPs. Code coverage is an important indicator to guide software testing, which plays a crucial role in ensuring the quality of testing. However, it is an urgent problem to find accurate coverage indictors to evaluate these tests. And when testing those existing widely used mainstream closed-source apps, we find that the existing coarse-grained coverage metrics like method coverage is bad coverage indictors for app testing that can exaggerate or minimize the actual coverage rate, which cannot obtain satisfactory results for the evaluation of test effects. To find a more reliable coverage indictor, this paper demonstrates the correctness of instruction coverage indictor and the inaccurate of method coverage in evaluating the test of closed-source APP from the perspective of probability and statistics. Then we shows how inaccurate the method coverage can be through an empirical evaluation on datasets of closed source APPs and open source APPs respectively. It is further verified that instruction coverage is a more effective evaluation indictor than methods coverage or activity coverage in the test of closed source APP for the first time.

It is difficult to apply traditional testing adequacy criteria when measuring the adequacy of convolutional neural network applications. However, only a small number of test cases applied to the CNN model can achieve neuron coverage of almost 100%, overturning the effectiveness of the neuronal coverage criteria. In this paper, we propose a model coverage criterion based on mutation testing for CNN, and applying model coverage criterion to a common CNN image classification models (LeNet-5). we focus on the testing accuracy of model. Experiments show that our method can find the local optimal model and play an important role in improving the testing adequacy of the set of models.

As a vehicle for recording and tracking defects, bug reports provide a basis for solving software quality problems.Currently, software testing is often carried out in a multi-person and parallel state.The integration process of numerous bug reports, such as moving fake or duplication bug reports, is facing severe challenges.Therefore, this paper proposes an automatic detection modus for bug reports based on the vector space model.After pre-processing the bug report, a matching library is created according to the test requirements and test report samples.The vector space model is used to calculate the similarity between the two, and the correctness of the bug report is detected based on this.Experiments with the data of a software test contest show that the modus proposed in this paper can correctly judge most bug reports, effectively improving the efficiency of de-false and de-duplication.

One of the key problems in app testing is to improve the test coverage of apps. However, current testing techniques, whether in code coverage or activity coverage, are not satisfactory. To address this limitation, we present an algorithm for generating test runtime-environment-set to exercise mobile apps. Our approach is based on code analysis to systematically test the targeted code of the Android apps. It analyzes the decompiled code that identifies the code related to Android SDK version, generating the corresponding test cases with the runtime-environment set. We also implement our approach on Android, and validate the method with the existing widely used strategies. An empirical study of the practical usefulness of the technique has been presented on 6 widely-used industrial apps. 18 unique crashes have been found, and the method coverage has been increased by far 9.8% to 130.4% on those apps.

Machine learning has been becoming increasingly popular and widely-used in various industry domains. The presence of the oracle problem, however, makes it difficult to ensure the quality of this kind of software. Furthermore, the popularity of machine learning and its application has attracted many users who are not experts in this field. In this paper, we report on using a recently introduced method called metamorphic exploration where we proposed a set of hypothesized metamorphic relations for an unsupervised clustering program, Weka, to enhance understanding of the system and its better use.

Computer science (CS) subjects have been rapidly growing in popularity, and demand for CS education and training has put increasing pressure on teaching resources in higher education (HE) and elsewhere. HE in the People's Republic of China (PRC) has also been developing, with one product of this evolution being Sino-foreign HE institutions (SfHEIs). Much of the popularity growth for CS can be linked to the growth of CS-based technology and innovation, especially in the form of Artificial Intelligence (AI) and Machine Learning (ML). AI/ML-based innovation has been forecast to offer increases in quality of life for consumers. However, AI/ML systems face a challenge for software quality assurance (SQA): They are so-called “untestable systems” - identifying the correctness of AI/ML system outputs or behaviour may not be feasible. Preparing SQA professionals to be able to ensure AI/ML SQA will require innovative and creative education and training. An SQA approach called metamorphic testing (MT) has a proven track record of alleviating the oracle problem, and has great potential as a testing methodology for AI/ML systems. Metamorphic exploration (ME) is a new addition to the MT literature, and involves developing the user's understanding of the system under study. This paper reports on experiences at an SfHEI of using ME and MT to test an AI/ML system.

With widely applied in various fields, deep learning (DL) is becoming the key driving force in industry. Although it has achieved great success in artificial intelligence tasks, similar to traditional software, it has defects that, once it failed, unpredictable accidents and losses would be caused. In this paper, we propose a test cases generation technique based on an adversarial samples generation algorithm for image classification deep neural networks (DNNs), which can generate a large number of good test cases for the testing of DNNs, especially in case that test cases are insufficient. We briefly introduce our method, and implement the framework. We conduct experiments on some classic DNN models and datasets. We further evaluate the test set by using a coverage metric based on states of the DNN.

As a vehicle for recording and tracking defects, bug report provides basis for solving software quality problems. However, moving fake or duplication bug report in multi-person and parallel software testing project is a labor-intensive job. Therefore, this paper proposes a method based on vector space model for automatic dealing with this problem. We built a matching library according to the test requirements and confirmed bug reports and used vector space model to calculate the similarity between the bug report and the matching library. Then the correctness of the bug report is detected based on this similarity.

University of Nottingham Ningbo China (UNNC) was the first Sino-foreign higher education institution (SmEI) to be established in Mainland China, and has been host to a number of student-staff project partnerships, including in the successful development of Open Education Resources (OERs). This paper reports on the completion of the first phase of an anticipated 2-year student-staff project to develop a language learning application as an OER. It explains the history and context of UNNC and the project, and discusses some of the software engineering processes completed to date. The work completed so far is described and reflected upon, and the future work and impact discussed.