Mobile cloud computing (MCC) is a technological paradigm for providing services to mobile device (MD) users. A compromised MD may cause harm to both its user and to other MCC customers. This study explores the use of machine learning (ML) models and stochastic methods for the protection of Android MDs connected to the mobile cloud. To test the validity and feasibility of the proposed models and methods, the study adopted a proof-of-concept approach and developed a prototype system named MINDPRESS. The static component of MINDPRES assesses the risk of the apps installed on the MD. It uses a device-based ML model for static feature analysis and a cloud-based stochastic risk evaluator. The device-based hybrid component of MINDPRES monitors app behavior in real time. It deploys two ML models and functions as an intrusion detection and prevention system (IDPS). The performance evaluation results of the prototype showed that the accuracy achieved by the methods for static and hybrid risk evaluation compared well with results reported in recent work. Power consumption data indicated that MINDPRES did not create an overload. This study contributes a feasible and scalable framework for building distributed systems for the protection of the data and devices of MCC customers.
In supporting the development of high-quality software, especially necessary in the era of LLMs, automated program repair (APR) tools aim to improve code quality by automatically addressing violations detected by static analysis profilers. Previous research tends to evaluate APR tools only for their ability to clear violations, neglecting their potential introduction of new (sometimes severe) violations, changes to code functionality and degrading of code structure. There is thus a need for research to develop and assess comprehensive evaluation frameworks for APR tools. This study addresses this research gap, and evaluates Sorald (a state-of-the-art APR tool) as a proof of concept. Sorald’s effectiveness was evaluated in repairing 3,529 SonarQube violations across 30 rules within 2,393 Java code snippets extracted from Stack Overflow. Outcomes show that while Sorald fixes specific rule violations, it introduced 2,120 new faults (32 bugs, 2088 code smells), reduced code functional correctness—as evidenced by a 24% unit test failure rate—and degraded code structure, demonstrating the utility of our framework. Findings emphasize the need for evaluation methodologies that capture the full spectrum of APR tool effects, including side effects, to ensure their safe and effective adoption.
—Cross-project defect prediction (CPDP) makes use of cross-project (CP) data to overcome the lack of data necessary to train well-performing software defect prediction (SDP) classifiers in the early stage of new software projects. Since the CP data (known as the source) may be different from the new project’s data (known as the target), this makes it difficult for CPDP classifiers to perform well. In particular, it is a mismatch of data distributions between source and target that creates this difficulty. Transfer learning-based CPDP classifiers are designed to minimize these distribution differences. The first Transfer learning-based CPDP classifiers treated these differences equally, thereby degrading prediction performance. To this end, recent research has proposed the Weighted Balanced Distribution Adaptation (W-BDA) method to leverage the importance of both distribution differences to improve classification performance. Although W-BDA has been shown to improve model performance in CPDP, research to date has failed to consider model performance in light of increasing target data or variances in data sampling. We provide the first investigation of when and to what extent the effect of increasing the target data and using various sampling techniques have when leveraging the importance of both distribution differences. We extend the initial W-BDA method and call this extension the W-BDA+‘ method. To evaluate the effectiveness of W-BDA+‘ for improving CPDP performance, we conduct eight experiments on 18 projects from four datasets where data sampling was performed with different sampling methods. We evaluate our method using four complementary indicators (i.e., Balanced Accuracy, AUC, F-measure and G-Measure). Our findings reveal an average improvement of 6%, 7.5%, 10% and 12% for these four indicators when W-BDA+‘ is compared to five other baseline methods (including W-BDA), for all four of the sampling methods used. Also, as the target to source ratio is increased with different sampling methods, we observe a decrease in performance for the original W-BDA, with our W-BDA+ approach outperforming the original W-BDA in most cases. Our results highlight the importance of adjusting for data imbalance and having an awareness of the effect of the increasing availability of target data in CPDP scenarios.
Just-In-Time (JIT) defect prediction aims to identify defects early, at commit time. Hence, developers can take precautions to avoid defects when the code changes are still fresh in their minds. However, the utility of JIT defect prediction has not been investigated in relation to crashes of mobile apps. We therefore conducted a multi-case study employing both quantitative and qualitative analysis. In the quantitative analysis, we used machine learning techniques for prediction. We collected 113 reliability-related metrics for about 30,000 commits from 14 Android apps and selected 14 important metrics for prediction. We found that both standard JIT metrics and static analysis warnings are important for JIT prediction of mobile app crashes. We further optimized prediction performance, comparing seven state-of-the-art defect prediction techniques with hyperparameter optimization. Our results showed that Random Forest is the best performing model with an AUC-ROC of 0.83. In our qualitative analysis, we manually analysed a sample of 642 commits and identified different types of changes that are common in crash-inducing commits. We explored whether different aspects of changes can be used as metrics in JIT models to improve prediction performance. We found these metrics improve the prediction performance significantly. Hence, we suggest considering static analysis warnings and Android-specific metrics to adapt standard JIT defect prediction models for a mobile context to predict crashes. Finally, we provide recommendations to bridge the gap between research and practice and point to opportunities for future research.
Software developers make use of on crowdsourcing during development. Beyond learning from others, developers use online portals such as Stack Overflow as a vehicle for collaboration. However, little is known about developers’ experiences on such platforms, particularly around problems that are encountered online. Such insights could benefit software developers in terms of recommendations for pitfalls to avoid, ways to exploit crowdsourced knowledge, and the provision of insights to improve online code sharing communities. We interviewed 50 practitioners to fill this gap, where outcomes show that software developers’ use of online portals is targeted, and such portals are a lifeline to modern software development. Practitioners are facilitated with code solutions and debugging, often in a very timely fashion. While these experiences are largely positive, practitioners also encounter negative experiences online, some of which could be significantly deleterious to the community. We discuss the implications of these findings, such as creating awareness of the quality and reliability of code snippets, improving code searches, code validation and outdated code detection and attribution of code snippets.
Context: Over the years, there has been debate about what constitutes software quality and how it should be measured. This controversy has caused uncertainty across the software engineering community, affecting levels of commitment to the many potential determinants of quality among developers. An up-to-date catalogue of software quality views could provide developers with contem-porary guidelines and templates. In fact, it is necessary to learn about views on the quality of code on frequently used online collaboration platforms (e.g., Stack Overflow), given that the quality of code snippets can affect the quality of software products developed. If quality models are unsuitable for aiding developers because they lack relevance, developers will hold relaxed or inappropriate views of software quality, thereby lacking awareness and commitment to such practices.Objective: We aim to explore differences in interest in quality characteristics across research and practice. We also seek to identify quality characteristics practitioners consider important when judging code snippet quality. First, we examine the literature for quality characteristics used frequently for judging software quality, followed by the quality characteristics commonly used by researchers to study code snippet quality. Finally, we investigate quality characteristics used by practitioners to judge the quality of code snippets.Methods: We conducted two systematic literature reviews followed by semi-structured interviews of 50 practitioners to address this gap. Results: The outcomes of the semi-structured interviews revealed that most practitioners judged the quality of code snippets using five quality dimensions: Functionality, Readability, Efficiency, Security and Reliability. However, other dimensions were also considered (i.e., Reusability, Maintainability, Usability, Compatibility and Completeness). This outcome differed from how the researchers judged code snippet quality.Conclusion: Practitioners today mainly rely on code snippets from online code resources, and specific models or quality characteristics are emphasised based on their need to address distinct concerns (e.g., mobile vs web vs standalone applications, regular vs machine learning applications, or open vs closed source applications). Consequently, software quality models should be adapted for the domain of consideration and not seen as one-size-fits-all. This study will lead to targeted support for various clusters of the software development community.(c) 2022 Elsevier Inc. All rights reserved.
Convolutional neural networks (CNNs) performance requires tuning of network architectures, which requires machine learning knowledge and significant time and effort. Thus, modern deep CNN in soil spectroscopy faces a major barrier as a result of this process. The components in convolutional neural networks (CNNs) for spectroscopic modelling are usually set heuristically before turning the hyperparameters. This can lead to poorly designed CNN models that are not fully optimised, or to models that take a long time to train to derive the optimal model. Although recent work has shown the importance of tuning the CNN hyperparameters, few studies have sought to tune the CNN components and hyperparameters concurrently. We propose such an approach when designing CNNs for soil spectroscopy, and test its effectiveness with the LUCAS soil library and dataset from the Kellogg Soil Survey Laboratory database. This approach has two phases. First, we automated the process of building the fully connected network which involved automating the selection of the different types of layers and the number of neurons per layer. In the second stage, we automated the selection of the hyperparameters for the network layers, and we experimented with two strategies for selecting the best combination. The first strategy involved adapting the population-based training (PBT) technique by replacing the random search used in PBT with a Bayesian optimisation method, which we call adapted-PBT. In the second strategy, we employed the Bayesian optimisation method. Our study revealed that our adapted-PBT was able to achieve high model performance when compared to using Bayesian optimisation in the second stage when predicting soil texture properties for all three predictive measures used (model efficiency coefficient (MCE), Root Mean Squared Error (RMSE), and the ratio of performance to interquartile range (RPIQ)). The results from our modelling were compared with two recent CNN studies that had used the LUCAS dataset and evaluated using the MCE, RMSE and RPIQ. This comparison indicated improvements (5% to 37%) in performance for all three soil properties (sand, silt and clay) in all performance metrics. These findings provide new insights that could help advance the use of CNNs in soil spectroscopy modelling by concurrently combining the building of the CNN components with tuning the hyperparameters.
Together with many success stories, promises such as the increase in production speed and the improvement in stakeholders’ collaboration have contributed to making agile a transformation in the software industry in which many companies want to take part. However, driven either by a natural and expected evolution or by contextual factors that challenge the adoption of agile methods as prescribed by their creator(s), software processes in practice mutate into hybrids over time. Are these still agile? In this article, we investigate the question: what makes a software development method agile? We present an empirical study grounded in a large-scale international survey that aims to identify software development methods and practices that improve or tame agility. Based on 556 data points, we analyze the perceived degree of agility in the implementation of standard project disciplines and its relation to used development methods and practices. Our findings suggest that only a small number of participants operate their projects in a purely traditional or agile manner (under 15 percent). That said, most project disciplines and most practices show a clear trend towards increasing degrees of agility. Compared to the methods used to develop software, the selection of practices has a stronger effect on the degree of agility of a given discipline. Finally, there are no methods or practices that explicitly guarantee or prevent agility. We conclude that agility cannot be defined solely at the process level. Additional factors need to be taken into account when trying to implement or improve agility in a software company. Finally, we discuss the field of software process-related research in the light of our findings and present a roadmap for future research.
Intense competition in the mobile apps market means it is important to maintain high levels of app reliability to avoid losing users. Yet despite its importance, app reliability is underexplored in the research literature. To address this need, we identify, analyse, and classify the state-of-the-art in the field of mobile apps' reliability through a systematic mapping study. From the results of such a study, researchers in the field can identify pressing research gaps, and developers can gain knowledge about existing solutions, to potentially leverage them in practice. We found 87 relevant papers which were then analysed and classified based on their research focus, research type, contribution, research method, study settings, data, quality attributes and metrics used. Results indicate that there is a lack of research on understanding reliability with regard to context-awareness, self-healing, ageing and rejuvenation, and runtime event handling. These aspects have rarely been studied, or if studied, there is limited evaluation. We also identified several other research gaps including the need to conduct more research in real-world industrial projects. Furthermore, little attention has been paid towards quality standards while conducting research. Outcomes here show numerous opportunities for greater research depth and breadth on mobile app reliability.
Cross-project defect prediction (CPDP), where data from different software projects are used to predict defects, has been proposed as a way to provide data for software projects that lack historical data. Evaluations of CPDP models using the Nearest Neighbour (NN) Filter approach have shown promising results in recent studies. A key challenge with defect-prediction datasets is class imbalance, that is, highly skewed datasets where non-buggy modules dominate the buggy modules. In the past, data resampling approaches have been applied to within-projects defect prediction models to help alleviate the negative effects of class imbalance in the datasets. To address the class imbalance issue in CPDP, the authors assess the impact of data resampling approaches on CPDP models after the NN Filter is applied. The impact on prediction performance of five oversampling approaches (MAHAKIL, SMOTE, Borderline-SMOTE, Random Oversampling and ADASYN) and three undersampling approaches (Random Undersampling, Tomek Links and One-sided selection) is investigated and results are compared to approaches without data resampling. The authors examined six defect prediction models on 34 datasets extracted from the PROMISE repository. The authors' results show that there is a significant positive effect of data resampling on CPDP performance, suggesting that software quality teams and researchers should consider applying data resampling approaches for improved recall (pd) and g-measure prediction performance. However, if the goal is to improve precision and reduce false alarm (pf) then data resampling approaches should be avoided.
Data sparsity is a challenge facing most modern recommendation systems. With cross-domain recommendation technique, one can overcome data sparsity by leveraging knowledge from relevant domains. This approach can be further enhanced by considering the latent sentiment information. However, as this latent sentiment information is derived from both relevant and irrelevant sources, the performance of the recommendation system may decline. This is a negative transfer (NT) problem, wheretheknowledgethatisderivedfrommultiplesourcesaffectsthesystem. Also, thesesourcedomainsareoftenimbalanced, which could further hurt the performance of the recommendation system. To this end, recent research has shown that NT is caused by domain divergence, source and target quality, and algorithms that are not carefully designed to utilise the target data to improve the domain transferability. While various research works have been proposed to prevent NT, these address only some of the factors that may lead to NT. In this paper, we propose a more systematic and comprehensive approach to overcoming NT in sentiment analysis by tackling the main causes of NT. Our approach combines the use of cost weighting learning, uncertainty-guided (aleatoric and epistemic) loss function over the target dataset, and the concept of importance sampling, to derive a robust model. Experimental results on a sentiment analysis task using Amazon review datasets validate the superiority of our proposed method when compared to three other state-of-the-art methods. To disentangle the contributions behind the success of both uncertainties, we conduct an ablation study exploring the effect of each module in our approach. Our findings reveal that we can improve a sentiment analysis task in a transfer learning setting from 4% to 10% when combining both uncertainties. Our outcomes show the importance of considering all factors that may lead to NT. These findings can help to build an effective recommendation system when including the latent sentiment information.
Stack Overflow is used to solve programming issues during software development. Research efforts have looked to identify relevant content on this platform. In particular, researchers have proposed various modelling techniques to predict acceptable Stack Overflow answers. Less interest, however, has been dedicated to examining the performance and quality of typically used modelling methods with respect to the model and feature complexity. Such insights could be of practical significance to the many practitioners who develop models for Stack Overflow. This study examines the performance and quality of two modelling methods, of varying degree of complexity, used for predicting Java and JavaScript acceptable answers on Stack Overflow. Our dataset comprised 249,588 posts drawn from years 2014-2016. Outcomes reveal significant differences in models’ performances and quality given the type of features and complexity of models used. Researchers examining model performance and quality and feature complexity may leverage these findings in selecting suitable modelling approaches for Q&A prediction.
In traditional machine learning environments, the use of non-parametric error estimation to set the discriminative threshold of a classifier to achieve the best accuracy is very effective. This method is not effective in a transfer learning environment because it is only reliable when both the training and testing data have similar distributions which is not the case in a transfer learning setting. Although the use of control variate techniques has been proposed to exploit the information about the error in the training sample to reduce the error in the test sample, this method yields a finite variance and the model uncertainty is not distributed among the variance. In this paper, we proposed and test a new transfer learning validation method called control linear minimum mean-squared error (CLMMSE) for source model selection under homogeneous transfer learning settings with the absence of an adequate pre-trained source model. Our approach adopts the Bayesian linear minimum mean-squared error (LMMSE) and integrates the idea of importance sampling into a control variate approach to provide an accurate estimate for the LMMSE that is then used to select the optimal source model. By combining importance sampling with the control variate technique to reduce further the variance, we can achieve a much tighter bound with the LMMSE. This approach reduces the risk in the target domain under data shift. Experimental results on synthetic data under two data shift settings demonstrate the efficacy of our approach. A further experiment on two real-world datasets shows that we were able to improve the accuracy of two state-of-the-art models tested; Bert (0.94\% to 65\%) and CodeBERT (1.82\% to 18.2\%) when compared to using previous selection methods.
Over the years, there has been debate around what constitutes software quality and how it should be measured. As a result, studies are needed to catalogue a repository of views of evidence in the software quality domain. This study will be significant because developers consult online collaboration platforms such as Stack Overflow. We examine early software quality models and those commonly used to study the quality of code snippets used by practitioners seeking help online. We then investigate practitioners' views of software quality of commonly reused software snippets/smaller code fragments. We conducted two systematic literature reviews followed by semi-structured interviews of 50 practitioners to address this gap. We observed that academics' view of software quality models has shifted over time, and practitioners' judge code snippets quality along five dimensions, such as Functionality, Readability, Efficiency, Security and Reliability. However, other dimensions such as Reusability, Maintainability, Usability, Compatibility and Completeness were also considered. Earlier software quality models are not frequently referenced by today's practitioners, who mostly rely on code snippets. Consequently, software quality models should be adapted for the domain or consideration and not seen as one-size-fits-all. This study will lead to targeted support for various clusters of the software development community.
Cross-project defect prediction (CPDP) models are used in new software project prediction tasks to improve defect prediction rates. The development of these CPDP models could be challenging in cases where there is little or no historical data. For this reason, researchers may need to rely on multiple sources and use transfer learning-based CPDP for building defect prediction models. These data are typically taken from similar and related projects, but their distributions can be different from the new software project (target data). Although, transfer learning-based CPDP models are designed to handle these distribution differences, but if not correctly handled by the model, may lead to negative transfer. To this end, recent works have focused on building transfer CPDP models, but little is known about how similar or dissimilar sources should be to avoid negative transfer. This paper provides the first empirical investigation to understand the effect of combining different sources with different levels of similarities in transfer CPDP. We introduce the use of the Population Stability Index (PSI) to interpret whether the distribution of the combined or single-source data is similar to the target data. This was validated using an adversarial approach. Experimental results on three public datasets reveal that when the source and target distribution are very similar, the probability of false alarm is improved by 3% to 7% and the recall indicator is reduced from 1% to 8%. Interestingly, we also found that when dissimilar source data are combined with different source datasets, the overall domain divergence is lowered, and the performance is improved. The results highlight the importance of using the right source to aid the learning process.
We review the literature on the nature of quality assurance in the context of comnplex systems developed using iterative and incremental approaches.
In the domain of software engineering, our efforts as researchers to advise industry on which software practices might be applied most effectively are limited by our lack of evidence based information about the relationships between context and practice efficacy. In order to accumulate such evidence, a model for context is required. We are in the exploratory stage of evolving a model for context for situated software practices. In this paper, we overview the evolution of our proposed model. Our analysis has exposed a lack of clarity in the meanings of terms reported in the literature. Our base model dimensions are People, Place, Product and Process. Our contributions are a deepening of our understanding of how to scope contextual factors when considering software initiatives and the proposal of an initial theoretical construct for context. Study limitations relate to a possible subjectivity in the analysis and a restricted evaluation base. In the next stage in the research, we will collaborate with academics and practitioners to formally refine the model.
David M. Raffo合作论文数Professor of Supply & Logistics Management and Information Systems4