Large language models (LLMs) have made significant progress in software engineering (SWE) tasks, such as code generation and automatic repair. However, current SWE agents have limited effectiveness when handling complex software defects and often overlook potentially useful information in failed patches. To address this, this study proposes an automatic program repair (APR) framework called DeIMerge, which is based on multi-agent collaboration and intelligent patch merging. After voting to select the optimal patch, the framework uses an LLM to analyse all failed patches deeply, fuse scattered repair clues, and generate high-quality merged patches. Experimental results show that this method increases the single-patch repair rate of open-source SWE agents from 27.3
Software architecture is the abstraction of a software system, that significantly influences software development and maintenance. As software evolves, continuous changes could deviate its architecture from the original design, leading to architecture degradation that causes a decline in software quality. Architecture refactoring becomes necessary to address or mitigate architecture degradation for improving overall quality. Although researchers have developed various architecture refactoring tools and techniques, there has been limited research on how architecture refactoring is practiced in real-world scenarios. In this paper, we conducted an empirical study by analyzing posts from Stack Overflow to understand architecture refactoring in practice. Through our analysis of 694 posts with 3,468 discussion threads, we identified 12 types of architecture refactoring based on two classification dimensions. Additionally, we categorized architecture problems faced by practitioners and explored their corresponding refactoring solutions. Furthermore, we revealed six potential risks that may result from architecture refactoring. We believe that our study can provide valuable insights for practitioners to perform architecture refactoring effectively. The findings can serve as a foundation for future research and offer practical guidance to improve architecture quality.
Context: Industrial Internet of Things (IIoT) has become a prominent topic recently, with an increasing number of IIoT open-source software (OSS) projects emerging, also within the Eclipse Foundation. Code cloning is a common practice that can adversely affect software maintenance. In the IIoT OSS domain, developers frequently reuse code and configurations for efficiency, which can lead to code clone proliferation and maintenance challenges. However, the extent and effects of code clones in the IIoT OSS domain remain understudied.Objective: This study aims to investigate the prevalence, evolution, and co-modification of code clones within the Eclipse IIoT OSS ecosystem.Methods: We collected 90 release versions from 15 projects in the Eclipse IIoT OSS ecosystem, and investigated their code clone situations based on source code and change history using the NiCad tool and our custom analysis module. The investigation covered clone distribution, patterns, evolution trends, co-modified clones, and cross-project clones.Results: 1) Code clones are prevalent in Eclipse IIoT OSS projects, with 16.3% of code lines involved in clones - nearly twice the proportion observed in traditional OSS projects; 2) Most code clones occur between commits, while there are still a significant proportion of code clones that each clone pair happens within a commit; 3) Most Eclipse IIoT projects remain stable in clone numbers during version iterations; 4) An average of 0.17% of the clones have been co-modified, which negatively affect maintenance; and 5) Cross-project clone pairs are prevalent, more in Java than in C projects, with rare co-modifications (0.02%) only in Java projects.Conclusions: We analyzed the distribution characteristics of code clones in Eclipse IIoT OSS projects, examining their specific situations, causes, and development trends. Our findings highlight the potential negative impacts of these clones on software maintenance, emphasizing the need to address these issues to improve overall software quality.
Vulnerabilities in Android applications pose significant risks to user data and system security. In practice, vulnerabilities are usually fixed through security patches submitted as code commits in software repositories. However, understanding security patches from commit histories remains challenging, since the security intent of a commit is not always explicitly reflected in its textual description.In this paper, we formulate security patch analysis as a fine-grained multi-class classification problem and propose a CodeBERT-based approach for six-class security patch classification in Android applications. The proposed method leverages pretrained code representations to capture semantic patterns from patches, where each patch is modeled as the combination of added and deleted code segments in the corresponding diff. We construct a manually annotated dataset of security-related patches collected from open-source Android projects and categorize them into six common security types: Network, Auth, Storage, Crypto, Application, and Code.Experimental results show that the proposed method achieves an accuracy of 0.72, a macro-F1 score of 0.72, and a macroAUC of 0.94. These results demonstrate that pretrained code representations are effective for capturing semantic differences in code changes and outperform traditional machine learning baselines in fine-grained security patch classification.
Context: In recent years, microservice architectures have attracted much attention as an approach to developing scalable and maintainable software systems. However, there is no systematic research on cross-service defects in microservice projects. Objective: To fill this gap, we conducted an empirical study of microservice projects from GitHub. This study aims to understand the types of cross-service defects and their involved fixing strategies in microservice projects through a comprehensive commit analysis. Method: Starting with an initial set of 3551 microservice-related repositories, we rigorously filtered and verified the projects to identify 78 projects that fully implemented microservice architectures. We extracted and analyzed 176 commits from these projects to gain insights into their cross-service defect patterns and fixing strategies. Results: Through analysis, we identified 10 types of cross-service defects, including service configuration defects, service build and dependency defects, service functionality defects, service communication defects, and service deployment defects. Based on specific commits, we summarized 13 fixing strategies, such as dependency, configuration, and method modifications, to examine the strategies utilized by microservice architectures to fix cross-service defects. Conclusion: We believe our study could contribute to the understanding of microservice development and offer a foundation for future research in this rapidly evolving field.
With the rapid development of software engineering technologies, multilingual programming, i.e., developing a program using multiple programming languages (PLs) collaboratively, has become increasingly popular. This practice is often adopted to reuse existing code, to leverage advantages of specific PLs, to satisfy various software quality needs, and to enhance software development efficiency. However, it also introduces cross-language bugs (CLBs), bugs caused by interactions between multiple PLs. Most existing bug detection tools and techniques are primarily designed for a single PL and are less effective at detecting CLBs. Given the significant advancements of Pre-trained Language Models (PLMs) in the code domain, this work investigates the performance of pre-trained Code Language Models (CodeLMs) in CLB prediction tasks. To this end, we designed and developed a cross-language code identification tool (CLCFinder), and used it to collect and construct a CLB dataset, which involves combinations of three PLs (i.e., Python-C/C++, Java-C/C++, and Python-Java) with nine types of interaction mechanisms. We fine-tuned 13 popular CodeLMs in different sizes on this dataset, in which 80% of the data was used for fine-tuning, 10% for validation, and 10% for testing. We then evaluated their performance in CLB detection. Additionally, we compared the fine-tuning performance of five top-performing models on current state-of-the-art (SOTA) single-language bug datasets to assess their application in CLB detection. Furthermore, we evaluated factors affecting the performance of fine-tuned CodeLMs in CLB detection, including the size of the fine-tuning dataset and the length of token sequence. Finally, we explored the impact of code comments on the performance of fine-tuned CodeLMs. The experimental results are that: First, all 13 CodeLMs exhibited varying degrees of performance improvement after fine-tuning. In particular, after fine-tuning, the UniXcoder-base model achieved the best performance, with an F1 score of 0.7407. Notably, within the scope of our experimental setup, small CodeLMs tended to achieve better performance than large CodeLMs, while large CodeLMs showed limited improvement. Second, the fine-tuned CodeLMs on single-language bug datasets performed poorly in CLB detection, further demonstrating the significant feature differences between CLBs and single-language bugs. Third, enlarging the fine-tuning dataset size significantly enhanced the model performance in CLB detection, but increasing token sequence length did not necessarily improve the model performance. Finally, regarding the impact of code comments, the performance varied significantly across different CodeLMs. Specifically, some CodeLMs’ performance was improved when fine-tuned with code comments, while others showed degraded performance.
The rapid expansion of the model context protocol (MCP) ecosystem enables large language model (LLM)-based agents to access a wide range of external tools via a standardized interface. However, identifying appropriate MCP servers for a specific development task remains challenging. Existing studies primarily focus on measuring the MCP ecosystem or optimizing tool invocation mechanisms, while systematic recommendation frameworks and reproducible benchmarks for real-world development tasks remain largely unexplored. To address this limitation, we formulate task-oriented MCP server recommendation as a structured retrieval-and-ranking problem that jointly considers semantic relevance and engineering constraints. We first construct Task2MCP, a task-centered dataset that systematically associates taxonomy-grounded development tasks with curated MCP servers. This dataset provides structured supervision and a reproducible evaluation environment for research on MCP tool recommendations. Building on this dataset, we propose T2MRec, a task-to-MCP server recommendation model. It models semantic relevance and structural compatibility to construct an initial candidate set. Then it improves coverage and ranking quality through centroid-based candidate expansion and constrained LLM-based re-ranking. In addition, we design and implement an interactive MCP server recommendation agent prototype that operates in conversational environments to support dynamic decision-making. The agent assists developers in efficiently evaluating and integrating tools by providing recommended MCP servers together with usage guidelines.
History coupling reflects how files change together during software evolution. Numerous studies have used history coupling information to identify bug-prone files. Meanwhile, various studies have captured the bug-prone files by examining architectural problems in them. However, there exists little work investigating the architectural connections embedded in historically coupled files. In this paper, we propose a suite of architecture hotspots that represents both history coupling information and architecture characteristics. We introduce the concept of history dependency between files, and formally define each architecture hotspot. Given the history dependencies, our approach will automatically identify architecture hotspots in a project. Through our evaluation of ten long-lived projects, we have presented that files involved in history-based architecture hotspots significantly contribute to a project’s bug-proneness. In addition, most of hotspots are long-lasting and their involved files continuously accumulate bug-proneness over software evolution. Moreover, we explored the impact significance of each hotspot type or their combinations on files’ bug-proneness. Finally, we conducted comparisons with the state of the art, and showed that our approach could capture a project’s bug-proneness more effectively. We believe that our proposed history-based architecture hotspots deserve special attention for maintenance and refactoring.
AI programming has become a popular topic in recent years. Code suggestion, with code suggestion being a key capability of AI programming. Copilot, an “AI programmer” that provides code suggestions from natural language descriptions, has been launched by GitHub and OpenAI. By far, Copilot has been widely used by millions of developers. However, little work has systematically evaluated the correctness of Copilot's suggestions. We conducted an empirical study on all 2,033 LeetCode problems to assess Copilot's code generation across four mainstream languages: C, Java, JavaScript, and Python. We have found that: 1) 70.0% of problems received at least one correct suggestion, with language-specific rates of 29.7% (C), 57.7% (Java), 54.1% (JavaScript), and 41.0% (Python); 2) Correctness decreases as problem difficulty increases, with acceptance rates of 89.3% (Easy), 72.1% (Medium), and 43.4% (Hard); 3) Acceptance rates vary across problem domains from 49.5% to 90.1%, while Graph problems challenge C and Python most, and Prefix Sum and Heap challenge Java and JavaScript most; 4) For the incorrect suggestions, we further summarize 17 types of error reasons accounting for their incorrectness and analyzed possible causes for why these errors occur. We believe our study can provide valuable insights into Copilot's capabilities and limitations.
Multilingual programming, which involves using multiple programming languages (PLs) in a single project, is increasingly common due to its benefits. However, it introduces cross-language bugs (CLBs), which arise from interactions between different PLs and are difficult to detect by single-language bug detection tools. This paper investigates the potential of pre-trained code language models (CodeLMs) in CLB detection. We developed CLCFinder, a cross-language code identification tool, and constructed a CLB dataset involving three PL combinations (Python-C/C++, Java-C/C++, and Python-Java) with nine interaction types. We fine-tuned 13 CodeLMs on this dataset and evaluated their performance, analyzing the effects of dataset size, token sequence length, and code comments. Results show that all CodeLMs performed poorly before fine-tuning, but exhibited varying degrees of performance improvement after fine-tuning, with UniXcoder-base achieving the best F1 score (0.7407). Notably, small fine-tuned CodeLMs tended to performe better than large ones. CodeLMs fine-tuned on single-language bug datasets performed poorly on CLB detection, demonstrating the distinction between CLBs and single-language bugs. Additionally, increasing the fine-tuning dataset size significantly improved performance, while longer token sequences did not necessarily improve the model performance. The impact of code comments varied across models. Some fine-tuned CodeLMs' performance was improved, while others showed degraded performance.
With the rapid advancement of automated driving technology, numerous manufacturers deploy vehicles with auto-driving features. This highlights the importance of ensuring the quality of automated driving software. To achieve this, characterizing bugs in automated driving software is important, as it can facilitate bug detection and bug fixes, thereby ensuring software quality. Automated driving software typically has a modular architecture, where software is divided into multiple modules, each designed for its own functionality for automated driving. This may lead to varying bug characteristics. Additionally, our recent study has shown a correlation between bugs caused by code clones and the functionalities of modules in automated driving software. Hence, we consider the modular structure when analyzing bug characteristics. In this paper, we analyze 3,078 bugs from two representative open-source Level-4 automated driving systems, Apollo and Autoware. By analyzing the bug report description, title, and developers’ discussions, we have identified 20 bug symptoms and 17 bug-fixing strategies, and analyzed their relationships with the respective modules. Our analysis achieves 12 main findings offering a comprehensive view of bug characteristics in automated driving software. We believe our findings can help developers better understand and manage bugs in automated driving software, thereby improving software quality and reliability.
Context: Nowadays, most deep learning frameworks (DLFs) use multilingual programming of Python and C/C++, facilitating the flexibility and performance of the DLF. However, inappropriate inter-language interaction may introduce design smells involving multiple programming languages (PLs), i.e., Inter-Language Design Smells (ILDS). Despite the negative impact of ILDS on multi-language DLFs, there is a lack of an automated approach for detecting ILDS in multi-language DLFs and a comprehensive understanding on ILDS in such DLFs. Objective: This work aims to automatically detect ILDS in multi-language DLFs written in the combination of Python and C/C++, and to obtain a comprehensive understanding on such ILDS in DLFs. Methods: We first developed an approach to automatically detecting ILDS in the multi-language DLFs written in the combination of Python and C/C++, including a number of ILDS and their detection rules defined based on inter-language communication mechanisms and code analysis. Then, we developed the CPSMELL tool that implements detection rules for automatically detecting such ILDS, and manually validated the accuracy of the tool. Finally, we performed an empirical study to evaluate the ILDS in multi-language DLFs. Results: We proposed seven ILDS and achieved an accuracy of 98.17% in the manual validation of CPSMELL in 5 popular multi-language DLFs. The study results revealed that among the 5 DLFs, TensorFlow, PyTorch, and PaddlePaddle exhibit relatively high prevalence of ILDS; each smelly file contains around 5 ILDS instances on average, with ILDS Long Lambda Function For Inter-language Binding and Unused Native Entity being relatively prominent; throughout the evolution process of the 5 DLFs, some ILDS were resolved to a certain extent, but the overall count of ILDS instances shows an upward trend. Conclusions: The automated detection of the proposed ILDS achieved a high accuracy, and the empirical study provides a comprehensive understanding on ILDS in the multi-language DLFs.
Bug fixing is a critical activity in the software development process. In issue tracking systems such as JIRA, each bug report is assigned a priority level to indicate the urgency and importance level of the bug. The priority may change during the bug fixing process, indicating that the urgency and importance level of the bug will change with the bug fixing. However, manually evaluating priority changes for bugs is a tedious process that heavily relies on the subjective judgment of developers and project managers, leading to incorrect priority changes and thus hindering timely bug fixes. Given the lack of research on bug priority change prediction, we propose a novel two-phase bug report priority change prediction method based on bug fixing evolution features and class imbalance handling strategy. Specifically, we divided the bug lifecycle into two phases: bug reporting and bug fixing, and constructed bug priority change prediction models for each phase. To evaluate the performance of our method, we conducted experiments on a bug dataset constructed from 32 non-trivial Apache projects. The experimental results show that our proposed bug fixing evolution features and the adopted class imbalance handling strategy can effectively improve the performance of prediction models. The F1-score of the prediction model constructed for the bug reporting phase reached 0.798, while the F1-weighted and F1-macro of the prediction model constructed for the bug fixing phase were 0.712 and 0.613, respectively. Furthermore, we explored the cross-project applicability of our prediction models and their performance at different priority levels. The findings indicate large variations in model performance across different projects, although the overall scores remain decent. Meanwhile, the predictive performance across various priority levels remained relatively consistently high.
In recent years, autonomous driving has drawn extensive attention. Software plays a crucial role in autonomous driving tasks including perception, planning, control, etc. As autonomous driving software systems often operate in safety-critical environments, their defects may cause catastrophic consequences. Therefore, predicting defective files in autonomous driving software before deployment is important to help developers accurately locate and address potential issues. Although prior studies have presented code clones are prone to cause defects, the potential of code clones as predictors of software defects remains unexplored, particularly in the context of autonomous driving software. In this paper, we propose a suite of code clone-based metrics, and employ them to develop defect prediction models for autonomous driving software. Through an empirical study on two representative L4 autonomous driving systems, Apollo and Autoware, we demonstrate that models relying solely on our clone-based metrics can effectively predict defective files with accuracy rates of 80–84
During software evolution, commits with various purposes, such as bug fixes, feature additions, improvements, etc., are continuously applied to software systems. This could drift software architecture from its planned design, and even cause architectural decay, that negatively affects software maintenance. Although prior studies have presented that even daily code commits could induce architectural changes, and the commit-level analysis has been widely used for multiple software comprehension and maintenance tasks, there is little work analyzing the architectural changes at the commit level. To bridge this gap, we conduct a study investigating the relationships between commits and architectural changes. Through our evaluation of thirty projects, we have shown that the architecture remains stable after most of the commits. However, there still exists a large portion of commits (27% of all studied commits) that have induced architectural changes, which deserve more attention. This further suggests the importance of analyzing architectural changes at the commit level. Meanwhile, we present a suite of commit-level metrics strongly correlated with architectural changes. Finally, we propose prediction models that can effectively forecast how much of the architecture would be changed after a commit.
Context: The advent of Autonomous Driving Systems (ADS) has marked a significant shift towards intelligent transportation, with implications for public safety and traffic efficiency. While these systems integrate a variety of technologies and offer numerous benefits, their security is paramount, as vulnerabilities can have severe consequences for safety and trust. Objective: This study aims to systematically investigate potential security weaknesses in the codebases of prominent open-source ADS projects using CodeQL, a static code analysis tool. The goal is to identify common vulnerabilities, their distribution and persistence across versions to enhance the security of ADS. Methods: We selected three representative open-source ADS projects, Autoware, AirSim, and Apollo, based on their high GitHub star counts and Level 4 autonomous driving capabilities. Using CodeQL, we analyzed multiple versions of these projects to identify vulnerabilities, focusing on CWE categories such as CWE-190 (Integer Overflow or Wraparound) and CWE-20 (Improper Input Validation). We also tracked the lifecycle of these vulnerabilities across software versions. This approach allows us to systematically analyze vulnerabilities in projects, which has not been extensively explored in previous ADS research. Results: Our analysis revealed that specific CWE categories, particularly CWE-190 (59.6%) and CWE-20 (16.1%), were prevalent across the selected ADS projects. These vulnerabilities often persisted for over six months, spanning multiple version iterations. The empirical assessment showed a direct link between the severity of these vulnerabilities and their tangible effects on ADS performance. Conclusions: These security issues among ADS still remain to be resolved. Our findings highlight the need for integrating static code analysis into ADS development to detect and mitigate common vulnerabilities. Meanwhile, proactive protection strategies, such as regular update of third-party libraries, are essential to improve ADS security. And regulatory bodies can play a crucial role in promoting the use of static code analysis tools and setting industry security standards.
In recent years, the development of Solidity smart contracts has been increasing rapidly in popularity. Code cloning is a common coding practice, and many prior studies have revealed that code clones could negatively impact software maintenance and quality. However, there is little work systematically analyzing the nature and impacts of code clones in solidity smart contracts. To bridge this gap, we investigate the prevalence, evolution, and bug-proneness of code clones in solidity smart contracts, and further identify the possible reasons for these clones' occurrences. With our evaluation of 26,294 smart contracts with 97,877 functions, we have found that code clones are highly prevalent in smart contracts. Additionally, on average, 32.01% of clones co-evolve, indicating the need for careful management to avoid consistency issues. Surprisingly, unlike in traditional software development, code clones in smart contracts are rarely involved in bug fixes. Finally, we identify three main factors that affect the occurrences of clones. We believe our study can provide valuable insights for developers to understand and manage code clones in solidity smart contracts.
Software defects are often expensive to fix, especially when they are identified late in development. Packages encapsulate logical functionality and are often developed by particular teams. Package-level defect prediction provides insights into defective designs or implementations in a system early. However, there is little work studying how to build prediction models at the package level. In this paper, we develop prediction models by using seven machine-learning algorithms and code metrics. After evaluating our approach on 20 open-source projects, we have presented that we can build effective models for predicting defective packages by using an appropriate set of metrics. However, there is no single set of metrics that can be generalized across all projects. Our study demonstrates the potential for machine-learning models to enable effective package-level defect prediction. This can guide testing and quality assurance to efficiently locate and fix defects.
In issue tracking systems, each bug is assigned a priority level (e.g., Blocker, Critical, Major, Minor, or Trivial in JIRA from highest to lowest), which indicates the urgency level of the bug. In this sense, understanding bug priority changes helps to arrange the work schedule of participants reasonably, and facilitates a better analysis and resolution of bugs. According to the data extracted from JIRA deployed by Apache, a proportion of bugs in each project underwent priority changes after such bugs were reported, which brings uncertainty to the bug fixing process. However, there is a lack of in-depth investigation on the phenomenon of bug priority changes, which may negatively impact the bug fixing process. Thus, we conducted a quantitative empirical study on bugs with priority changes through analyzing 32 non-trivial Apache open source software projects. The results show that: (1) 8.3% of the bugs in the selected projects underwent priority changes; (2) the median priority change time interval is merely a few days for most (28 out of 32) projects, and half (50. 7%) of bug priority changes occurred before bugs were handled; (3) for all selected projects, 87.9% of the bugs with priority changes underwent only one priority change, most priority changes tend to shift the priority to its adjacent priority, and a higher priority has a greater probability to undergo priority change; (4) bugs that require bug-fixing changes of higher complexity or that have more comments are likely to undergo priority changes; and (5) priorities of bugs reported or allocated by a few specific participants are more likely to be modified, and maximally only one participant in each project tends to modify priorities.