Reproducing system-level concurrency bugs requires both input data and the precise interleaving order of system calls. This process is challenging because such bugs are non-deterministic, and bug reports often lack the detailed information needed. Additionally, the unstructured nature of reports written in natural language makes it difficult to extract necessary details. Existing tools are inadequate to reproduce these bugs due to their inability to manage the specific interleaving at the system call level. To address these challenges, we propose SysPro, a novel approach that automatically extracts relevant system call names from bug reports and identifies their locations in the source code. It generates input data by utilizing information retrieval, regular expression matching, and the category-partition method. This extracted input and interleaving data are then used to reproduce bugs through dynamic source code instrumentation. Our empirical study on real-world benchmarks demonstrates that SysPro is both effective and efficient at localizing and reproducing system-level concurrency bugs from bug reports.

With the growing ubiquity of multi-core architectures, concurrent systems have become essential but increasingly prone to complex issues such as data races and deadlocks. While modern issue-tracking systems facilitate the reporting of such problems, labeling concurrency-related bug reports remains a labor-intensive and error-prone task. This paper presents a linguistic-pattern-based framework for automatically identifying concurrency bug reports. We derive 58 distinct linguistic patterns from 730 manually labeled concurrency bug reports, organized across four levels: word-level (keywords), phrase-level (n-grams), sentence-level (semantic), and bug report-level (contextual). To assess their effectiveness, we evaluate four complementary approaches-matching, learning, prompt-based, and fine-tuning-spanning traditional machine learning, large language models (LLMs), and pre-trained language models (PLMs). Our comprehensive evaluation on 12 large-scale open-source projects (10,920 issue reports from GitHub and Jira) demonstrates that fine-tuning PLMs with linguistic-pattern-enriched inputs achieves the best performance, reaching a precision of 91

To govern smart contracts running on Ethereum, multiple Ethereum Request for Comment (ERC) standards have been developed, each defining a set of rules governing contract behavior. Violating these rules can cause serious security issues and financial losses, signifying the importance of verifying ERC compliance. Today's practices of such verification include manual audits, expert-developed program-analysis tools, and large language models (LLMs), all of which remain ineffective at detecting ERC rule violations. This paper introduces SymGPT, a tool that combines LLMs with symbolic execution to automatically verify smart contracts' compliance with ERC rules. We begin by empirically analyzing 132 ERC rules from three major ERC standards, examining their content, security implications, and natural language descriptions. Based on this study, SymGPT instructs an LLM to translate ERC rules into a domain-specific language, synthesizes constraints from the translated rules to model potential rule violations, and performs symbolic execution for violation detection. Our evaluation shows that SymGPT identifies 5,783 ERC rule violations in 4,000 real-world contracts, including 1,375 violations with clear attack paths for financial theft. Furthermore, SymGPT outperforms six automated techniques and a security-expert auditing service, underscoring its superiority over current smart contract analysis methods.

Software fault localization identifies the code locations responsible for reported issues and is a fundamental step toward automated debugging and program repair. Recent retrieval-based approaches formulate fault localization as a dense retrieval task by learning a shared embedding space between issue reports and source code. However, these methods encode all issue reports using a fixed query representation, despite the substantial diversity of real-world issue reports in length, structure, and debugging information. To address this limitation, we propose HyperFL, a query-adaptive representation learning framework for software fault localization. HyperFL employs a lightweight hypernetwork to generate query-specific LoRA parameters for the query encoder, enabling dynamic query adaptation while keeping the code encoder fixed and reusable. Experiments on a real-world issue localization benchmark demonstrate that HyperFL consistently improves retrieval performance across multiple embedding backbones, achieving up to 13.3

Automated bug reproduction is a challenging task, with existing tools typically relying on textual steps-to-reproduce, videos, or crash logs in bug reports as input. However, images provided in bug reports have been overlooked. To address this gap, this paper presents an empirical study investigating the necessity of including images as part of the input in automated bug reproduction. We examined the characteristics and patterns of images in bug reports, focusing on (1) the distribution and types of images (e.g., UI screenshots), (2) documentation patterns associated with images (e.g., accompanying text, annotations), and (3) the functional roles they served, particularly their contribution to reproducing bugs. Furthermore, we analyzed the impact of images on the performance of existing tools, identifying the reasons behind their influence and the ways in which they can be leveraged to improve bug reproduction. Our findings reveal several key insights that demonstrate the importance of images in supporting automated bug reproduction. Specifically, we identified six distinct functional roles that images serve in bug reports, each exhibiting unique patterns and specific contributions to the bug reproduction process. This study offers new insights into tool advancement and suggests promising directions for future research.

Bug report reproduction is a crucial but time-consuming task to be carried out during mobile app maintenance. To accelerate this process, researchers have developed automated techniques for reproducing mobile app bug reports. However, due to the lack of an effective mechanism to recognize different buggy behaviors described in the report, existing work is limited to reproducing crash bug reports, or requires developers to manually analyze execution traces to determine if a bug was successfully reproduced. To address this limitation, we introduce a novel technique to automatically identify and extract the buggy behavior from the bug report and detect it during the automated reproduction process. To accommodate various buggy behaviors of mobile app bugs, we conducted an empirical study and created a standardized representation for expressing the bug behavior identified from our study. Given a report, our approach first transforms the documented buggy behavior into this standardized representation, then matches it against real-time device and UI information during the reproduction to recognize the bug. Our empirical evaluation demonstrated that our approach achieved over 90% precision and recall in generating the standardized representation of buggy behaviors. It correctly identified bugs in 83% of the bug reports and enhanced existing reproduction techniques, allowing them to reproduce four times more bug reports.

Recently, dynamic concurrency bug predictions have kept making notable progress in improving concurrency coverage while ensuring soundness. Most of them rely solely on dynamic information in traces and overlook the static semantics of the program when predicting bugs. To ensure soundness, they assume that any (memory) read can fully affect subsequent program execution via control-flow and data-flow. However, the assumption over-approximates constraints among (memory) writes and reads and hence limits reordering space over thread interleaving, ultimately leading to false negatives. From program semantics, only a subset of reads actually affect their subsequent executions. Therefore, by refining dependencies between reads and subsequent executions based on static program semantics, one can refine the assumption and eliminate unnecessary constraints. This can bring a chance to explore more thread interleaving space and uncover more concurrency bugs. However, refining dependencies can compromise soundness and bring heavy overhead. To tackle these challenges, this paper introduces the concept of Necessary Consistent Read Event (NRE) and a hybrid analysis algorithm. NRE refines dependencies between reads and their subsequent events and is used to identify necessary constraints where a read probably affects the execution of its subsequent events. Next, we design an efficient and accurate hybrid analysis algorithm to calculate NREs for each event in the trace. The hybrid analysis algorithm maps events to program SSA instructions and simulates executions based on the original trace. NRE and the algorithm can enhance the capabilities of existing concurrency bug prediction methods at a low cost, regardless of the type of concurrency bug they target. In this paper, we focused on data race and developed NRE and the algorithm as a prototype tool RECONP. We conducted a set of comparative experiments on MySQL with M2 and Seqcheck. The results show that RECONP can detect 46.9% and 22.4% more data races than M2 and Seqcheck, respectively. And the hybrid algorithm only accounts for 34% of the total time cost.

The AAA (Arrange, Act, Assert) pattern provides a unified structure for unit test cases, potentially benefiting comprehension and maintenance. However, its adoption and implementation in practice remain insufficiently understood. This study investigates the prevalence of AAA pattern usage, identifies recurring deviations and design issues within AAA structures, and assesses developers’ receptiveness to AAA-based improvements.We conducted an empirical study on 735 real-life unit test cases randomly selected from seven open-source projects. We manually analyzed these test cases, identified AAA-related issues, and proposed fixes to developers. Our analysis found that 77% of test cases follow the AAA structure. We identified three recurring patterns deviating from AAA and four design issues within A blocks. Comparison with classic test smells revealed unique insights provided by AAA analysis. Of 27 improvement proposals sent to developers, 78% received positive feedback. These findings show that the AAA pattern is widely adopted in practice, but deviations from and design issues within AAA patterns are common. Our analysis provides a novel perspective on test case quality, complementing traditional test smell analysis. The high acceptance rate of our improvement proposals suggests that developers value AAA-based enhancements. These findings can guide the development of tools for improving AAA practice in unit tests.

Coverage-guided Greybox Fuzzing (CGF) is one of the most effective dynamic software testing techniques, which focus on improving the code coverage. The methodology automatically generates new offspring test cases by mutating existing test cases and analyzing program execution, preserving the interesting test cases as seeds for subsequent mutations. However, existing CGF tools often neglect the similarity between seeds. The mutation of similar seeds can yield a multitude of similar offspring test cases, subsequently executing similar code segments of the program under test. This challenge hinders the improvement of code coverage, consequently impacting the efficiency of fuzz testing.To address this issue, this paper proposes a fuzz testing method BaSFuzz based on difference analysis for seed bytes. The method leverages both byte similarity and structure similarity to analyze the differences between seed bytes. Subsequently, it computes a similarity score for each seed and reorders the seed queue in ascending order of similarity scores.Based on this method, a prototype tool is developed and compared with AFL, AFLFast, MOpt, AFL++-Hier and HTFuzz on 12 target programs. The experimental results indicate that BaSFuzz achieved 190.14%, 143.9%, 10.93%, 374.85% and 11.79% more edge coverage compared to the five tools, respectively. Additionally, BaSFuzz triggered unique crashes 3.57 times, 1.46 times, 42.38%, 2.85 times, and 33.44% more than the five tools, respectively.

The large demand of mobile devices creates significant concerns about the quality of mobile applications (apps). Developers need to guarantee the quality of mobile apps before it is released to the market. There have been many approaches using different strategies to test the GUI of mobile apps. However, they still need improvement due to their limited effectiveness. In this article, we propose DinoDroid, an approach based on deep Q-networks to automate testing of Android apps. DinoDroid learns a behavior model from a set of existing apps and the learned model can be used to explore and generate tests for new apps. DinoDroid is able to capture the fine-grained details of GUI events (e.g., the content of GUI widgets) and use them as features that are fed into deep neural network, which acts as the agent to guide app exploration. DinoDroid automatically adapts the learned model during the exploration without the need of any modeling strategies or pre-defined rules. We conduct experiments on 64 open-source Android apps. The results showed that DinoDroid outperforms existing Android testing tools in terms of code coverage and bug detection.

Automatic Speech Recognition (ASR) systems have become essential in modern life, powering human-computer interactions and voice-activated virtual assistants. These systems, which predominantly rely on deep neural networks, exhibit a probabilistic nature that can lead to unpredictable and erroneous outputs, particularly in varied and dynamic real-world settings. Consequently, it is crucial to thoroughly test ASR systems to identify and resolve bugs that may impair functionality and user experience. Current works for ASR testing fall short in diversity and coverage. To address this, we propose a novel coverage-guided test method for ASR utilizing four neuron coverage criteria to guide the generation of diverse test data and ensure comprehensive testing across multiple dimensions. We evaluated the effectiveness of our method using three popular datasets and two DNN-based ASR systems. The results show that the generated data significantly improve coverage rates, enabling more comprehensive testing.

The AAA pattern, i.e. arrange, act, and assert, provides a unified structure for unit test cases, which benefits comprehension and maintenance. However, there is little understanding regarding whether and how common real-life developers structure unit test cases following AAA in practice. In particular, are there recurring anti-patterns that deviate from the AAA structure and merit refactoring? And, if test cases follow the AAA structure, could they contain design flaws in the A blocks? If we propose refactoring to fix the design of test cases following the AAA, how do developers receive the proposals? Do they favor refactoring? If not, what are their considerations? This study presents an empirical study on 435 real-life unit test cases randomly selected from four open-source projects. Overall, the majority (71.5 AAA structure. And, we observed three recurring anti-patterns that deviate from the AAA structure, as well as four design flaws that may reside inside of the A blocks. Each issue type has its drawbacks and merits corresponding refactoring resolutions. We sent a total of 18 refactoring proposals as issue tickets for fixing these problems. We received 78 refactoring. From the rejections, we learned that return-on-investment is a key consideration for developers. The findings provide insights for practitioners to structure unit test cases with AAA in mind, and for researchers to develop related techniques for enforcing AAA in test cases.

Information Retrieval-based Fault Localization (IRFL) techniques aim to identify source files containing the root causes of reported failures. While existing techniques excel in ranking source files, challenges persist in bug report analysis and query construction, leading to potential information loss. Leveraging large language models like GPT-4, this paper enhances IRFL by categorizing bug reports based on programming entities, stack traces, and natural language text. Tailored query strategies, the initial step in our approach (LLmiRQ), are applied to each category. To address inaccuracies in queries, we introduce a user and conversational-based query reformulation approach, termed LLmiRQ+. Additionally, to further enhance query utilization, we implement a learning-to-rank model that leverages key features such as class name match score and call graph score. This approach significantly improves the relevance and accuracy of queries. Evaluation on 46 projects with 6,340 bug reports yields an MRR of 0.6770 and MAP of 0.5118, surpassing seven state-of-the-art IRFL techniques, showcasing superior performance.

MOTIVATION:The rapid expansion of Bioinformatics research has led to a proliferation of computational tools for scientific analysis pipelines. However, constructing these pipelines is a demanding task, requiring extensive domain knowledge and careful consideration. As the Bioinformatics landscape evolves, researchers, both novice and expert, may feel overwhelmed in unfamiliar fields, potentially leading to the selection of unsuitable tools during workflow development. RESULTS:In this article, we introduce the Bioinformatics Tool Recommendation system (BTR), a deep learning model designed to recommend suitable tools for a given workflow-in-progress. BTR leverages recent advances in graph neural network technology, representing the workflow as a graph to capture essential context. Natural language processing techniques enhance tool recommendations by analyzing associated tool descriptions. Experiments demonstrate that BTR outperforms the existing Galaxy tool recommendation system, showcasing its potential to streamline scientific workflow construction. AVAILABILITY AND IMPLEMENTATION:The Python source code is available at https://github.com/ryangreenj/bioinformatics_tool_recommendation.

Continuous integration (CI) has become a popular method for automating code changes, testing, and software project delivery. However, sufficient testing prior to code submission is crucial to prevent build breaks. Additionally, testing must provide developers with quick feedback on code changes, which requires fast testing times. While regression test selection (RTS) has been studied to improve the cost-effectiveness of regression testing for lower-level tests (i.e., unit tests), it has not been applied to the testing of user interfaces (UI) in application domains such as mobile apps. UI testing at the UI level requires different techniques such as impact analysis and automated test execution. In this paper, we examine the use of RTS in CI settings for UI testing across various open-source mobile apps. Our analysis focuses on using Frequency Analysis to understand the need for RTS, Cost Analysis to evaluate the cost of impact analysis and test case selection algorithms, and Test Reuse Analysis to determine the reusability of UI test sequences for automation. The insights from this study will guide practitioners and researchers in developing advanced RTS techniques that can be adapted to CI environments for mobile apps.

SummaryContext: Software performance is crucial for ensuring the quality of software products. As one of the non‐functional requirements, the few efforts devoted to software performance have often been neglected until a later phase in the software development life cycle (SDLC). The lack of clarity of what software performance research literature is available prevents researchers from understanding what software performance research fields are available. It also creates difficulty for practitioners to adopt state‐of‐the‐art software performance techniques. Software performance research is not as organized as other established research topics such as software testing. Thus, it is essential to conduct a systematic mapping study as a first step to provide an overview of the latest research literature available in software performance. Objective: The objective of this systematic mapping study is to survey and map software performance research literature into suitable categories and to synthesize the literature data for future access and reference. Method: This systematic mapping study conducts a manual examination by querying research literature in noble journals and proceedings in software engineering in the past decade. We examine each paper manually and identify primary studies for further analysis and synthesis according to the pre‐defined inclusion criteria. Lastly, we map the primary studies based on their corresponding classification category. Results: This systematic mapping study provides a state‐of‐the‐art literature mapping in software performance research. We have carefully examined 222 primary studies out of 2000+ research literature. We have identified six software performance research categories and 15 subcategories. We generate the primary study mapping and report five research findings. Conclusions: Unlike established research fields, it is unclear what types of software performance research categories are available to the community. This work takes the systematic mapping study approach to survey and map the latest software performance research literature. The study results provide an overview of the paper distribution and a reference for researchers to navigate research literature on software performance.

Unit testing focuses on verifying the functions of individual units of a software system. It is challenging due to the high inter dependencies among software units. Developers address this by mocking—replacing the dependency by a “fake” object. Despite the existence of powerful, dedicated mocking frameworks, developers often turn to a “hand-rolled” approach—inheritance. That is, they create a subclass of the dependent class and mock its behavior through method overriding. However, this requires tedious implementation and compromises the design quality of unit tests. This work contributes a fully automated refactoring framework to identify and replace the usage of inheritance by using Mockito—a well received mocking framework. Our approach is built upon the empirical experience from five open source projects that use inheritance for mocking. We evaluate our approach on nine other projects. Results show that our framework is efficient, generally applicable to new datasets, mostly preserves test case behaviors in detecting defects (in the form of mutants), and decouples test code from production code. The qualitative evaluation by experienced developers suggests that the auto-refactoring solutions generated by our framework improve the quality of the unit test cases in various aspects, such as making test conditions more explicit, as well as improved cohesion, readability, understandability, and maintainability with test cases. Finally, we submit 23 pull requests containing our refactoring solutions to the open source projects. It turns our that, 9 requests are accepted/merged, 6 requests are rejected, the remaining requests are pending (5 requests), with unexpected exceptions (2 requests), or undecided (1 request). In particular, among the 21 open source developers that are involved in the reviewing process, 81% give positive votes. This indicates that our refactoring solutions are quite well received by the open source projects and developers.

The AAA pattern (i.e., Arrange-Act-Assert ) is a common and natural layout to create a test case. Following this pattern in test cases may benefit comprehension, debugging, and maintenance. The AAA structure of real-life test cases, however, may not be clear due to their high complexity. Manually labeling AAA statements in test cases is tedious. Thus, we envision that an automated approach for labeling AAA statements in existing test cases could benefit new developers and projects that practice collective code ownership and test-driven development. This paper contributes an automatic approach based on machine learning models. The “secret sauce” of this approach is a set of three learning features that are based on the semantic, syntax, and context information in test cases, derived from the manual tagging process. Thus, our approach mimics how developers may manually tag the AAA pattern of a test case. We assess the precision, recall, and F-1 score of our approach based on 449 test cases, containing about 16,612 statements, across 4 Apache open source projects. To achieve the best performance in our approach, we explore the usage of six machine learning models; the contribution of the SMOTE data balancing technique; the comparison of the three learning features; and the comparison of five different methods for calculating the semantic feature. The results show our approach is able to identify Arrangement , Action , and Assertion statements with a precision upwards of 92%, and recall up to 74%. We also summarize some experience based on our experiments—regarding the choice of machine learning models, data balancing algorithm, and feature engineering methods—which could potentially provide some reference to related future research.

As part of the process of resolving issues submitted by users via bug reports, Android developers attempt to reproduce and observe the crashes described by the bug reports. Due to the low-quality of bug reports and the complexity of modern apps, the reproduction process is non-trivial and time-consuming. Therefore, automatic approaches that can help reproduce Android bug reports are in great need. However, current approaches to help developers automatically reproduce bug reports are only able to handle limited forms of natural language text and struggle to successfully reproduce crashes for which the initial bug report had missing or imprecise steps. In this paper, we introduce a new fully automated approach to reproduce crashes from Android bug reports that addresses these limitations. Our approach accomplishes this by leveraging natural language processing techniques to more holistically and accurately analyze the natural language in Android bug reports and designing new techniques, based on reinforcement learning, to guide the search for successful reproducing steps. We conducted an empirical evaluation of our approach on 77 real world bug reports. Our approach achieved 67% precision and 77% recall in accurately extracting reproduction steps from bug reports, reproduced 74% of the total bug reports, and reproduced 64% of the bug reports that contained missing steps, significantly outperforming state of the art techniques.