Software testing is essential to ensure the reliability and correctness of software systems. However, the effectiveness of testing is highly dependent on the quality of the test suites themselves. Mutation analysis, a powerful technique for evaluating the quality of tests, introduces small changes into the code and checks whether the tests detect them. Despite its strengths, mutation analysis faces challenges in scalability due to the high computational cost of compiling and running tests against mutants. This paper presents MediumDarwin, a substantially upgraded version of the original LittleDarwin, initially introduced as a research prototype. Our enhanced version retains the original foundational architecture but introduces significant new capabilities and performance optimisations that transform it into a robust platform for both industrial use and advanced research. The enhancements made to LittleDarwin include: (1) persistent storage of mutation results in a relational database to facilitate advanced analysis, (2) coverage-based test selection optimisation to minimise test executions, (3) implementation of mutant schemata to reduce compilation overhead, (4) enhanced mutation operators alongside safeguards against non-compilable mutants, and (5) dynamic subsumption graph computation for efficient mutant analysis. These innovations collectively improve the tool's scalability and practical utility in software quality assurance in both industrial and research contexts. A screencast demonstrating the use of MediumDarwin is available at https://www.youtube.com/watch?v=Zsd3pZt63AE.
Android apps are getting bloated by continuously integrating possibly unnecessary functional modules. This trend of software bloat negatively impacts the performance of static analysis tools. As a result, analysis reports are more likely to contain false positives and experience analysis timeouts. Consequently, developers are forced to manually inspect and troubleshoot errors, as well as restart the analysis process, making analyzers more time-consuming and less user-friendly. However, existing approaches for Android app debloating almost only consider how to remove redundant code elements or functional features from the perspective of users, thus they are unsuitable for the analyzer-oriented app debloating task in most cases. To fill this gap, we propose an Android app debloating approach that employs a novel Multi-layer Dependence Graph (MDG) structure to represent the app under analysis. We hierarchically construct the MDG by sequentially analyzing and capturing dependence at the class, method, and statement levels. Throughout this process, we dynamically identify hotspot classes and narrow down the scope for further dependence extraction, thereby alleviating the challenge of a too complicated graph structure caused by the excessive app size. We implement our approach as the tool FlowSlicer, a novel MDG-based static Android app debloater. We evaluate FlowSlicer by utilizing it to debloat the input app first and then observing the performance difference of two analysis processes which accept the original and the debloated app as input respectively. The evaluation is performed on both the hand-crafted and the real-world apps in our benchmark. Our results show that FlowSlicer is not only capable of effectively debloating Android apps but also enhancing the performance of static analyzers. For instance, cooperating with FlowSlicer, the analyzer FlowDroid could detect 212 more leaks in real-world apps in our benchmark.
Logs are central to understanding what is happening inside CI/CD pipelines. Today's platforms often treat logs as passive artefacts rather than active indicators of risk. This research introduces DevSecLogs, a system designed to integrate security intelligence and enforce integrity throughout the pipeline. We are utilising Natural Language Processing (NLP) methodologies such as log tokenisation, topic clustering, and semantic filtering through Latent Dirichlet Allocation (LDA), specifically the topic modelling technique. We also implemented a deep learning-based anomaly detector, particularly a hybrid CNN-LSTM model, to surface irregular log behaviours, such as skipped test phases, unauthorised access attempts, or irregular command patterns. To keep the integrity of these logs, the system stores each step in a blockchain-integrated structure that makes any changes evident and ensures a permanent audit trail. The system is currently live on IBM Cloud, with planned integration into CI/CD tools such as Jenkins and Tekton, to enable secure, real-time log monitoring within DevSecOps workflows.
Generative artificial intelligence (GenAI) has taken software engineering by storm, as large language models (LLMs) have quickly come to generate large volumes of human-like text and code. Recent work has shown that, although excellent at mimicking human writing, these models have challenging limitations when writing code. Authors have identified how the code generated can be longer, more complex, worse performing, and contain more serious vulnerabilities than that written by humans. As the use of GenAI now shifts from a bleeding edge novelty to an amenity consolidated as the new status quo for software engineering, a pressing concern arises: It is still unclear how AI-written code impacts the long-term maintenance and evolution of software codebases. This PhD project investigates the previous concern by studying how AI-written code impacts four key qualities of source code: the system's ability to fulfill functional and non-functional requirements, the architectural cohesion, and the comprehensibility of the source code. By interacting with practitioners, analyzing software repositories, and proposing new approaches to using LLMs for software engineering, we expect to develop the theory and basis necessary to balance the benefits in productivity from GenAI with the everpressing needs for long-term software maintenance and evolution.
Code review is a fundamental practice in software engineering, ensuring code quality, fostering collaboration, and reducing defects. While research has extensively examined various aspects of this process, most studies assume that all code reviews follow a standardized evaluation workflow. However, our industrial partner, which uses Merge Requests (MRs) mechanism for code review, reports that this assumption does not always hold in practice. Many MRs serve alternative purposes beyond rigorous code evaluation. These MRs often bypass the standard review process, requiring minimal oversight. We refer to these cases as deviations, as they disrupt expected workflow patterns. For example, work-in-progress (WIP) MRs may be used as draft implementations without the intention of being reviewed, MRs with huge changes are often created for code rebase, and library updates typically involve dependency version changes that require minimal or no review effort. We hypothesize that overlooking MR deviations can lead to biased analytics and reduced reliability of machine learning (ML) models used to explain the code review process. This study addresses these challenges by first identifying MR deviations. Our findings show that deviations occur in up to 37.02 % of MRs across seven distinct categories. In addition, we develop a detection approach leveraging few-shot learning, achieving up to 91 % accuracy in identifying these deviations. Furthermore, we examine the impact of removing MR deviations on ML models predicting code review completion time. Removing deviations significantly enhances model performance in 53.33 % of cases, with improvements of up to 2.25 times. Additionally, their exclusion significantly impacts model interpretation, strongly altering overall feature importance rankings in 47 % of cases and top-k rankings in 60 %. Our contributions include: (1) a clear definition and categorization of MR deviations, (2) a novel AI-based detection method leveraging few-shot learning, and (3) an empirical analysis of their exclusion impact on ML models explaining code review completion time. Our approach helps practitioners streamline review workflows, allocate reviewer effort more effectively, and ensure more reliable insights from MR analytics.
MoveMethod is a hallmark refactoring. Despite a plethora of research tools that recommend which methods to move and where, these recommendations do not align with how expert developers perform Movemethod. Given the extensive training of Large Language Models and their reliance upon naturalness of code, they should expertly recommend which methods are misplaced in a given class and which classes are better hosts. Our formative study of 2016 LLM recommendations revealed that LLMs give expert suggestions, yet they are unreliable: up to 80 % of the suggestions are hallucinations. We introduce the first LLM fully powered assistant for MoveMethod refactoring that automates its whole end-to-end lifecycle, from recommendation to execution. We designed novel solutions that automatically filter LLM hallucinations using static analysis from IDEs and a novel workflow that requires LLMs to be self-consistent, critique, and rank refactoring suggestions. As MoveMethod refactoring requires global, project-level reasoning, we solved the limited context size of LLMs by employing refactoring-aware retrieval augment generation (RAG). Our approach, MM-assist, synergistically combines the strengths of the LLM, IDE, static analysis, and semantic relevance. In our thorough, multi-methodology empirical evaluation, we compare MM-assist with the previous state-of-the-art approaches. MMASSIST significantly outperforms them: (i) on a benchmark widely used by other researchers, our Recall@1 and Recall@3 show a $1.7 x$ improvement; (ii) on a corpus of 210 recent refactorings from Open-source software, our Recall rates improve by at least $\mathbf{2. 4 x}$. Lastly, we conducted a user study with $\mathbf{3 0}$ experienced participants who used MM-ASSIST to refactor their own code for one week. They rated $\mathbf{8 2. 8 \%}$ of MM-aSSIST recommendations positively. This shows that MM-ASSIST is both effective and useful.
API documentation is often the most trusted resource for programming. Many approaches have been proposed to augment API documentation by summarizing complementary information from external resources like Stack Overflow. Existing extractive summarization approaches excel in producing faithful summaries that accurately represent the source content without input length restrictions. Nevertheless, they suffer from inherent readability limitations. On the other hand, our empirical study on the abstractive-based summarization method, i.e., GPT-4, reveals that GPT-4 can generate coherent and concise summaries but presents limitations in terms of informativeness and faithfulness. We introduce APIDOCBOOSTER, an extract-then-abstract framework that seamlessly fuses the advantages of both extractive (i.e., enabling faithful summaries without length limitation) and abstractive summarization (i.e., producing coherent and concise summaries). APIDocBooster consists of two stages: (1) Context-aware Sentence Section Classification (CSSC) and (2) UPdate SUMmarization (UPSUM). CSSC classifies APIrelevant information collected from multiple sources into API documentation sections. UPSUM generates extractive summaries distinct from original API documentation and then abstractive summaries guided by extractive summaries through in-context learning. To enable automatic evaluation, we construct the first dataset for API documentation augmentation. Our automatic evaluation results reveal that each stage in APIDocBooster outperforms its baselines by a large margin. Our human evaluation also demonstrates the superiority of APIDOCBOOSTER over GPT-4 and shows that it improves the informativeness, relevance and faithfulness by $\mathbf{1 6. 2 2 \%}, \mathbf{1 9. 4 4 \%}$, and $\mathbf{3 7. 1 4 \%}$, respectively.
Open source software (OSS) forms the backbone of industrial data workflows and enterprise systems. However, many OSS projects face operational risks due to informal or centralized governance. This paper presents a practical case study of data.table, a high-performance R package widely adopted in production analytics pipelines, which underwent a community-led governance reform to address scalability and sustainability concerns. Before the reform, data.table faced a growing backlog of unresolved issues and open pull requests, unclear contributor pathways, and bottlenecks caused by reliance on a single core maintainer. In response, the community initiated a redesign of its governance structure. In this paper, we evaluated the impact of this transition through a mixed-methods approach, combining a contributor survey (n=17) with mining project repository data. Our results show that following the reform, the project experienced a 200
Preprocessor directives such as #ifdef have historically been the predominant mechanism for managing variability in legacy $C / C++$ codebases. While they offer a simple way to include or exclude code at compilation time, they lack explicit variability representation and make it difficult to reason about features beyond conditional inclusion. Over time, these directives tend to become scattered, redundant, and obsolete resulting in tangled and rigid implementations that hinder maintenance, reuse, and long-term evolution. With the growing maturity of Software Product Line Engineering (SPLE) and the emergence of industrial tools such as Mobioos Forge (MF), it is now possible to rethink the variability management in legacy systems. MF enables the specification of variability through feature models, the building of feature-to-code mappings, and the static derivation of variants without relying on the compiler. These capabilities pave the way for modernizing configurable codebases by making variability explicit, analyzable, and maintainable. In this paper, we present an integrated toolchain for the automated recovery of Software Product Lines from legacy C/C++ systems. Our approach identifies features and their interactions, synthesizes a hierarchical feature model, and generates traceability links to locate feature-specific code fragments. Built as an extension to MF, our approach can also leverages its static derivation engine to safely remove obsolete features and generate concrete product variants. We validate our approach on large-scale, real-world open-source systems, including Chromium, which contains over $\mathbf{2 5 0 0}$ distinct features. The results demonstrate the effectiveness of our method in revealing hidden variability structures, enabling safe variant generation, and supporting the modernization of legacy configurable systems through SPLE principles.
AI-generated code has become an integral part of the mainstream developer workflow today. However, in community-driven platforms like Stack Overflow (SO), where trust, authorship, and credibility are important, it can lead to serious complications. While recent studies have focused on detecting AI-generated code, they have mostly worked with long code samples from repositories and assignments. In contrast, code snippets on SO are often small and context-specific, and thus may prove more challenging for detection. Moreover, another aspect overlooked in prior studies concerns recognizing adversarially prompted AI code deliberately crafted to resemble human-written code. To address these limitations, we have first introduced a large-scale dataset comprising 3500 pairs of SO and ChatGPT answers, along with a curated set of 4500 adversarially prompted AI responses. Next, we evaluate existing code language models over this newly curated dataset. Our evaluation shows that existing models perform well on standard AI answers but fail to detect adversarial ones. Finally, to improve detection, we propose an ensemble approach combining stylometric features of code along with the code embeddings. Our approach shows consistent improvements across multiple models and improves resistance to adversarial prompted code. Our overall findings open promising directions for future research into understanding the nuances of AI code detection with adversarial prompting and code stylometry.
Developing large-scale industrial systems requires high-quality requirements to avoid costly rework and project delays. However, linguistic ambiguities in natural language (NL) requirements have been a long-standing challenge, often introducing misinterpretations and inconsistencies that propagate throughout the development lifecycle. Such ambiguous NL requirements necessitate early detection and well-reasoned explanations to clarify and prevent further misunderstandings among stakeholders. While solutions have been developed to detect ambiguities in NL requirements, the advent of generative large language models (LLMs) offers new avenues for explanation-augmented requirements ambiguity detection. This paper empirically investigates LLMs for ambiguity detection and explanation in real-world industrial requirements by adopting an in-context learning paradigm. Our results from three industrial datasets show that LLMs achieve a 20.2% average performance increase in classifying ambiguous requirements when prompted with ten relevant in-context demonstrations (10 -shot), compared to no demonstrations (0 -shot). Additionally, we conducted human evaluations of the LLM-generated outputs with eight industry experts along four dimensions-naturalness, adequacy, usefulness and relevance-to gain practical insights. The results show an average rating of 3.84 out of 5 across evaluation criteria, indicating that the approach is effective in providing supporting explanations for requirement ambiguities.
Performance testing is crucial to ensuring that web applications meet user expectations under varying workloads. Activities such as stress, load, and smoke testing are designed to simulate different kinds of simultaneous user interactions and assess system behavior. Despite its recognized importance in quality assurance of large-scale web-based systems, witnessed by numerous studies proposing solutions to support these activities, the real-world adoption and evolutionary dynamics of performance tests have received limited attention in the literature. To fill this gap, we analyzed 77 open-source web projects using Apache JMETER and LOCUST. Our study investigates how performance tasks are performed (adoption time, load design, types of tasks), the characteristics of projects that adopt them, and their longterm maintenance. Our findings reveal that performance tests in open-source projects are simple, with a focus on singleuser behaviors and minimal requests, and most tests have low concurrency. Load tests are the most common, followed by smoke and stress tests. Projects with performance tests tend to be larger and more actively maintained. However, tests are mostly long-lived but rarely updated, suggesting potential risks to their relevance and coverage over time. Finally, by creating a taxonomy of performance test changes, we observe recurring patterns of modifications, including workload adjustments, network request changes, and updates to system monitoring.
The adoption of generative AI (GenAI) in software development practice has introduced significant changes for developers and organizations. However, for now the actual impact is still unknown and the changes to developer practices, behavior, and software quality are unclear. In my thesis, I explore this research area with the goal of contributing valuable insights for practitioners and researchers.
Large Language Models (LLMs) have transformed code assistants by enabling personalization, interactivity, and higher abstraction. However, these assistants often struggle with a common limitation; they generate responses based on a limited set of relevant code snippets retrieved from the codebase using semantic similarity search. This mechanism prevents them from viewing the code structure holistically, making it difficult to give accurate and complete answers to questions on code dependencies and structure. This paper introduces a dependency-aware code assistant that answers structural questions developers cannot easily pose to general-purpose assistants like GitHub Copilot. We achieve this by enriching the LLM with dependency facts obtained from a code graph generated by a static-analysis pipeline customized specifically for industry-scale codebases. The dependency information is queried from a Neo4j database, which stores the code graph, via Text-to-Cypher translation powered by LLMs. Cypher is a query language, designed specifically for querying graph-structured data. We evaluated our solution at Philips Healthcare. Specifically, we performed a benchmark with 420 collected questions and a user study with seven industrial software engineers. By analyzing the results, we identified common mistakes made by GPT40 in the Text-to-Cypher translation to query code graphs, including syntax, schema and semantic errors. This work lays the foundation for advancing Cypher query generation on industryscale code graphs and for augmenting graph-based code analysis with LLMs.
Behavioral bugs caused by incorrect state changes are particularly challenging to identify because they depend on specific code execution paths. While code property graph (CPG) combine multiple code views through abstract syntax trees (AST), their built-in redundancy from syntax details and fixed connection rules make them hard to scale-a major problem when analyzing large software systems. We introduce QVoG, a new framework that improves CPG by combining graphbased code analysis with state behavior checking. Our main innovation lies in simplifying the CPG at the statement level by consolidating control and data flows into meaningful code blocks and optimizing the edges. This approach reduces the graph size by more than 10 times compared to AST-based methods while maintaining accuracy. This lightweight design allows easy integration of state tracking, where we match object lifecycle rules to simplified CPG connections using replaceable patterns. The combination of streamlined graphs and state-aware analysis helps QVoG effectively find difficult-to-identify behavioral bugs, successfully detecting 25 issues (including 17 confirmed cases and 2 official CVE) in real-world projects. Importantly, QVoG analyzes raw source code without requiring compilation and supports projects exceeding 1 million lines of code.
Development teams for mobile applications can receive thousands of user reviews daily. At the same time, these developers use different communication channels, such as the GitHub issue tracker. Although GitHub issues are accessible and manageable for developers, their content often differs starkly from what users write in app reviews. Issues may lack steps to reproduce bugs or insights that justify the priority of new feature requests. The sheer volume of user reviews for a popular app, combined with their heterogeneity and varying quality, makes manual integration into issue trackers unfeasible. We present an approach that automatically augments GitHub issues with informative user reviews to bridge the gap between user feedback and developer-managed issues. Using a state-of-the-art large language model (LLM), our approach automatically retrieves user reviews with high semantic textual similarity (STS) to the issue content and suggests reviews that augment developers' understanding of the issue. In this paper, we present large-scale quantitative and qualitative analyses to assess the feasibility of enriching development workflows with user-written information. Using over 37,000 issues and 750,000 reviews from 19 popular Free/Libre/Open Source Software (FLOSS) mobile applications, our approach augments 3,017(8%) issues with 7,287 (1%) potentially informative reviews. In addition to providing insights into user-reported bugs and feature requests, the information from these matches points toward a novel and promising way to leverage user reviews for concerted app evolution.
Time Series Management System (TSMS) is a specialized database management system designed for storing, querying, and analyzing time series data. Its correctness is essential for accurate data processing. However, logic bugs can lead to erroneous query outputs, severely compromising the reliability of data analysis. Compared with traditional relational database SQL, time series SQL exhibits significant syntactic and semantic differences, making existing tools inapplicable. To the best of our knowledge, the detection of logic bugs remains an open problem. In this paper, we propose TSGuard, a tool for detecting logic bugs in TSMSs via time series algebra. The core idea of TSGuard is to convert time series SQL queries into equivalent time series algebra expressions, evaluate these expressions to derive the expected result set, and then compare it with the actual query result set to detect potential logic bugs in the TSMS. Additionally, we introduce a feedback mechanism for query generation and develop query syntax validators for different TSMSs to improve the efficiency of logic bug detection. Through extensive testing, TSGuard discovered 48 previously unknown bugs, including 45 logic bugs and 3 crash bugs.
Large Language Models (LLMs) are increasingly applied to data-centric tasks in software maintenance and evolution, such as quality assurance and migration. While recent methods constrain LLM outputs using grammars or regular expressions, these syntactic techniques fail to enforce deeper semantic constraints involving numeric dependencies, conditional logic, and checksums. We present ClauseBandit, a framework that combines LLMs with Satisfiability Modulo Theories (SMT) solvers to generate structured data satisfying such constraints from natural language specifications. ClauseBandit introduces a Bayesian inference approach that selects the most plausible SMT formula using posterior probabilities derived from formula self-consistency and data likelihoods. Evaluated on 27 structured generation tasks inspired by industrial use-cases, ClauseBandit successfully selected valid SMT formulas for $\mathbf{7 4. 1 \%}$ of tasks. Our approach enables LLM-based structured generation that goes beyond syntax, producing semantically valid, reusable constraint specifications from natural language.
Code clones are a well-known source of technical debt, often degrading software maintainability. Modern AI coding assistants can unintentionally introduce clones or even licenseincompatible code, posing maintenance and legal challenges. This research proposes a Large Language Model (LLM)-powered clone refactoring assistant that proactively detects duplicative code during development and suggests high-level refactorings. By bridging traditional clone detection with LLM-driven code generation or transformation, the approach aims to remove redundancies early while ensuring changes remain behavior preserving and legally compliant. The assistant will integrate into development workflows to prevent clone propagation, flag potential intellectual property risks, and incorporate developer feedback for explainable, trustworthy operation. We will evaluate its impact on code quality and developer productivity, and assess how AI enhancements influence long-term maintenance efforts.
Computation over sensitive data requires that the computation function is secure and trusted. Existing approaches either do not enforce formal verification, require the user to verify the proof, or lack secure attestation guarantees. In addition, neither addresses the issue of having users once again inspect the application after upgrading the code running in the enclave. We propose an approach that uses a formal specification to guarantee that the behavior of the computation function conforms to the desired functionality. By combining automated verification with attestation on a trusted execution environment, we ensure that only conformant applications are executed. At the same time, we allow updates of the computation function without changing the attestation response, as long as the formal specification still holds. We implement and evaluate the system on several functions; our results show an average overhead of only 50 %. Finally, we demonstrate the validity of the system using a real-world application, Dafny-EVM.
