Synthetic oversampling (OVS) is a common approach for addressing class imbalance in software defect prediction (SDP). While foundational SDP studies established rigorous evaluation standards, recent evaluations frequently rely on narrow experimental setups and restricted subsets of OVS techniques. Given the rapid introduction of OVS technique variants, it remains unclear whether the longstanding OVS baselines in SDP are still adequate or require revision. To address this gap, this study benchmarks 89 OVS techniques across eight learning algorithms and 25 imbalanced defect datasets, establishing an experimental space sufficient to determine algorithm-specific responses to these diverse OVS techniques. The evaluation pipeline incorporates Bayesian hyperparameter optimization, 20 independent repetitions, four standard performance metrics, and the Brunner’s test with Cliff’s d effect sizes. Results demonstrate that OVS improves the performance of tree and tree-based ensemble algorithms but offers limited or no gains for distance-based, kernel-based, and neural network algorithms. Furthermore, the conventional probability of detection and false alarm (pd−pf) Pareto analysis is augmented with the Matthews correlation coefficient (MCC) as a balanced recovery criterion. This augmentation identifies a candidate set of 52 learning algorithm × OVS technique configurations, organized into four operational zones based on explicit pf tolerance. The findings confirm that the current OVS baselines in SDP require revision, with standard SMOTE and widely adopted SDP-specific OVS techniques absent from the recommended configurations capable of maintaining operational false-alarm control.
Specifying and maintaining high-quality user stories is challenging because project contexts evolve, natural-language artifacts are ambiguous, and manual assessment requires substantial effort. The INVEST criteria provide a structured framework for evaluating whether user stories are Independent, Negotiable, Valuable, Estimable, Small, and Testable. This study examines run-to-run variability and model–expert alignment through a two-phase design. The first phase comprised a dependent replication in which 60 original Spanish-language user stories were evaluated by three experts, GPT-5.1 Thinking (ChatGPT interface), and GPT-5.1 (API-based). The second phase extended the design through structured refinement of the original stories followed by reevaluation of the refined versions under the same Likert (1–5) scale and INVEST rubric. Results were criterion- and execution-condition-specific. Evaluator-related differences in the initial evaluation were detected in a limited subset of comparisons. Across all criterion–condition combinations, paired analyses generally indicated smaller absolute expert–model discrepancies after refinement and reevaluation, although magnitudes were heterogeneous and zero changes were frequent. In the initial evaluation, model ratings more often fell below the expert median when ratings differed; this pattern persisted less uniformly after refinement. Agreement for the average of the three expert ratings was high under ICC(2,3), while ordinal agreement varied by criterion in one refined condition. Overall, the findings are consistent with a potential complementary role for large language models in INVEST-based evaluation and structured refinement, but they do not establish causal improvement, model superiority, or equivalence with expert judgment. However, human oversight remains essential for contextual validation, interpretation, and backlog prioritization.
Efficient debugging is essential for software maintenance, as identifying, reproducing, and correcting bugs are critical to ensuring software reliability. However, debugging remains a time-consuming and cost-sensitive activity due to the complexities involved in analyzing and resolving bugs throughout the software development lifecycle. Recent advancements in Artificial Intelligence, Machine Learning, and particularly Natural Language Processing (NLP), and Large Language Models (LLM) offer promising opportunities to enhance debugging process. Since bug reports are typically written in natural language, NLP techniques can streamline debugging tasks, such as bug categorization, localization, and resolution. To assess the potential of NLP in debugging, we conduct a Systematic Literature Review (SLR) of 87 research papers published between 2011 and 2025. Our study presents a comprehensive taxonomy of research efforts, evaluates the effectiveness and limitations of various NLP methods, and highlights best practices across key software debugging tasks, including bug reproduction, bug localization, and bug report analysis. Furthermore, we identify significant challenges in real-world applications, including issues with scalability, accuracy, and adaptability to diverse software environments. Finally, we derive seven findings from our result analysis and propose future research directions corresponding to the findings, to advance NLP-driven debugging practices. This study provides a holistic overview of NLP in debugging for researchers, practitioners, and tool developers, helping to uncover trends, address gaps, and inspire new approaches to improving software maintenance workflows.
Security patch identification is an essential task in software maintenance, which aims to identify vulnerability-related patches from open-source software (OSS) repositories. Previous works tend to adopt pre-trained models to improve the model performance. These models are often built on billions of parameters and require sufficient computing resources and enough data for supervised fine-tuning, to be applied on downstream classification tasks. However, existing studies mainly address class imbalance through re-sampling or re-weighting strategies. Re-sampling may introduce synthetic noise or disrupt the original distribution of commit data, while fixed re-weighting schemes lack dynamic awareness of the model's learning status, leading to over-smoothing, unstable optimization, and underfitting. Moreover, increasingly large pre-trained models require substantial computational resources for training and deployment, limiting their practical use in low-resource scenarios. In this work, we propose to boost security patch identification with cumulative learning and knowledge distillation, which aims to address security patch identification in low-resource scenarios while maintaining high-quality identification performance. To alleviate the data imbalance problem, inspired by cumulative learning, we designed a novel adaptive smoothing loss, which dynamically controls the smoothing coefficient a through the training epoch. In the early stage, a higher smoothing coefficient is settled to balance the influence of the majority class, while it is gradually reduced to ensure that the predictions of the minority classes are closely consistent with their true labels in the later stages. To bridge the gap between high-quality identification performance and low computational resources, based on the teacher-student network theory, we adopted the Kullback-Leibler divergence to minimize the difference between the teacher's distribution and the student's. The teacher model is further used to generate pseudo-labels for unlabeled data, thereby enhancing the performance of the student model. Experiment results on two open-source datasets demonstrate that our proposed approach not only improves the best baselines by 14.14% and 8.81% on minority class, but also reduces 68% model size. The code and data are shared in https://github.com/AppleMax1992/low-resource-commit-classification.
