Open-source artificial intelligence and machine learning (AI/ML) repositories hosted on GitHub have become central infrastructures for software innovation, experimentation, reuse, and community signaling. Yet repository visibility does not necessarily imply sustainable maintenance conditions. This article examines how popularity, maintenance burden, ownership structure, documentation-support signals, and LLM/GenAI orientation are associated in highly starred GitHub repositories represented in a public repository-metadata dataset.The empirical analysis uses Version 1 of the Kaggle dataset “Top GitHub Repositories (2026)”, which contains GitHub repository-level metadata across 15 software domains. The original file contained 3,000 repository-domain records; after deduplication based on full repository names, the final analytical sample comprised 2,562 unique repositories. The dataset is not a probability sample of all AI/ML repositories; rather, it represents highly starred GitHub repositories across software and AI/ML-related domains. Stars and forks were used as indicators of visibility and engagement, while open issues were treated as an observable proxy for unresolved maintenance burden. The analysis combined descriptive statistics, nonparametric group comparisons, Spearman correlations, domain-level comparisons, Negative Binomial generalized linear models, log-transformed OLS robustness checks, multicollinearity diagnostics, and stratified/interaction analyses.The results indicate that ownership structure is the clearest structural predictor of maintenance burden. Organization-owned repositories accumulate substantially more open issues than user-owned repositories, and this association remains after controlling for repository age, update recency, size, domain, primary language, documentation-support signals, and LLM/GenAI orientation. By contrast, the raw visibility advantage of organization-owned repositories weakens in multivariate models. LLM/GenAI repositories display a distinct attention profile, achieving higher visibility despite being markedly younger. GitHub-native documentation-support signals show a negative association with popularity, a finding that is interpreted cautiously as a platform-specific signal that may reflect documentation externalization rather than documentation quality.These findings show that popularity and maintenance burden do not evolve symmetrically in highly starred GitHub repositories across software and AI/ML-related domains. Governance form, scale, and platform-specific coordination affordances jointly shape how repository visibility is converted into unresolved maintenance load.
Context: Unified Modeling Language (UML) is widely used in software engineering education to teach system analysis and design. However, manually translating textual software specifications into consistent UML diagrams is challenging for learners due to ambiguity, inconsistent interpretation, and the lack of cross-diagram alignment. Existing NLP- and LLM-based approaches generate diagrams but often lack a unified intermediate representation to ensure structural consistency and explainability. Objective: This study aims to develop a unified graph-based intermediate representation, called UMLGraphIR, designed to provide interpretable feedback for learners through graph-based structural analysis. The goal is to enable reliable transformation of textual specifications into consistent UML diagrams through a structured workflow while supporting explainable evaluation and validation. Methods: The proposed framework follows a multi-stage transformation pipeline: (1) extracting actors, actions, and relationships using an LLM-based component identification process; (2) representing these elements using a structured JSON schema; (3) converting the JSON into a weighted adjacency matrix encoding UML relationships such as association, include, and extend; and (4) generating a canonical graph using Graphviz and PlantUML for visualization and validation. A Library Management System case study demonstrates the process, and evaluation is performed through graph-based structural comparison with instructor-defined reference diagrams. Results: The framework successfully generated use case, class, and sequence diagrams with improved structural alignment and semantic accuracy, achieving a mean structural accuracy of 92.58% across 20 software specifications compared to 68.62% for student-generated baselines. The unified graph-based representation enhanced cross-diagram consistency, enabled quantifiable structural validation through node convergence (94.1%), edge convergence (90.9%), and relationship type accuracy (92.3%), and reduced modeling errors commonly found in student-generated diagrams. A pilot observational study further showed that students who revised their diagrams using UMLGraphIR-generated feedback improved their mean accuracy to 88.50%. The UMLGraphIR Streamlit prototype demonstrated the practical feasibility of graph-based UML generation, validation, and feedback delivery, providing interpretable feedback through graph-level identification of missing elements, relationship mismatches, and structural inconsistencies. Conclusion: UMLGraphIR offers a consistent, explainable, and scalable approach for transforming textual specifications into UML diagrams. By establishing a unified graph-based representation, the framework improves diagram quality, supports cross-diagram coherence, and enhances educational feedback mechanisms. The approach provides a foundation for future extensions such as multimodal inputs, additional UML diagram types, and broader integration into model-driven engineering workflows.
Context: Equipping Self-Adaptive Systems (SASs) with AI-driven decision-making enables them to be more efficient especially in dynamic environments by analyzing data and predicting suitable adaptation strategies. However, SASs remain vulnerable to uncertainty, which arises when AI models encounter unforeseen situations or environmental changes not represented during the training phase. Such uncertainty can lead to erroneous predictions, misadaptation, and degraded service quality in highly dynamic or heterogeneous settings. Traditional mitigation strategies, such as periodic retraining or continuous learning, are often time-consuming, costly, and reactive, exposing the system to mispredictions until adaptation is completed. Objective: enhance SASs robustness and reliability in uncertain and unpredictable environments through the design of FALCON; which extends the standard architecture of SASs with AI-based decision-making and a formal model to analyze the quality of the adopted adaptation strategies under uncertainty. Method: Falcon introduces an additional MAPE-K loop that continuously monitors the deployed AI models performance, tracking metrics such as prediction confidence and response time. Falcon addresses uncertainty by dynamically switching between specialized, fine-tuned models, using transfer learning to deploy a model trained in a similar environment whenever the current model shows low confidence. This allows the system quickly adaptation to changing conditions without requiring full retraining, maintaining high prediction accuracy and reliable adaptation when encountering unforeseen scenarios. To ensure that model selection is rigorous and avoid mispredictions, the additional MAPE-K loop incorporates a formal decision-making process based on Satisfiability Modulo Theories (SMT), which analyzes the deployed models and selects the most suitable replacement according to environmental constraints. Results: An experimental evaluation on the crop rotation problem demonstrates that Falcon achieves high prediction accuracy and reliability, with the majority of predictions exceeding 80% confidence. Conclusion: By integrating runtime model monitoring, adaptive model switching, and formal verification, Falcon provides SASs with proactive decision-making to maintain high-quality adaptation under uncertainty.
Context: The rapid evolution of Generative AI is transforming software engineering, enabling Large Language Models (LLMs) to generate code from natural-language prompts. This capability is increasingly applied to security-critical programming tasks, such as penetration testing, forensic analysis, traffic analysis, and OSINT. However, it remains unclear which prompt engineering strategies security analysts should adopt to obtain reliable and correct code in these scenarios. Objectives: This paper aims to systematically evaluate the impact of different prompting strategies on the quality of LLM-generated code for security-oriented tasks, assessing both functional correctness and robustness. Methods: We present a systematic evaluation of 12 prompting strategies across 4 state-of-the-art LLMs. Our evaluation framework combines textual similarity metrics with execution-based analysis conducted in a Docker-based environment, measuring correctness against ground truth on valid inputs and robustness on edge and invalid inputs. Results: Reasoning-oriented strategies, such as Chain-of-Thought and Few-Shots Prompting, consistently improve the generated code, achieving up to ＋24.6% improvement in ROUGE-L, average correctness rates of 63%–64%, and robustness rates of 50%. In contrast, refinement-based strategies such as Iterative Prompting and Question Refinement degrade structural similarity by up to -50% in BLEU-4, reducing correctness and robustness to 33% and 26%, respectively. Models producing more functionally correct code also exhibit broader behavioral coverage, reaching up to 78.24% average statement coverage. Furthermore, textual similarity alone proves insufficient to predict functional behavior. Conclusion: These findings highlight that prompt design substantially influences the reliability of LLM-generated code in security-critical scenarios, and that execution-based evaluation is essential to complement textual metrics. Reasoning-oriented prompting strategies are recommended for security analysts working with LLM-based code generation tools.
Context: Database evolution and management are often complex and error-prone processes, largely driven by the need to adapt to changing requirements. Objective: This study aimed to collect information on current research in database design and evolution, examine the approaches used, categorize the environments that existing solutions target, along with their inputs, and evaluate whether the existing methods are reversible. In addition, the study aimed to develop a taxonomy for systematically classifying and comparing the identified approaches. Method: The systematic literature review was conducted following established guidelines, employing both backward and forward snowballing techniques. The literature search was performed across four databases. After filtering and snowballing, 76 relevant papers published between 2020 and 2025 were identified, with one additional key study included manually, resulting in a final set of 77 papers. The studies were grouped into classification units and classified using an inductively developed taxonomy covering problem scope, target environment, solution input, workload dependency, transformation strategy, and reversibility support. Results: We categorized existing research using a six-dimensional taxonomy, enabling a structured comparison of the existing approaches. Most solutions primarily focused on NoSQL databases. Conceptual models were predominantly used as input. We compared various approaches identified in the literature and highlighted their lack of reversibility. The approaches differed significantly between solutions for relational databases. In NoSQL databases, model-driven and workload-driven strategies were prevalent. In multi-model database solutions, concepts from category theory were widely employed, and several approaches also addressed polystore systems. Conclusion: Our findings indicate a significant gap in the recent literature regarding the reversibility of evolution as a means to enhance testability and robustness. In terms of input specifications, although the use of textual instructions is increasing, there is a notable lack of solutions based on textual requirements. In contrast, solutions that employ conceptual models are much more common.
Context: API vulnerability detection in deep learning (DL) frameworks is hindered by two major challenges: insufficient test coverage and low-quality test cases. These issues arise from inefficient exploration of the high-dimensional and sparsely populated input space. Existing fuzzing techniques often fail to improve coverage for critical code modules. Furthermore, they lack effective strategies for generating diverse, high-impact test inputs, which results in stagnated coverage and missed vulnerabilities. Objectives: This paper aims to enhance the efficiency and effectiveness of fuzz testing in DL frameworks by addressing critical module under-coverage and improving the quality of test case generation through dynamic, directed, and adaptive strategies. Methods: We propose D3Fuzz, a coverage-guided, large language model(LLM)-driven fuzzing framework that integrates three core techniques: (1) a hierarchical multi-task prompting mechanism powered by LLM to generate semantically rich test inputs guided by real-time coverage feedback; (2) a coverage-driven decaying factor prioritization model that dynamically adjusts test input weights to focus on under-tested critical modules; and (3) a stochastic perturbation mechanism combined with dynamic decay factors to escape local coverage plateaus and encourage exploration. Results: Experimental evaluations on two mainstream DL frameworks, PyTorch and JAX, demonstrate the effectiveness of D3Fuzz. On PyTorch, it achieves a 31% increase in C++ coverage, a 61.9% improvement in Python coverage, and a 5.36× increase in vulnerability detection efficiency over the baseline tool, ∇Fuzz. On JAX, it yields a 57.7% improvement in C++ coverage, a 61.7% enhancement in Python coverage, and a 1.34× improvement in vulnerability detection efficiency. Conclusion: D3Fuzz demonstrates that combining LLM-based semantic understanding with dynamic coverage feedback can substantially improve both the breadth and depth of fuzzing in DL frameworks. It offers a promising direction for future research in automated vulnerability detection.
Context: The wide adoption of Large Language Models (LLMs) has raised growing concerns about their trustworthiness, resulting in the emergence of numerous benchmark datasets. However, the trustworthiness evaluation of LLMs remains challenging due to two issues: (1) the large volume of existing datasets leads to high evaluation costs, and (2) the unintended use of benchmark datasets in LLM training causes inflated performance scores. Objective: To address these two challenges, this study designs a general dataset refinement approach aimed at improving the efficiency and reliability of trustworthiness evaluation for LLMs. Methods: We propose a dataset refinement approach, TrustDA, comprising two components: (1) data distillation, and (2) data augmentation. The distillation component adopts a three-stage filtering process to distill a subset of samples that both discriminate model performance and ensure coverage of diverse risk types. The augmentation component applies perturbation rules on the distilled dataset to generate augmented samples that retain the original semantics and evaluation objective. Results: Experimental results show that TrustDA reduces the dataset size to 10% of the original while maintaining comparable evaluation performance. Furthermore, the augmented datasets generated by TrustDA result in a 12.98% reduction in the average trustworthiness scores on six datasets, mitigating the performance inflation caused by data contamination. Conclusions: This study designs and empirically validates a general dataset refinement approach TrustDA, which effectively enhances the efficiency and reliability of LLM trustworthiness evaluation in three key dimensions. In future work, we plan to extend TrustDA to cover a broader range of trustworthiness dimensions.
Context: Security is a major software development concern. Security vulnerabilities in Open-Source Software (OSS) can have severe consequences. OSS project participants widely employ peer code review to increase code quality and prevent security flaws. However, even with mandatory code reviews, many OSS projects still contain security flaws. The presence of these flaws highlights the importance of evaluating the effectiveness of code reviews in addressing security issues. Objectives: This Systematic Literature Review analyzes how peer code review helps identify OSS security vulnerabilities. The study synthesizes existing evidence on vulnerability types identified or missed, current security practices, tool supports, reviewer challenges, and factors enhancing detection. Method: We conducted a search across five major digital libraries. We screened 1067 studies and selected 20 primary studies using predefined inclusion and exclusion criteria. We analyzed the data using qualitative analysis and categorized vulnerabilities using the Common Weakness Enumeration (CWE) framework. Results: Peer code reviews effectively detect low-hanging, localized, well-known vulnerabilities (e.g., buffer overruns, race conditions, crashes, XSS) but tend to miss subtle, complex, context-dependent issues (e.g., input validation weaknesses, access control weaknesses, resource lifecycle weaknesses). Effective security practices are tied to reviewer expertise, familiarity with the codebase, frequency of small code changes, and organizational support. Security analysis tools, particularly Static Application Security Testing (SAST) tools, reduce reviewer workload and improve detection, but cannot replace human judgment. Reviewers still suffer from constraints of security knowledge and experience, time, and comprehending complex systems. Reviewer effectiveness improves with security training, domain experience, contextual familiarity, formalized review processes, and favorable review environments. Conclusion: Peer code review contributes to OSS security, but is not sufficient alone. Strengthening the effectiveness of peer code review requires integrating structured processes and checklists, security training, organizational incentives, and lightweight automated tool support to help reviewers detect subtle, complex vulnerabilities, and reinforcing secure development practices.
Context: Critical Software-Intensive Systems (CSIS) underpin essential infrastructures, yet compliance audits and actionable risk prediction remain disconnected. Audit data are high-dimensional and low-sample-size (HDLSS); metric descriptions are discarded by numerical aggregation, and rule-based schemes (AHP, weighted sum) suffer poor cross-team reproducibility. Objectives: We propose a framework combining (i) data-driven objectivity, (ii) semantic use of metric descriptions without end-to-end fine-tuning, and (iii) stability under HDLSS, while preserving metric-level representational traceability. Methods: The Semantically Aligned Gaussian Process (SAGP) framework first builds a 196-metric catalogue (135 vulnerability + 61 resilience); a frozen Qwen3-0.6B encodes each description into a semantic basis matrix M∈R196×1024. Standardised features x∈R196 are multiplied by M and reduced via Principal Component Analysis (PCA) to 20 dimensions, on which Gaussian Process Regression (GPR) with an isotropic RBF kernel predicts risk. All preprocessing is fitted strictly within training folds of 10-fold cross-validation; significance is reported jointly by Friedman, Nadeau–Bengio corrected t-test, Holm-corrected Wilcoxon, and Cohen’s dz. Results: On 400 real-world CSIS, SAGP achieves the best observed performance among 19 methods (R2=0.955, MSE =26.0, MAE =3.09), reducing MSE by 54% and MAE by 36% over GPR on raw features (ΔR2=+0.054; Nadeau–Bengio p=0.008, dz=1.57, large effect). Ablations suggest LLM semantic alignment is the main contributor to the gain (the largest single ablation effect, though the shuffled-basis comparison is not statistically significant in every setting); non-semantic transformations (Gaussian, TF–IDF/SVD bases) fail to reproduce it. Conclusion: Frozen-LLM semantic projection combined with GPR offers a viable path to accurate HDLSS risk prediction with metric-level traceability; the gain originates from inter-metric semantic proximity preserved by the LLM, not from its spectral structure.
Context: The integration of Generative AI — and in particular Large Language Models (LLMs) — into software engineering is reshaping software testing by enabling the automatic generation of test artifacts, supporting test maintenance, and accelerating quality assurance activities. However, the existing literature remains fragmented and often focuses on effectiveness metrics (e.g., coverage and pass rates) while providing limited insight into how LLMs are architecturally embedded into end-to-end testing workflows. Objective: This study aims to consolidate and structure the current research on GenAI-driven testing systems by characterizing (i) the testing activities and levels they support, (ii)the architectural patterns, pipeline organizations, and integration strategies they adopt,and (iii) how these systems are evaluated in terms of effectiveness, reliability, and efficiency. Method: We conducted a systematic literature review of 37 peer-reviewed primary studies published between 2023 and October 2025, following Kitchenham’s guidelines. Using a rigorous search, screening, and data-extraction protocol, we synthesized the selected corpus along the three complementary dimensions above. Results: The reviewed studies report recurring benefits, notably reduced manual effort and increased automation across several testing tasks. At the same time, we identify persistent limitations, including sensitivity to prompt formulation, insufficient semantic grounding, reproducibility constraints (e.g., dependence on proprietary APIs), and heterogeneous evaluation practices with limited standardization of metrics and benchmarks. Conclusion: Overall, effective GenAI-driven testing requires not only powerful LLMs but also well-engineered architectural integration and reliable evaluation protocols. We therefore outline key open challenges and research directions toward more dependable, reproducible, and systematically assessable GenAI-driven testing systems.
Context: Automated Program Repair (APR) based on Large Language Models (LLMs) has made rapid progress in recent years. Existing methods are no longer limited to direct patch generation; many of them introduce reasoning traces, feedback, or contextual information before repair. However, such information is often collected in a coarse-grained and weakly targeted manner, making it difficult for the model to distinguish domain constraints, intended program behavior, and the root cause of the defect. As a result, their repair performance remains limited when handling complex bugs. Objective: Human programmers usually do not fix bugs by directly editing code. Instead, they often follow a diagnostic process: understanding the relevant domain context, clarifying the intended functionality of the code, and then analyzing why the current implementation violates the expected behavior. This process fits the nature of program repair, because a correct patch should satisfy domain constraints, preserve intended functionality, and address the root cause of the failure. Inspired by this observation, we propose CAMP, a Cognition-Aligned Multi-stage APR framework, which organizes pre-repair reasoning according to this diagnostic process. We also investigate whether such structured reasoning can generalize to different programming languages and recent real-world bugs. Methods: CAMP consists of two stages: Chain-of-Thought (CoT) collection and bug repair. In the CoT collection stage, CAMP explicitly decomposes pre-repair reasoning into three progressive steps: domain knowledge extraction, code functionality analysis, and bug cause analysis. The refined reasoning information is then integrated into the repair prompt to guide patch generation. We compare CAMP with 10 state-of-the-art APR methods on Defects4J V1.2 and V2.0. To evaluate generalization and practical applicability, we further test CAMP on DebugBench, a multilingual dataset covering C++, Java, and Python, and on RecentAPR-98, a newly constructed dataset of recent real-world Java bugs. Results: CAMP repairs 120 bugs on Defects4J V1.2 and 109 bugs on Defects4J V2.0, achieving performance comparable to or better than existing state-of-the-art baselines. On DebugBench, CAMP shows strong repair capability across C++, Java, and Python, indicating good cross-language generalization. On RecentAPR-98, under the same experimental setting, CAMP repairs 8 more bugs than the previous state-of-the-art method. In terms of cost, CAMP consumes about 53,615 tokens per bug on average, corresponding to approximately $0.0095, reducing the cost by at least 93.2% compared with mainstream LLM-based repair methods. Conclusion: These results show that CAMP’s effectiveness does not come from simply increasing the number of reasoning steps, but from organizing repair-relevant information before patch generation. By separating domain knowledge, functionality understanding, and bug cause analysis, CAMP provides the model with more targeted repair evidence, thereby improving repair effectiveness, generalization, and cost efficiency.
Context: Software testing is an often used method to ensure software quality, with developers spending a significant amount of time on it. However, software testing education is underrepresented in curricula, particularly with regard to Conceptual Modelling (CM), where model-level validation is important. As a consequence, graduate students are insufficiently prepared for industry-level testing. Objective: We want to understand students’ behavioural patterns during model testing and their relationship to testing effectiveness. Methods: We use process mining to analyse 554 interaction logs from students using a conceptual model simulation tool. Results: We identify three behavioural profiles: the coverage chaser (using feedback iteratively), the problem solver (balancing feedback with requirements focus), and the disconnected tester (avoiding feedback-driven iteration). High-coverage students tended to integrate feedback into test loops; low-coverage students did not. Notably, high and average-coverage students achieved similar, generally high grades, a pattern consistent with a performance plateau where systematic testing is associated with sufficient educational performance without requiring maximum coverage. Conclusions: Our findings suggest that systematic use of coverage feedback is associated with more effective testing outcomes, suggesting that coverage-feedback use may be a promising focus for future instructional support in CM education.
Context: With Android development increasingly dependent on third-party SDKs, sneaky SDKs pose long-term privacy threats. They deliberately disguise themselves as legitimate software while secretly harvesting user data. However, when faced with diverse and sneaky malicious behaviors, existing Android malicious software detection approaches suffer drawbacks, including the disadvantages of software labeling errors and low detection accuracy. Objective: In this paper, we propose ASD-SDK, an Android privacy-leaking malware detection approach that includes label correction. Methods: Specifically, we use contrastive learning to measure the information distance between different categories and design a drift information feature to represent the relationship between features and labels, providing support for detection learning. To effectively learn malware information across multiple views, we design a multi-channel model that ensures the effective use of information from each feature. We also design a custom weighted cross-entropy loss function that dynamically adjusts weights to focus on important categories. Results: We conduct real-world experiments to prove the effectiveness of ASD-SDK. The results show that the accuracy can achieve almost 97.30%, and its kappa coefficient is about 96.40%. Conclusion: These results indicate that ASD-SDK is more accurate and effective than previous studies. This detector corrects labels through behavioral information analysis and enhances its ability to detect malicious software with sneaky third-party SDKs.
Context: The many applications and use cases that GenAI has, or might have in the future, can contribute to a wide array of benefits across many fields, including those requiring creativity and critical thinking. However, GenAI tools are considered general-purpose AI models with systemic risk, and some of their use cases are reported to have a potentially negative impact on human behaviour. Objectives: The goal of this paper is to understand how reliance, and specially unknowing over-reliance, on GenAI might limit human creativity and critical thinking in different fields, and to explore what mitigation strategies can be proposed across the European Union (EU). Methods: Thus, this paper presents a bibliometric analysis of the white and grey literature aimed to quantify the research distribution across various fields on GenAI and its impact on creativity and critical thinking. A posterior analysis of the risks associated with using GenAI in different field is then used to connect said risks to specific articles and recitals in the Regulation (EU) 2024/1689 of the European Parliament and of the Council, also known as the Artificial Intelligence Act. Results: The results show that there are gaps yet to be covered by existing laws, regulations, commitments, and guidelines, either because specific use cases are not considered, because exceptions exist or the scope is vaguely formulated, or because the appropriate articles have not yet entered into force. Conclusion: Taken together, the results show the great impact that GenAI might have on different fields and highlight gaps that future regulations should address to mitigate them.
Context: Deep Neural Networks (DNNs) have been widely applied in safety-critical domains. Under limited testing budgets, test case prioritization aims to execute high-value test samples as early as possible to improve fault detection efficiency. Methods based on dynamic learning trajectories provide richer behavioral information for prioritization. However, directly using full-cycle training snapshots incurs high computational and storage costs, and the trajectories often contain many redundant snapshots. Meanwhile, existing studies still insufficiently characterize dynamic trajectory features and fail to capture critical behavioral changes and discriminative information across training stages, limiting the effectiveness of dynamic information for test case prioritization. Objective: This paper aims to design a scalable DNN test case prioritization approach that effectively exploits dynamic learning behaviors while reducing computational overhead. Method: We propose DLTR, a DNN test case prioritization method based on dynamic learning trajectory pruning and feature enhancement. DLTR identifies a training stabilization point from loss trends and adaptively prunes training snapshots by retaining denser snapshots in early unstable stages and sparser ones in later stable stages. Based on the pruned trajectories, DLTR jointly extracts global and local temporal features to characterize prediction instability and phase-wise behavioral variations. A ranking model is trained on a validation set with defect-related relative supervision and then used to prioritize test cases without accessing test labels. Result: Experiments on four public datasets, including SMS, IMDB, CIFAR-10, and Tiny-ImageNet, show that DLTR reduces training snapshots by 79%–85% and achieves end-to-end prioritization speedups of 3.30× to 7.06×. Under constrained budgets, DLTR improves fault detection performance by up to 74%, and under full budgets, the cumulative improvement reaches 88%. Conclusion: DLTR effectively balances prioritization effectiveness and computational efficiency, providing a practical and scalable solution for large-scale DNN testing.
