The Linux kernel, a cornerstone of modern computing and Free Software, forms the foundation of the GNU/Linux ecosystem and critical services like the Internet. Its development model integrates diverse workflows that extend beyond technical concerns. Despite its success, the project relies on key individuals and ad-hoc tools, raising sustainability and workforce renewal concerns. Moreover, known challenges within its development model remain underexplored in academic research. This paper presents ongoing efforts to develop taxonomies that describe these workflows, aiming to identify bottlenecks and guide the creation of Free Software tools to support the kernel long-term sustainability.
Analyzing software requirements is a complex task, particularly for projects with a large volume of requirements, and when conducted manually, this task is time-consuming and prone to human errors. Moreover, once the software implements the requirements, it is essential to conduct tests to ensure the correct validation. Within a software institute, each new requirement can be assigned to a test team (teams 1 and 2) responsible for ensuring coverage by updating or creating test cases. There are instances in which a requirement is not assigned to either the team or is assigned to both. Each test team is tasked with validating a specific scope of requirements, making it crucial that each requirement is analyzed and validated by an appropriate test team. If a test team fails to validate a requirement within its scope, it can result in software vulnerability. To mitigate these issues, this paper described the use of pre-trained models, such as BERT, XLNet, and ELECTRA, to automate the process of requirement classification, thereby determining which test team should validate each new requirement. We compared the models based on accuracy, precision, recall, F1-Score, and Area Under Curve (AUC) macro metrics. Notably, the XLNet model demonstrated superior performance among the models, achieving 93.16% AUC Macro, while the BERT model achieved 91.28% AUC, and the ELECTRA model achieved 90.17% AUC.We also applied the non-parametric Friedman test to statistically validate the results, followed by the Conover squared rank test, with a significance level of 0.05. The results indicate that XLNet outperformed the BERT and ELECTRA models, exhibiting a superior capacity for assigning requirements to the correct test teams. Given the promising results of this research, this study aims to demonstrate the viability of using pre-trained models as a solution to optimize the testing process in the software industry.
An effective classification of security-related software requirements is crucial to mitigate potential threats and ensure robust system design. This study investigates the performance of Large Language Models (LLMs) in classifying security-related requirements compared to traditional Machine Learning (ML) methods. Using the SecReq, DOSSPRE and PROMISE+ datasets, we evaluated ten LLMs across various prompt engineering strategies. The results demonstrate that LLMs achieve high accuracy and outperform traditional ML-based models in several evaluation scenarios and that prompt engineering can significantly enhance the model’s ability to identify security-related requirements. This work underscores the domaingeneralization capabilities of LLMs and their potential to streamline requirements classification without the complexity of feature engineering or dataset-specific fine-tuning often required by ML-based approaches. Researchers, practitioners, and tool developers can leverage these findings to advance automated approaches in security requirements engineering.
O mercado de desenvolvimento de software está crescendo rapidamente, e projetos de software livre (SL) oferecem oportunidades valiosas para novatos. Contudo, as barreiras técnicas e não técnicas ainda dificultam as primeiras contribuições. Embora estudos busquem mitigar esses desafios, persistem as limitações. Exemplos trabalhados ajudam aprendizes a cometer menos erros, mas é difícil encontrar exemplos estruturados. Muitos desenvolvedores recorrem ao Stack Overflow, estratégia que nem sempre é eficaz. Recentemente, modelos de linguagem como o ChatGPT transformaram a busca por soluções. Considerando a reutilização frequente de códigos do Stack Overflow e o uso crescente do ChatGPT, este estudo investigou se respostas dessas fontes poderiam funcionar como exemplos trabalhados para apoiar o aprendizado e a resolução de problemas de novatos. Para isso, usamos uma issue do projeto Pandas como estudo de caso e conduzimos entrevistas individuais com o protocolo Thinking Aloud, seguidas de análise qualitativa. Os resultados sugerem que exemplos trabalhados dessas fontes podem auxiliar novatos em suas primeiras contribuições a projetos de SL.
The asynchronous evolution of test and production code can compromise software quality and maintainability. However, identifying and analyzing such co-evolution dynamics remains a complex task, often hindered by the lack of scalable tools and comprehensive visualizations. This paper presents Highlight Test Code, a web-based platform for analyzing the co-evolution of test and production code in open-source repositories. The platform implements a multi-stage pipeline encompassing repository mining, time series generation, clustering, and statistical correlation. It is built upon a curated dataset of 526 GitHub repositories across six programming languages. Highlight Test Code offers visual analytics and AI-generated insights that assist users in identifying test evolution patterns, evaluating co-evolution levels, and understanding their relationship with maintenance activities and project characteristics. The tool enables researchers and practitioners to assess testing practices at scale. Video link: https://youtu.be/U29eEg_gXXM. Software License: General Public License (GPL).
Desde o surgimento dos GPTs no mercado, há uma demanda crescente por estudos que investiguem seu impacto em diversas áreas, incluindo a educação. Este estudo é um relato de experiência estruturado que explora o impacto de ferramentas baseadas em IA — especificamente o modelo GPT-4o via API da OpenAI — na aprendizagem de alunos em uma disciplina de Engenharia de Software. A ferramenta proposta foi projetada com system prompts específicos para reduzir a carga cognitiva dos alunos, permitindo que se concentrassem na tarefa pedagógica. Dados de uso de 28 participantes foram coletados e analisados utilizando métodos estatísticos e processamento de linguagem natural. Os resultados indicam que alunos sem experiência profissional escreveram prompts mais longos, enquanto aqueles com tal experiência tenderam a curtir mais interações. Prompts curtidos tinham maior probabilidade de conter código, e alunos com experiência profissional também se mostraram mais propensos a incluir trechos de código em suas consultas. Este estudo contribui ao propor uma metodologia estruturada para coletar, processar e analisar dados de interação com IAs generativas em contextos educacionais, ajudando a orientar futuras pesquisas empíricas na área.
O ensino de programação enfrenta desafios relacionados ao engajamento e à retenção de estudantes, especialmente em disciplinas introdutórias que frequentemente apresentam altas taxas de evasão. Diversos fatores — como a complexidade dos conceitos iniciais em lógica de programação, estruturas de controle, manipulação de variáveis, desenvolvimento de algoritmos, uso de funções, arrays e listas, e estruturas de dados básicas como pilhas e filas — aliados à falta de motivação, podem dificultar a aprendizagem e o progresso dos alunos na área. Este artigo apresenta o BitBeak, uma ferramenta educacional gamificada que utiliza narrativas em estilo mangá para tornar o ensino de programação mais acessível e envolvente. O impacto dessa abordagem foi avaliado por meio do modelo MEEGA+, analisando usabilidade, experiência do jogador e percepção de aprendizado. O estudo envolveu estudantes universitários que interagiram com a plataforma, com aplicação de métricas quantitativas e qualitativas para mensurar engajamento e retenção de conhecimento. Os resultados indicam que a combinação entre gamificação e mangás aumenta a motivação e a compreensão conceitual dos alunos, apoiando os processos de ensino e aprendizagem em programação.
Context: Developers’ interactions on collaborative software development platforms like GitHub are key to maintaining technical alignment and community engagement. However, uncivil behaviors such as disrespectful, sarcastic, or offensive comments can undermine these efforts, discouraging contributions and harming code quality. Goal: This study introduces PeacemakerBot, an automated moderation tool that detects and warns developers of incivility signs in GitHub conversations. Method: We leverage Large Language Models (LLMs) to analyze conversations, identify signals of incivility, and generate reformulation suggestions in real time. To evaluate it, we conducted a user study with six developers, followed by a survey based on the Technology Acceptance Model (TAM) to understand their perception of the tool’s usefulness. Results: Our results suggest that PeacemakerBot successfully identifies multiple types of incivility and promotes more constructive conversations. The moderation feedback loop allows users to revise flagged comments, enhancing awareness and reducing harmful language over time. Conclusion: Our tool fills a key gap in OSS by providing AI-assisted moderation to enhance the social climate and inclusiveness of developer interactions. Video link: https://doi.org/10.5281/zenodo.15485535
Context: Although the use of Artificial Intelligence (AI) in software engineering is growing, the potential of Large Language Models (LLMs) to automate complex tasks in the software development process remains largely unexplored. Information-intensive decisionmaking tasks, such as selecting Design Thinking (DT) techniques for requirements elicitation, may significantly benefit from LLMbased solutions. Objective: This paper presents the automation of selecting DT techniques for specific project contexts, using LLMs as decision-support tools. We developed DT Selection Universe GPT, an LLM-based solution built on a structured repository containing 46 DT techniques and designed to operate using a prompt guided by the CRISPE model. Method: We investigated professionals’ perceptions of task automation through a qualitative study with four experienced participants from industry (INDT). The participants interacted with the tool and then participated in semi-structured interviews to share their perceptions of the experience. Results: Participants reported positive perceptions of using LLMs to automate the task of technique selection. They highlighted the system’s alignment with professional language, its adaptability to different contexts, the clarity of the recommendations provided, and the reliability of the results as key differentiators. Conclusion: The initial findings indicate that the DT Selection Universe GPT solution, developed using LLMs, shows strong potential to automate decision-support tasks for selecting DT techniques, offering benefits for novice and experienced professionals.
Context: Replication studies play an important role in strengthening the empirical foundations of Software Engineering (SE). However, the existing literature reveals that the reporting of Threats to Validity (TTVs) remains inconsistent or superficial, potentially undermining the reliability of the replication results. Objective: The goal of this study is to analyze how replication studies consider TTVs present in original studies in SE. Method: We conducted a Systematic Literature Review (SLR) that resulted in 83 replication studies published between 2022 and 2024. We analyzed the presence and specificity of TTVs in four validity dimensions (construct, internal, external, and conclusion), considering different research methods and types of replication. Results: Our analysis shows that replication studies in Empirical Software Engineering (ESE) tend to report threats to validity more frequently and in greater detail than original studies, particularly with regard to external and internal validity. Nevertheless, threats related to the validity of the conclusion and construct remain underreported. We observed that controlled experiments generally address the different types of TTVs more comprehensively, whereas surveys and case studies provide more limited coverage. With respect to types of replication, close and differentiated replications are predominant, while conceptual and internal replications remain underexplored in the field. Conclusion: Although there is growing attention to the identification of TTVs in replication studies, reporting remains uneven across validity dimensions and study types. More structured and diverse replication strategies are needed, along with better guidelines to support comprehensive TTV reporting and enhance the rigor and methodological value of replication efforts in ESE.
Os portais web das universidades federais são, além de importantes veículos de comunicação pública, ferramentas de transparência ativa. Portanto, devem conciliar diferentes aspectos, como usabilidade e experiência do usuário, com diversos requisitos legais. Considerando a natureza complexa desse contexto, este estudo analisa o cumprimento dos requisitos legais nos portais das 69 universidades federais brasileiras. A partir de uma pesquisa exploratória nas páginas iniciais desses portais, foram analisados diferentes aspectos, como a disponibilização de documentos acadêmicos, a apresentação da carta de serviços e a disponibilidade das informações de transparência ativa, requisitos definidos por um conjunto de leis. Os resultados apontam variações com as normativas, evidenciando lacunas na disponibilidade dos dados e a importância não apenas de se aprofundar as discussões sobre o tema, mas também de se propor novas abordagens de Engenharia de Software e Interação Humano-Computador para esse contexto.
Performance testing plays a critical role in maintaining software quality by ensuring systems meet their expected efficiency and responsiveness. However, defining precise test oracles for performance testing remains a significant challenge. As a result, many software projects lack reliable performance test oracles, hindering the development of comprehensive test suites. Approximate test oracles have emerged as a promising alternative, offering practical means of validation in the absence of exact specifications. In this work, we explore the use of n-version testing, a technique traditionally used for fault detection through the comparison of multiple system versions, as a foundation for constructing approximate performance test oracles. Our approach leverages the performance history of recent versions of the system under test (SUT) to define an acceptable performance range. Testers configure key parameters such as the number of prior versions to consider, the strategy for computing reference performance, and the tolerance margin. When the current version’s performance falls outside the derived tolerance band, an alert is raised to trigger further investigation. In our preliminary investigation using a real-world proprietary software system (an image gallery application), we used historical performance data to demonstrate that our proposed approach would have been capable of detecting a known performance bug, previously confirmed by the development team.
Defining and managing appropriate service granularity remains a recurring challenge in the design and evolution of microservices-based systems, directly affecting modularity, maintenance, and operational efficiency. This paper presents early empirical evidence supporting Granulify, a continuous granularity-management approach that dynamically adjusts service boundaries throughout the system lifecycle. Although still preliminary, the results already point to tangible benefits in modularity, maintainability, and operational cost, underscoring the method’s industrial applicability. The proposal is being validated through the reengineering of a real-world investment management platform from a major financial institution in Brazil. The analysed platform manages over 40 million transactions monthly and supports more than 250K internal users across investment, compliance, and trading domains. Preliminary results indicate patterns of architectural fragmentation, productivity impacts, and signs of granularity saturation. We believe that continuous application of this approach will contribute to more informed architectural decisions, balancing modularity, maintenance effort, and operational costs. This ongoing research aims to consolidate Granulify as a practical solution to support teams in the evolutionary management of granularity in microservices architectures.
Refactoring is essential for improving software maintainability, yet it often remains a validation-intensive and developer-guided task—particularly in Python projects shaped by fast-paced experimentation and iterative workflows, as is common in the machine learning (ML) domain. Recent advances in large language models (LLMs) have introduced new possibilities for automating refactoring, but many existing approaches rely on single-model prompting and lack structured coordination or task specialization. This study presents an empirical evaluation of a modular LLM-based multi-agent system (LLM-MAS), orchestrated through the MetaGPT framework, which enables sequential coordination and reproducible communication among specialized agents for static analysis, refactoring strategy planning, and code transformation. The system was applied to 1,719 Python files drawn from open-source ML repositories, and its outputs were compared against both the original and human-refactored versions using eight static metrics related to complexity, modularity, and code size. Results show that the agent consistently produces more compact and modular code, with measurable reductions in function length and structural complexity. However, the absence of a validation agent led to 281 syntactically invalid outputs, reinforcing the importance of incorporating semantic and syntactic verification to ensure transformation correctness and build trust in automated refactoring. These findings highlight the potential of LLM-based multi-agent systems to automate structural code improvements and establish a foundation for future domain-aware refactoring in ML software.
Usability defects in software systems result in challenges for users during their interactions with the software. To address these challenges, usability inspection is key for detecting defects during software development, allowing them to be fixed before the defects reach end users. It also plays a critical role after software implementation, during the software maintenance phase. This paper presents Solaria-GPT, a tool based on a tailored version of ChatGPT, designed to assist software engineers in identifying and classifying usability defects following Nielsen’s heuristics. Solaria-GPT facilitates two interactions: the detection of usability defects in user interfaces and the classification of each defect based on the violated heuristic. To evaluate the tool, we conducted a study to assess the performance of Solaria-GPT across textual and media-based inputs (screenshots and videos), focusing on its accuracy rate (correct heuristic violation), utility rate (valid defect identified by Solaria-GPT), and new defect rate (new defect identified by Solaria-GPT). The results indicated a 96.23% accuracy rate in heuristic classification from textual inputs. For utility rate and new defect rate, the Solaria-GPT achieved 86.67% and 86.67%, respectively, for screenshots, and 16.13% and 87.10%, respectively, for videos. Comparisons with other large language models (Claude, Qwen, Gemini, and Deepseek) demonstrated that Solaria-GPT outperformed all alternatives across metrics. These findings suggest that Solaria-GPT is a promising tool for enhancing software usability during the software development lifecycle. Demo Video: https://doi.org/10.5281/zenodo.15275798
[Context] Agile Software Development (ASD) and reuse strategies are increasingly used to improve software productivity and maintainability. However, while reuse relies on structured and traceable artifacts, ASD often depends on informal elements such as user stories, limiting opportunities for systematic reuse. A recent taxonomy proposes classifying user stories to support traceability and asset reuse, but manual classification remains labor-intensive and error-prone. [Objective] This study investigates whether Large Language Models (LLMs) can automate the classification of user stories using a reuse-oriented taxonomy, reducing manual effort while preserving annotation quality. [Method] We adopted an explanatory sequential mixed-methods approach. First, a two-step prompting protocol was applied to classify user stories from 12 real-world projects using GPT-4-turbo. Then, we compared model outputs to expert annotations, measuring agreement and qualitatively analyzing disagreements to identify causes and propose corrective actions. [Results] The LLM achieved a 48.1% agreement rate with human labels, with project-specific performance ranging from 14.0% to 84.4%. Notably, in 46% of disagreement cases, the LLM’s classifications were judged more appropriate than the human label, and in only 25% the human labels were judged to be correct, highlighting inconsistencies in the human annotation process despite prior validation. [Conclusion] These initial findings suggest that LLMs can effectively assist in classifying user stories for reuse purposes. Beyond reducing labeling effort, they offer the potential as reviewers in collaborative workflows to improve consistency, transparency, and the overall quality of software artifact organization.
In software engineering, effective communication is key to ensuring software quality. On GitHub projects, one of the primary channels for such communication is pull request discussions. These discussions often contain high-level design decisions. Understanding communication dynamics within development teams is crucial in software engineering, and sentiments and emotions play a significant role in this context. While there are datasets available for sentiment analysis in this field, those focusing on specific emotions are rare, and some contexts, such as pull request discussions, remain underrepresented. To address these gaps, we propose a novel methodology for capturing sentiments and emotions in contextspecific data, resulting in the creation of PRemo. PRemo includes ≈1.8K manually labeled pull-request messages from 36 active opensource industry-relevant projects. It provides data on individual emotions (and their intensity), the surrounding context, and evaluator confidence. Built using a robust triple validation and two-pass labeling process, the dataset leverages an established psychological emotion model. Already applied in prior research, PRemo is a valuable resource for advancing emotion analysis in software engineering.
Program debugging is one of the most time consuming activities carried out by developers because, in large extent, it is performed in ad hoc fashion. Significant research efforts have been directed to the development of new debugging techniques, in special, fault localization techniques. Spectrum-based Fault Localization (SBFL) is a debugging technique that has been experimented on programs similar to those developed in industry with promising results. To support the automated application of SBFL, tools have been developed, yet in the academic realm. Few of them, though, aim to support SBFL in the context of the Continuous Integration (CI) practice. We present the Jaguar Portal platform, designed and developed to receive, store, and provide access to SBFL information in CI environments. The platform encampsulates the Jaguar 2 tool, which is responsible for executing the SBFL technique during the build process in the CI pipeline on GitHub. Furthermore, it provides web interfaces that display SBFL analyses with code snippets and suspicious line markings.We believe Jaguar Portal is a step towards the adoption of SBFL techniques in industrial settings. Demo video: https://www.youtube.com/@JaguarPortalSBFL.
Cloud computing has become a commonly adopted solution to increase business operations efficiency and scalability. However, costs related to cloud infrastructure are increasingly prohibitive for some organizations in long-term deployment scenarios. This expense barrier has led to the emergence of a cloud repatriation trend, where existing software systems are moved back to on-premises environments. Although this initiative enhances cost control management and autonomy, it reintroduces technical challenges. This work investigates, through an empirical study, the applicability of Long Short-Term Memory (LSTM) networks for workload prediction to support auto-scaling decisions in on-premises environments. Initially, we contrasted the prediction accuracy of LSTM with classical methods, utilizing both standard benchmark datasets and access logs collected from an industrial-grade software system. In the second phase, we examined how varying temporal resolutions affect precision and computational expense. Results show the LSTM consistently outperforms classical methods, reducing MAPE by up to 18.02% and RMSE by 8.77% on benchmark datasets. Considering the in-field dataset, it outperforms in long-term predictions, especially regarding MAE and MAPE. Throughout all analyzed temporal resolutions, the 10-minute resolution optimally balanced accuracy with efficiency. Moreover,we noted that LSTM requires less training time than classical methods, emphasizing its applicability for real-world scenarios. Therefore, the results indicate that LSTM emerges as a viable solution to enable proactive auto-scaling within on-premises environments, potentially reducing costs in the context of cloud repatriation.
Over the past decade, the Laboratory for Extension in Development of Solutions (LEDS) has evolved from a small local initiative into an academic ecosystem that integrates teaching, research, and extension in Software Engineering. Founded in 2012 at the Federal Institute of Espírito Santo (IFES), LEDS simulates a real-world software development environment where students engage in multidisciplinary projects that serve public and private sector demands. Through project-based learning and interdisciplinary collaboration, LEDS fosters the development of both technical competencies and essential soft skills, enhancing student engagement and preparing them to face real-world challenges. This paper revisits LEDS over ten years after its inception, highlighting its organizational practices and impacts on student education and institutional innovation, through the creation of a research and extension culture grounded in experiential learning. Finally, we reflect on the lessons learned and challenges and offer practical insights for education professionals and researchers seeking to implement academic software factories as drivers of Software Engineering educational transformation and industry collaboration.
