Objective: This study explores the quality of tests in Dart, the main language for mobile application development with the Flutter framework. Methods: The study begins by using the DNose tool, used to detect 14 types of test smells in code written in the Dart language. Next, we evaluate the tool’s precision, accuracy, recall, and F1-score. Using this tool, we conduct a detailed analysis of tests in open-source projects extracted from the language’s central repository. Results: The study starts with a dataset of 5,410 Dart-language projects, from which we were able to clone 4,154 repositories after processing. Based on the cloned projects, we generated a dataset containing 907,566 occurrences of test smells. Through our analysis, we characterized the specific types of test smells most frequently encountered and identified their causes. We observed the presence of test smells in 74% of test files. Another noticeable characteristic among the analyzed projects was the scarcity of tests, with 1,873 projects having one or no tests, which led us to expand the number of analyzed projects to a broader base. Conclusion: This research makes a significant contribution by providing insights into the quality of tests in projects from Dart’s official repository, as well as by offering an open-source tool for detecting 14 types of test smells.
The main goal of reviews in software development is to grant the quality and correctness of the products. One of those tasks that we can highlight is image review. This process could require more time to process and tends to be less accurate due to human error. Considering that developers who perform this type of task also have other responsibilities, the longer the review takes, the more work time is required, and this can affect the time of deliverables in the industry. Therefore, the proposal of this research is to use Optical Character Recognition (OCR) and Computer Vision (CV), to automate the process of image review, for the interpretation and validation of the content, both textual and objects. To carry out this research a set of 100 images was used in manual and automated review process and was verified that the automated approach reduced the execution time by 96.45%, and reduced the number of errors by 4%, proving that it is feasible in this development context. The application of this tool helped to increase efficiency by reducing the time spent on manual reviews, increased the consistency of image analysis, ensuring better accuracy compared to manual reviews. It also promoted improvements in development and the team, through a more productive work environment.
Context: In this study, we investigated the integration of Large Language Models (LLMs) in test teams, aiming to understand how these tools can improve the test process and improve productivity. The main goal was to determine if the implementation of LLMs can enhance the productivity of test professionals. Methodology: We performed an empirical study with 14 participants, and the results were analysed quantitatively and qualitatively. For the quantitative analysis, we analysed the frequency of activities, and for the qualitative analysis, we analysed the participants’ perception of productivity through a scoring system, in which participants gave a score between 0 to 5 for their perception of learning from before and after the implementation of LLM in the team. Results: The results show a significant improvement in productivity (p-value = 0,0193) after the implementation of LLM. Furthermore, the qualitative analysis provided data on the impact of LLM in learning, use experience, trust perception and productivity. Conclusion: It was noted that the integration of LLMs in test teams can be a valuable tool to improve productivity and efficiency of test professionals.
Background:Website layouts often change with new design trends and front-end frameworks. Quality assurance is necessary during these changes, but manual testing takes much time and money. Manual tests are the standard way to maintain quality, but they are slow and expensive. Changes in graphical interfaces can cause errors or break features, which affects quality.Writing manual tests is the most time-consuming part of the process. Aims: This paper presents a tool that uses ChatGPT to create Natural Language Tests from screenshots and operator instructions. The goal is to reduce the time spent on manual test creation and to maintain quality in both the tests and the application. Method:We used two evaluation methods. First, we conducted a survey with 18 software testing professionals and students to compare tests made by ChatGPT and by humans. Second, we used Natural Language Processing techniques to measure the similarity between ChatGPT-generated tests and human-made tests. Results: The qualitative analysis showed that ChatGPT tests exceed human tests in completeness by a difference of 5.56%, achieving 36.11% acceptance rate. Human tests exceeded ChatGPT tests in clarity by 6.95%, reaching 41.67% acceptance rate. The quantitative analysis found that 66.7% of ChatGPT tests shared over 50% similarity with human tests. Conclusions: Our tool can help automate the creation of software tests. The similarity between AI-generated and human-made tests shows that this approach can save time and reduce costs, while keeping test quality at an acceptable level. This framework can help maintain quality during changes in website layouts and application development.
