The growing availability of open health datasets has advanced medical research and healthcare innovation. This study proposes a Large Language Model (LLM) approach that enables Exploratory Data Analysis (EDA) through natural language queries by integrating Retrieval-Augmented Generation (RAG) and post-processing mechanisms. It was evaluated using five open health datasets, encompassing both structured and unstructured data. The results show that the approach effectively describes datasets, identifies outliers, and produces diverse visualizations, including histograms and correlation heatmaps. The method demonstrates the feasibility of using LLMs to automate and democratize EDA, enhancing accessibility and interpretability in health data exploration.

While the decomposition of monolithic applications into microservices has been widely studied, most existing approaches stop at architectural analysis or service identification, offering little support for the actual generation and deployment of microservices. This creates a gap between design and implementation, often requiring significant manual effort from developers. It also limits researchers, who lack practical tools to test and experiment with new decomposition strategies in real-world scenarios. To address these challenges, we present JS-Distributor, a tool that automates the transformation of monolithic JavaScript applications into microservices. JS-Distributor generates much of the required boilerplate code for Node.js, including server setup and inter-service communication code, supporting both HTTP APIs and asynchronous messaging systems. By bridging the gap between decomposition and deployment, the tool enables rapid experimentation with different microservice architectures. We evaluated JS-Distributor using a benchmark system commonly adopted in microservice research, successfully distributing the original monolith into functional microservices while preserving its original logic. Demo video: https://doi.org/10.5281/zenodo.15477214

We are currently witnessing growing concerns regarding fairness in online marketplace environments, particularly in situations involving platform-owned first-party selling, biased recommendation algorithms and trust formation. Regulatory measures, such as those imposed by the European Commission and the US Federal Trade Commission, underscore the importance of fair practices on these platforms. While machine learning ranking models are widely used in ranking multiple offers from various sellers, they can inadvertently introduce biases, raising regulators' concerns. Despite recent advances in fair ranking algorithms, the long-term impact of fairness in ranking remains understudied. The contributions of this research are threefold: 1) it establishes a simulated online marketplace environment to complement traditional static experiments; 2) it assesses the long-term impact of fairness in utility of ranking algorithms; and 3) it evaluates state-of-the-art fairness techniques in dynamic environments. Findings reveal how utility in ranking fairness algorithms can be affected by the application of fairness techniques and how data drift impacts regular and fair ranking algorithms in a long-term scenario.

Keyword extraction is an important step for text interpretation, serving to identify and highlight the most significant words or phrases within a text. This step is essential for various applications such as summarization, indexing, and information retrieval. This paper presents a custom-built keyword extraction pipeline named USKE (Unsupervised Statistical Keyword Extraction) and compares its performance to large language models (LLMs). USKE is able to deliver fast and simple results based in statistical methods even when dealing with large datasets. Our evaluation demonstrates that although LLMs can achieve good results in single sentences with minimal context, they require a lot of post-processing and may output inconsistent answers, while USKE excels in efficiency and scalability.

In recent years, optimizing classification pipelines has become increasingly critical due to the growing volume of textual data and the computational challenges associated with exhaustive hyperparameter tuning. This paper proposes a similarity-based approach for selecting the most promising vectorization configurations – specifically, Bag of Words (BoW), Term Frequency-Inverse Document Frequency (TF-IDF), and Word2Vec – by analyzing the average cosine similarity of the generated vectors; by preselecting configurations that yield more diverse textual representations, our method relies on the hypothesis that increased diversity in text representations enhances the discriminative capacity of classification models. Experimental evaluations conducted on five different datasets demonstrate that the similarity-based approach achieves accuracy and F1-Score results very close to those obtained via exhaustive search, with notable reductions in processing time; furthermore, correlation analyses reveal a strong inverse relationship between vector similarity and model performance for BoW and TF-IDF, and a moderate relationship for Word2Vec. These findings validate the efficacy of the proposed method as a practical alternative to hyperparameter selection in vectorization pipelines, offering significant benefits for applications where exhaustive exploration is unfeasible.

Download This Paper Open PDF in Browser Add Paper to My Library Share: Permalink Using these links will ensure access to this page indefinitely Copy URL Copy DOI

The software development process has been studied since the beginning of technological evolution. Development practices have evolved, requiring processes capable of supporting intensive work, paving the way for agile methodologies. With the growing need for continuous integration (CI) and continuous deployment/delivery (CD), new data architectures have emerged, allowing for scalable, maintainable, and reusable environments, collectively known as DevOps (Development + Operations). In this context, the DevDocOps approach integrates continuous documentation into the software development lifecycle. However, little has been published regarding the benefits of this approach. To address this, an empirical study was conducted, applying findings from the literature to a real development environment by integrating continuous documentation into the data engineering development lifecycle. Based on feedback from developers and technical lead, the results highlight the importance of technical documentation in an agile development environment and demonstrate how automating this process can improve the quality and efficiency of software deliveries.

NoSQL databases are designed to allow data to be stored without a fixed schema. In this scenario, writing data access code (queries and associated code) is challenging for two main reasons: (i) the query developer needs to know all details regarding the logical/physical schema; (ii) whenever the schema needs to change, all queries need to be rewritten. The literature presents some solutions to mitigate these challenges by making queries independent from the schema, but they are either restricted to simple queries or depend on a prescribed schema. In this paper, we propose a solution based on conceptual data modeling and code generation to enable the writing of complex data retrieval queries in a document-oriented database, independent from the logical/physical schema. We created a language for a classic ER algebra, with a concrete syntax and generative operational semantics targeting MongoDB. Based on the ideas of model management, the generative semantics uses a model that establishes the mapping between entities, relationships and document collections. From a single ER-based data retrieval query, native query code is automatically generated for any logical/physical schema. Additionally, the resulting data structure produced by the generated code is always the same. Tests show that our implementation produces consistent results. Also, we show that our solution is able to obtain up to 500% gains in throughput and response time by simply changing the schema, without modifying the queries. Our solution requires a little extra effort from the database designer, to create the mapping, but makes the job of the query developer easier. It also results in more flexibility. Whenever the schema changes, only the mapping needs to be updated, not the queries.

AutoML aims to find the best Machine Learning (ML) pipeline in a complex and high-dimensional search space by evaluating multiple algorithm configurations. However, training multiple ML algorithms is time-consuming, and as AutoML tools are frequently time-constrained, the exploration of the search space may find sub-optimal results. In this work, we explore the application of curriculum learning techniques to overcome this limitation. Curriculum and anti-curriculum learning have improved model performance and accelerated the training process on previous empirical investigations using optimization-based models by ordering examples during model training based on their difficulty. We apply and compare curriculum strategies on an AutoML system to accelerate the search space exploration and find good-performing machine learning pipelines efficiently. The results indicate that AutoML can benefit from a curriculum strategy. Furthermore, in most of the evaluated scenarios, the curriculum strategies led to better classification results.

Data preparation is a mandatory phase in the machine learning pipeline. The goal of data preparation is to convert noisy and disordered data into refined data that can be used by the algorithms. However, data preparation is time-consuming and requires specialized knowledge about the data and algorithms. Therefore, automating data preparation is essential to decrease the effort made by data scientists to develop satisfactory models. Despite its relevance, current AutoML platforms disregard or make simple hardcoded data preparation pipelines. Trying to fill this gap, we present a meta-learning-based recommendation system for data preparation. Our system recommends five pipelines, ranked by their relevance, making it useful for users with varying degrees of experience. Using the top-1 pipeline we demonstrated that our proposal allows a better performance of an AutoML system. Furthermore, the accuracy rates of our method were comparable to those achieved by a reinforcement-learning-based algorithm with the same goal, but it was up to two orders of magnitude faster. Moreover, we tested our method in a real-world application and evaluated its benefits and limitations in this scenario.

Network Architecture Search (NAS) has achieved impressive results and generated models comparable with humans' classifications. Automating the definition of a neural architecture reduces the need for expert work efforts and mitigates human bias from architecture design. NAS techniques usually consist of an algorithm to search for the best architecture in a predetermined space of parameters or functions. Due to the number of deep neural architectures' parameters, this search space includes millions of parameters, which makes NAS a cost procedure and may lead the search to overfit the training set. To reduce NAS search spaces' complexity and still obtain competitive results, we propose CoNAS, a committee of NAS-based models, by restricting the search spaces to perform Differentiable ARchiTecture Search (DARTS). Our results point to improved accuracy over DARTS on CIFAR-10, training the networks from scratch. and Imagnette, using a transfer learning approach.

Improving the interaction between consumers and marketplaces, focusing on reaching higher conversion rates is one of the main goals of e-commerce companies. Offering better results for user queries is mandatory to improve user experience and convert it into purchases. This paper investigates how named entity recognition can extract relevant attributes from product titles to derive better filters for user queries. We conducted several experiments based on MITIE and BERT applied to smartphones/cellphones product titles from the largest Brazilian retail e-commerce. Both of our strategies achieve outstanding results with a general F1 score of around 95%. We concluded that using a classical machine learning pipeline is still more useful than relying on large pre-trained language models, considering the model’s throughput and efficiency. Future work may focus on evaluating the scalability and reusability capacity of both approaches.

Cross-platform development solutions can help to make software available on different devices and platforms. But these are normally restricted to preconfigured platforms and consider that each individual solution is equal or similar to each other. As a result, developers have to resort to native development and build individual solutions, one for each device/platform, that cooperate to deliver the desired global functionality. This article presents an approach that takes advantage of existing solutions and have support for extending and including new platforms, and distributing functionality across devices. The approach is based on a general-purpose language that raises the abstraction level in order to keep the software free from platform details. Automatic transformations produce executable code that can be properly divided and deployed separately into different platforms. The proposed approach was evaluated in four ways. In the first evaluation, an existing cross-platform system was recreated using the approach. The second and third evaluations was conducted with expert and novice developers, who tested the approach in practice. The fourth evaluation introduced support for cross-platform testing. Results have brought evidence supporting the following main contributions: use of a single environment, the ability to reuse similar concepts between platforms and the potential to reduce costs.

Sharing a software product among multiple customers requires dealing with common and variable features. A Software Product Line approach may be used to manage these issues both during initial development and evolution. Many technologies can be used to implement a product line, and the micro services architectural style is a way to make the evolution process easier, since it can contain the impact of changes within smaller units, which are individually developed and deployed. Although this is an important subject, widely used in the industry, not many research papers investigate the impact of micro services in a real software product line, specially during evolution, when they are supposed to bring the most benefits. This research evaluated how the evolution of a software product line can benefit from the use of micro services. The study was based on a real software product line for the Enterprise Resource Planning domain, involving two versions of the product line: one without micro services, which was the original architecture and one with micro services, obtained after a migration from the original architecture to a micro services architecture. The results show that micro services brought benefits to the evolution of the software product line, but only in adaptive maintenance tasks. In evolutive tasks, where new services had to be created, the developers had to spend extra effort. The paper also describes some lessons learned during the migration. These may serve as guidance for researchers and practitioners interested in using micro services in a software product line.

Machine learning (ML) is becoming critical to many businesses. Keeping an ML solution online and responding is therefore a necessity, and is part of the MLOps (Machine Learning operationalization) movement. One aspect for this process is monitoring not only prediction quality, but also system resources. This is important to correctly provide the necessary infrastructure, either using a fully-managed cloud platform or a local solution. This is not a difficult task, as there are many tools available. However, it requires some planning and knowledge about what to monitor. Also, many ML professionals are not experts in system operations and may not have the skills to easily setup a monitoring and benchmarking environment. In the spirit of MLOps, this paper presents an approach, based on a simple API and set of tools, to monitor ML solutions. The approach was tested with 9 different solutions. The results indicate that the approach can deliver useful information to help in decision making, proper resource provision and operation of ML systems.

NoSQL databases are designed to fulfill performance and scalability requirements, normally by allowing data to be stored without a fixed schema. For this reason, it is not rare that new usage and performance requirements appear during a system's life cycle, demanding changes to be made in the schema, challenging the developer with extra adaptation effort to update data access code (database queries). The literature presents some solutions to reduce this effort by making queries independent from the schema, but the solutions are normally restricted to simple queries or a predefined mapping. In this paper, we present evidence showing that a classic ER algebra and a Model Management approach can be used to implement a solution that works with complex queries in any schema. The algebra defines operations that can be used by developers to specify complex queries in terms of Entities and Relationships. We created a language for this algebra, with a concrete syntax and a generative operational semantics targeting a document-oriented database. As in Model Management, the generative semantics is guided by the mapping information between Entities, Relationships, and Documents, and is able to generate, for a single ER-based input query, native query code for different schemas, all producing the same results in terms of data structure. Test results show that our implementation is consistent with the algebra's definition, producing evidence that this approach can lead to schema independence in complex NoSQL queries.

Currently, with the increasing number of connected devices and users, there are forces that lead to the distribution of software systems. Behind these forces is the need for increased reliability and performance, privacy and security issues, among other factors. However a distributed architecture has inherent complexity and is difficult to change. Thus, distributed-related design decisions are normally taken in the early stages of the software life cycle. Nonetheless, in a dynamic scenario, it may be difficult to predict how technologies or users may change in the future. Therefore, the ability to rapidly change how a software system is distributed may provide extra competitive advantage to institutions, allowing developers to better respond to market changes, or to more easily experiment with real different distribution configurations before delivering a final product or update. In this paper an approach to ease this task is presented. Using source code analysis, model-driven engineering and code generation, applications can be semi-automatically decomposed into services that can be easily distributed across multiple servers, without changing the original functionality. As a proof-of-concept, we used the approach to successfully distribute Apache Tomcat in distinct ways, in configurations that involve up to ten different servers.

Among research opportunities in software engineering for cloud computing model, interoperability stands out. We found that the dynamic nature of cloud technologies and the battle for market domination make cloud applications locked-id, i.e, proprietary, non-portable and non-interoperable. In general context of cloud computing, interoperability goes beyond communication between systems like in other fields, it goes in direction of more dynamic, heterogeneous, complex and composed applications that take advantage of best features from different providers and services simultaneously. Interoperability in cloud constitutes a great challenge that must be overcome for that, in the future, software be more dynamic and improved. Objective: This paper aims at identifying how interoperability in cloud computing has been addressed in the existing literature, offering an up-to-date view of concepts relate to how to develop interoperable software that takes advantage of different cloud models. Thus, providing a basis for further research in the field and consolidating e better exploring existing concepts. Method: To fulfill this objective, we surveyed literature. We defined six research questions and conducted the study according to a protocol that included planning, and execution. Results: A first result of the review is that there is no well established definition for cloud interoperability. This study also identified cloud interoperability concepts (e.g., cloud brokers, multi-cloud and cloud federation), requirements for interoperable applications and existing cloud interoperability solutions, showing that these are either too specific for particular situations. Finally, the survey found no evaluation models for cloud interoperability solutions. We also present a discussion on the findings of this study. Conclusion: Since the study observed that there are no well-established cloud interoperability solutions yet, we conclude that the issues raised by lack of interoperability persist. Selecting one interoperable solution or even a cloud standard can free the system from the underlying providers, but it would still be locked into the selected particular solution.

Context: In cloud computing, lock-in refers to the difficulty of porting an application and/or data from one cloud platform to another. Current attempts to address this problem revolve around standardization of APIs and frameworks. We propose a different path, using model-driven engineering (MDE).Objective: Our goal is to build a repository of MDE transformations and use code generation to reduce the development effort for each platform, thus reducing repetitive programing tasks, increasing portability and minimizing lock-in side-effects.Method: To attain this objective, we developed an MDE approach to handle persistence for Google App Engine and Azure, and discuss how MDE can reconcile the differences between features of each platform persistence model. A controlled experiment has been performed to evaluate the proposal, in which subjects were asked to use two versions of the same application implemented using our MDE approach. Both versions, one for each platform, were generated from the same domain model.Results: According to the subjects, no differences in functionality were perceptible between the two versions. Indeed, applications were more easily ported between the two chosen cloud providers without noticeable differences in terms of persistence functionality.Conclusion: The main contribution of our work is to show that there is an alternative path to the standardization of cloud technologies. MDE can increase the portability of the applications by reducing the negative impacts of lock-in. A limitation of our approach, that is inherent to most MDE approaches, is that if the generated code needs to be adapted or modified, the MDE life-cycle can be broken. Changes in the generated code have to be replicated, either in the models or in the transformations, which is not a trivial task.