Text-based emotion analysis, an important task in Natural Language Processing (NLP), aims to identify and understand emotional tendencies in text. Recently, given their strong performance in text classification, Graph Neural Networks (GNNs) have been utilized in various emotion recognition studies. They have excellent structural modeling abilities but lack context encoding strength. On the other hand, Large Language Models (LLMs) such as BERT and GPT are specially designed to model the text context. Aiming to utilize both their advantages, we investigated several ways to combine GNNs with LLMs for the emotion recognition task. First, we used BERT to generate embeddings for the graph document nodes. Next, we extended the system to include a description of the input data’s emotional content obtained from GPT as an additional node embedding. For experiments and system evaluation, we used the GoEmotions dataset. The results clearly show that combining GNN and LLM improves the emotion classification performance by 20
Federated learning systems face critical security risks from data poisoning attacks, where malicious clients manipulate training data to compromise model integrity. Traditional detection methods focus on isolating clients that frequently deviate from the average weight update across training rounds. Building upon this concept, this paper introduces an advanced detection strategy that identifies malicious clients through the analysis of similarities in their updates rather than deviations from the average. Our method computes the Euclidean distance between clients’ weight updates vectors over the training rounds. If some clients consistently appear in close proximity to each other, beyond a predefined threshold, they are flagged as potentially malicious. This approach not only refines detection by focusing on synchronization patterns among attackers but also enhances the robustness of the federated model against coordinated data poisoning attacks. We demonstrate the efficacy of our detection method through systematic experiments and discuss optimal hyperparameter tuning strategies, offering a significant step forward in securing federated learning environments.
The presence of complex time series and multimodal data in economics and finance necessitates the development of advanced analytical models capable of interpreting complex patterns and dynamics. Despite their fast development as LLM, the current AI systems cannot do that. This paper introduces how a novel computational architecture, NeuDen, which combines the strengths of evolving spiking neural networks (eSNNs) with the interpretability of evolving dynamic neuro-fuzzy systems (eDNFS), can be used to model high-dimensional temporal data in economics. The NeuDen model is applied to the analysis of remittance inflows, demonstrating its ability to not only model data but also uncover the underlying trends in intricate remittance data. This application highlights the model's potential as a tool for efficient modelling of economic data that also helps understand the dynamics of economic processes.
Data-based approach for diagnosis of thyroid disorders is still at its early stage. Most of the research outcomes deal with binary classification of the disorders, i.e. presence or not of some pathology (cancer, hyperthyroidism, hypothyroidism, etc.). In this paper we explore deep learning (DL) models to improve the multi-class diagnosis of thyroid disorders, namely hypothyroid, hyperthyroid and no pathology thyroid. The proposed DL models, including DNN, CNN, LSTM, and a hybrid CNN-LSTM architecture, are inspired by state-of-the-art work and demonstrate superior performance, largely due to careful feature selection and the application of SMOTE for class balancing prior to model training. Our experiments show that the CNN-LSTM model achieved the highest overall accuracy of 99 % , with precision, recall, and F1-scores all exceeding 92 % across the three classes. The use of SMOTE for class balancing improved most of the model’s performance. These results indicate that the proposed DL models not only effectively distinguish between different thyroid conditions but also hold promise for practical implementation in clinical settings, potentially supporting healthcare professionals in more accurate and efficient diagnosis.
The paper presents a pilot study aimed at developing a novel approach for discovering, tracking over time, and summarizing the content of available publications on a given significant topic. A key result of this study is the proposed methodology for summarizing information from multiple articles on a specified topic by using large language models like BART and GPT with the help of fine tuning and prompt engineering. The developed approach has been applied in the creation of a software tool that collects publicly available information on a given topic from reliable sources for a specified period of time and displays in a concise form the most essential of the content of the publications found, thereby tracking the history of information on the topic.
This study investigates the machine learning techniques for unsupervised image classification and quality assessment in the domain of ultrasound imaging. Leveraging Convolutional Neural Networks (CNNs) for feature extraction and subsequent integration into a Support Vector Machine (SVM) model, we explored a novel approach aimed at accurate image classification. The dataset comprises high-frequency images in the form of image sequences depicting the facial skin of females. The study's primary emphasis was to categorize ultrasound images based on learned deep features, offering a distinctive framework for unsupervised image classification. The investigation employed CNNs to extract deep features from images, enhancing the SVM model's performance in accurately categorizing images. The incorporation of gamma correction as a preprocessing step further augmented the accuracy and sensitivity of the models. The SVM model exhibited exceptional performance, achieving accuracy rates exceeding 95.43
This paper investigates the efficacy of the Neuro-Evolution of Augmenting Topologies (NEAT) algorithm on PSPACE-complete problems, specifically utilizing the Sokoban puzzle. NEAT, which evolves both neural network topologies and weights, provides a promising approach for solving complex problems without predefined network architectures. We implemented NEAT using the neat-python library and tested it against several reinforcement learning (RL) algorithms, including Deep Q-Network (DQN) and Proximal Policy Optimization (PPO), within the OpenAI gym-sokoban environment. Our experiments involved extensive configuration variations to identify optimal settings for NEAT. Key findings indicate that NEAT solved the Sokoban problem, outperforming traditional RL variants. Our results highlight the importance of incremental structural growth and the protection of topological innovations. This study confirms NEAT’s applicability to PSPACE-complete problems.
Style transfer is essential for high-quality machine dubbing. While numerous approaches for style transfer in speech synthesis have been developed, cross-lingual style transfer remains a significant challenge. In this paper we introduce a novel speech synthesis method which realizes style learning across different languages and speakers. Our approach features a transformer-based architecture with a speech prompted text encoder, a duration predictor and a flow matching generative decoder. The text encoder is conditioned on the noisy source language speech, which is entered as speech prompt for style adaptation. The flow matching generative decoder produces high quality speech conditioned on the text encoder output. Empirical evaluations demonstrate that our TTS system generates speech that is objectively closer to the recordings of professional voice talents compared to a strong baseline model. Audio samples are available on our demo page ( http://lml.bas.bg/ stoyan/dubbing ).
This paper addresses the challenges of searching and extracting information from lecture videos as found on online platforms. With the increasing popularity of educational videos, learners are faced with the growing challenge of video search and finding relevant sequences within videos to answer questions. This research describes the development of a framework that relies on Retrieval-Augmented Generation (RAG) to address this challenge with a focus on the nature of lecture videos, which usually rely heavily on audio data (voice of the lecturer) and image data (lecture slides) that can partially be extracted as text. The resulting artifact allows a user to ask a question based on a corpus of videos and receive an answer based on the videos’ contents, along with information regarding the most relevant videos and sequences within the corpus. This research illustrates that a RAG system leveraging textual context from lecture videos is effective at improving question answering accuracy compared to relying solely on a standalone Large Language Model (LLM) without contextual enhancements. Further findings revealed that boolean answer correctness can aid in optimizing hyperparameters, that the independent retrieval of text from audio and visual modalities outperforms more complex strategies, and that retrieval followed by re-ranking outperforms other strategies.
Most artificial intelligence (AI) applications are designed under the model-centric AI (MCAI) approach, where data scientists aim to optimize the machine learning (ML) models starting with fixed, preprocessed data. However, businesses often struggle with limited datasets, changes in data over time, and limited ML knowledge, making it difficult to maintain data quality. One potential approach is data-centric AI (DCAI), which systematically improves the data quality used to build AI systems. However, rapid growth in data volume leads to challenges in selecting the most suitable operations for enhancing data quality thereby maintaining data accuracy, completeness, consistency, and reliability in real-world applications. To address this gap, we propose a novel framework, namely Layered Data-Centric AI (LDCAI) by expanding upon the existing DCAI Pipeline. LDCAI employs a three-layered approach, starting with the DCAI layer which generates the initial training data. This data is then analyzed and refined in the Data Analysis (DA) layer by a team of data scientists and domain experts. The refined data is later sent to the Data Quality Control (DQC) layer for additional quality checks and improvements by data scientists. The entire process can be iterated as needed to continuously improve data quality, which can be fed back into the DCAI layer’s model to achieve better predictions and outcomes. The implementation is illustrated through a scalable and customizable architecture designed for optimizing data quality within AI systems.
This paper introduces an efficient approach, the dynamic coefficient polynomial model, which emulates crop growth dynamics using NDVI. This model, a significant improvement over traditional models like the NDVI mean and static polynomial models, is designed to be adaptable over time and incorporates spatial variables to account for the diverse growth conditions experienced in different regions. Consequently, the model’s responses and adaptations are influenced by the specific crop growth dynamics observed within these spatial dimensions, adding a new dimension to crop growth forecasting. Our results show that the proposed model achieves a higher accuracy than the other machine learning models, which is about 90.3
This work presents a biologically plausible approach to reinforcement learning control of benchmark cart-pole system with continuous state and discrete actions. The proposed solution utilizes brain-inspired spiking neural networks (SNN), constructed with the aid of the NEST simulator in Python. To update the SNN connection weights, the Temporal Difference TD(0) algorithm is combined with Spike Timing Dependent Plasticity (STDP), reflecting principles of reinforcement learning. It is shown that SNN actor-critic architecture solves the task several times faster than classical neural networks.
Deep neural networks (DNNs) have demonstrated remarkable performance in various computer vision tasks. However, they face challenges that can inhibit their performance and transparency such as the learning of spurious patterns and a lack of explanatory power. This paper addresses these challenges in the domain of affect recognition, particularly for facial expressions. Our first contribution focuses on the integration of domain-specific knowledge into DNNs. To achieve this, we improve on a regularization method that constrains class co-occurrences, thereby outperforming existing state-of-the-art approaches. Our second contribution evaluates the impact of this regularization by employing an adapted explainable AI (XAI) method that incorporates expert knowledge. The results reveal that the regularization term encourages the learning of more generalized features. Consequently, XAI methods enhance the transparency of DNNs, contributing to the development of more reliable AI systems.
Number of engineering systems can be characterized as complex since they have a dynamic and nonlinear behaviour incorporating a stochastic uncertainty. On the other hand, as a machine learning method, Gaussian processes (GP) provide a practical, probabilistic approach to learning in kernel machines. This makes them very suitable for obtaining probabilistic, nonparametric black-box models of stochastic nonlinear dynamic systems. In this paper, a novel study on the modeling and adaptive optimal control of a tubular reactor is made by using Gaussian processes. Such reactor is a typical example of a nonlinear distributed parameters system, whose first-principles dynamic models consists of partial differential equations. The purpose is to obtain a nonlinear autoregressive models with exogenous input (NARX) of the output concentration and temperature of the reactor by applying a GP modeling approach. The identified surrogate models are then used to design an adaptive model predictive controller to achieve optimal performance of the reactor despite of the stochastic changes in the feed temperature. The performance of the adaptive MPC based on GP models is studied by simulation experiments.
The usage of memes and other visual material coupled with text on social media has been on the rise recently. Recognizing that visual signals are consumed quickly and can trigger emotional responses. It has become essential to discern the sentiment of such content, as it could significantly influence social media users. The paper focuses on the sentiment of memes on popular social networking platforms such as Instagram, Reddit, Facebook, and Tumblr. Our goal is to understand how these memes affect people in a positive, negative, or neutral way. We create a balanced dataset of 5,592 memes using distant supervision, i.e., automatically assigning sentiment labels based on different social media attributes, e.g., hashtags. We verify the accuracy of these labels by manually checking a random subset of the data. We conduct unimodal and multimodal experiments to explore how different cues contribute to identifying sentiment. Our results show that multimodal approaches, combining images and text, effectively identify the emotions in memes. We further experiment with novel closed and open-source LLMs, and we show that they outperform traditional multimodal approaches. The dataset is released publicly.
Motivated by recent research findings on glioblastoma multiforme (GBM) highlighting the insufficiency of relying solely on imaging data for predicting the O6-methylguanine-DNA methyltransferase (MGMT) methylation status, this study takes an innovative approach by integrating both preoperative clinical and non-invasive acquisition data. A stacked dataset, combining predictions from models developed on these two distinct data sources, is employed for the analysis. Leveraging the Hard Voting Classifier ensemble (Random Forest (RF), XGBoost (XGB), Support Vector Machine (SVM)) on a resampled dataset with replacement, the models demonstrated an average accuracy of 85.9
Emergencies can cause complex challenges for emergency hospitals, requiring an effective allocation of limited resources to reduce the loss toll. In this paper, we address a declarative modeling approach for the problem of assigning casualties to hospitals in an agent-based simulation context for emergency crises. Our approach leverages autonomous and decision-making-capable agents to optimize resource allocation while addressing various constraints, including but not limited to resource shortages and patient prioritization criteria. In addition, we present simulation outcomes that illustrate the effectiveness of the proposed approach in various randomly generated mass casualty incident scenarios. Furthermore, we conclude that our approach offers a promising direction to increase the efficacy and resilience of the rescue team’s responses in the face of mass casualty incidents.
Recent advances in large language models (LLMs) have led to new summarization strategies, offering an extensive toolkit for extracting important information. However, these approaches are frequently limited by their reliance on isolated sources of data. The amount of information that can be gathered is limited and covers a smaller range of themes, which introduces the possibility of falsified content and limited support for multilingual and multimodal data. The paper proposes a novel approach to summarization that tackles such challenges by utilizing the strength of multiple sources to deliver a more exhaustive and informative understanding of intricate topics. The research progresses beyond conventional, unimodal sources such as text documents and integrates a more diverse range of data, including YouTube playlists, pre-prints, and Wikipedia pages. The aforementioned varied sources are then converted into a unified textual representation, enabling a more holistic analysis. This multifaceted approach to summary generation empowers us to extract pertinent information from a wider array of sources. The primary tenet of this approach is to maximize information gain while minimizing information overlap and maintaining a high level of informativeness, which encourages the generation of highly coherent summaries.
A wide range of algorithms for computing textual similarity have been proposed. Much recent work has been aimed at calculating lexical similarity, but in general such calculations have to be treated as components in larger algorithms for computing similarity between sentences. In the current paper we describe a refinement of the well-known dynamic-time warping (DTW) algorithm for calculating the string edit distance between a pair of texts. The refined version of this algorithm allows for a range of constrained permutations without increasing the complexity of the underlying algorithm.
Automatic identification of intended tag meanings is a challenge in large image collections where human authors assign tags inspired by emotional or professional motivations. Algorithms for automatic tag disambiguation need "golden" collections of manually created tags to establish baselines for accuracy assessment. Here we show how to use the MIRFLICKR-25000 collection to evaluate the performance of our algorithm for tag sense disambiguation which identifies meanings of image tags based on WordNet or Wikipedia. We present three different types of observations on the disambiguated tags: (i) accuracy evaluation, (ii) evaluation of the semantic similarity of the individual tags with the image category and (iii) the semantic similarity of an image tagset to the image category, using different word embedding models for the latter two. We show how word embeddings create a specific baseline so the results can be compared. The accuracy we achieve is 78.6%.
