With the success of language models in deep learning, multiple new time series embeddings have been proposed. However, the interpretability of those representations is often still lacking compared to word embeddings. This paper tackles this issue, aiming to present some criteria for making time series embeddings applied in deep learning models more interpretable using higher-level features in symbolic form. For that, we investigate two different approaches for extracting symbolic approximation representations regarding the frequency and the trend information, i.e. the Symbolic Fourier Approximation (SFA) and the Symbolic Aggregate approXimation (SAX). In particular, we analyze and discuss the impact of applying the different representation approaches. Furthermore, in our experimentation, we apply a state-of-the-art Transformer model to demonstrate the efficacy of the proposed approach regarding explainability in a comprehensive evaluation using a large set of time series datasets.
Question generation is the parallel task of question answering, where given an input context and optionally, an answer, the goal is to generate a relevant and fluent natural language question. Although recent works on question generation have experienced success by utilizing sequence-to-sequence models, there is a need for question generation models to handle increasingly complex input contexts with the goal of producing increasingly elaborate questions. Multi-hop question generation is a more challenging task that aims to generate questions by connecting multiple facts from multiple input contexts. In this work we apply a transformer model to the task of multi-hop question generation, without utilizing any sentence-level supporting fact information. We utilize concepts that have proven effective in single-hop question generation, including a copy mechanism and placeholder tokens. We evaluate our model's performance on the HotpotQA dataset using automated evaluation metrics and human evaluation, and show an improvement over the previous works.
Improving agent capabilities and increasing availability of computing platforms and Internet connectivity allows for more effective and diverse collaboration between human users and automated agents. To increase the viability and effectiveness of human-agent collaborative teams, there is a pressing need for research enabling such teams to maximally leverage relative strengths of human and automated reasoners. We study virtual and ad-hoc teams, comprising a human and an agent, collaborating over a few episodes where each episode requires them to complete a set of tasks chosen from given task types. Team members are initially unaware of the capabilities of their partners, and the agent, acting as the task allocator, has to adapt the allocation process to maximize team performance. The focus of the current paper is on analyzing how allocation decision explanations can affect both user performance and the human workers' outlook including factors such as motivation and satisfaction. We investigate the effect of explanations provided by the agent allocator to the human on performance and key factors reported by the human teammate on surveys. Survey factors include the effect of explanations on motivation, explanatory power, and understandability, as well as satisfaction with and trust / confidence in the teammate. We evaluated a set of hypotheses on these factors related to positive, negative, and no-explanation scenarios through experiments conducted with MTurk workers.
Similarity-based retrieval of semantic graphs is a crucial task of Process-Oriented Case-Based Reasoning (POCBR) that is usually complex and time-consuming, as it requires some kind of inexact graph matching. Previous work tackles this problem by using Graph Neural Networks (GNNs) to learn pairwise graph similarities. In this paper, we present a novel approach that improves on the GNN-based case retrieval with a Transfer Learning (TL) setup, composed of two phases: First, the pretraining phase trains a model for assessing the similarities between graph nodes and edges and their semantic annotations. Second, the pretrained model is then integrated into the GNN model by either using fine-tuning, i.e., the parameters of the pretrained model are further trained, or feature extraction, i.e., the parameters of the pretrained model are converted to constants. The experimental evaluation examines the quality and performance of the models based on TL compared to the GNN models from previous work for three semantic graph domains with various properties. The results show the great potential of the proposed approach for reducing the similarity prediction error and the training time.
Deep learning has succeeded in various applications, including image classification and feature learning. However, there needs to be more research on its use in Intelligent Tutoring Systems or Serious Games, particularly in modeling user behavior during learning or gaming sessions using multi-modal data. Creating an effective user model is crucial for developing a highly adaptive system. To achieve this, it is necessary to consider all available data sources to inform the user’s current state. This study proposes a user-sensitive deep multi-modal architecture that leverages deep learning and user data to extract a rich latent representation of the user. The architecture combines a Long Short-Term Memory, a Convolutional Neural Network, and multiple Deep Neu-ral Networks to handle the multi-modality of data. The resulting model was evaluated on a public multi-modal dataset, achieving better results than state-of-the-art algorithms for a similar task: opinion polarity detection. These findings suggest that the latent representation learned from the data is useful in discriminating behaviors. This proposed solution can be applied in various contexts where user modeling using multi-modal data is critical for improving the user experience.
Emergency departments (EDs) face significant challenges in providing timely care due to the increase in patient volume and limited resources. To improve patient flow management, new strategies based on artificial intelligence, machine learning, computer modeling, and simulation have been developed, including serious computer games and virtual reality. We performed a systematic review of the use of serious games and virtual reality to train healthcare professionals in the ED.
Task-oriented dialog systems empower users to accomplish their goals by facilitating intuitive and expressive natural language interactions. State-of-the-art approaches in task-oriented dialog systems formulate the problem as a conditional sequence generation task and fine-tune pre-trained causal language models in the supervised setting. This requires labeled training data for each new domain or task, and acquiring such data is prohibitively laborious and expensive, thus making it a bottleneck for scaling systems to a wide range of domains. To overcome this challenge, we introduce a novel Zero-Shot generalizable end-to-end Task-oriented Dialog system, ZS-ToD, that leverages domain schemas to allow for robust generalization to unseen domains and exploits effective summarization of the dialog history. We employ GPT-2 as a backbone model and introduce a two-step training process where the goal of the first step is to learn the general structure of the dialog data and the second step optimizes the response generation as well as intermediate outputs, such as dialog state and system actions. As opposed to state-of-the-art systems that are trained to fulfill certain intents in the given domains and memorize task-specific conversational patterns, ZS-ToD learns generic task-completion skills by comprehending domain semantics via domain schemas and generalizing to unseen domains seamlessly. We conduct an extensive experimental evaluation on SGD and SGD-X datasets that span up to 20 unique domains and ZS-ToD outperforms state-of-the-art systems on key metrics, with an improvement of +17% on joint goal accuracy and +5 on inform. Additionally, we present a detailed ablation study to demonstrate the effectiveness of the proposed components and training mechanism
An abundance of biomedical data is generated in the form of clinical notes, reports, and research articles available online. This data holds valuable information that requires extraction, retrieval, and transformation into actionable knowledge. However, this information has various access challenges due to the need for precise machine-interpretable semantic metadata required by search engines. Despite search engines' efforts to interpret the semantics information, they still struggle to index, search, and retrieve relevant information accurately. To address these challenges, we propose a novel graph-based semantic knowledge-sharing approach to enhance the quality of biomedical semantic annotation by engaging biomedical domain experts. In this approach, entities in the knowledge-sharing environment are interlinked and play critical roles. Authorial queries can be posted on the "Knowledge Cafe," and community experts can provide recommendations for semantic annotations. The community can further validate and evaluate the expert responses through a voting scheme resulting in a transformed "Knowledge Cafe" environment that functions as a knowledge graph with semantically linked entities. We evaluated the proposed approach through a series of scenarios, resulting in precision, recall, F1-score, and accuracy assessment matrices. Our results showed an acceptable level of accuracy at approximately 90%. The source code for "Semantically" is freely available at: https://github.com/bukharilab/Semantically
In humans, intrinsic motivation is an important mechanism for open-ended cognitive development; in robots, it has been shown to be valuable for exploration. An important aspect of human cognitive development is episodic memory which enables both the recollection of events from the past and the projection of subjective future. This paper explores the use of visual episodic memory as a source of intrinsic motivation for robotic exploration problems. Using a convolutional recurrent neural network autoencoder, the agent learns an efficient representation for spatiotemporal features such that accurate sequence prediction can only happen once spatiotemporal features have been learned. Structural similarity between ground truth and autoencoder generated images is used as an intrinsic motivation signal to guide exploration. Our proposed episodic memory model also implicitly accounts for the agent's actions, motivating the robot to seek new interactive experiences rather than just areas that are visually dissimilar. When guiding robotic exploration, our proposed method outperforms the Curiosity-driven Variational Autoencoder (CVAE) at finding dynamic anomalies.
In machine learning, the structure of feature space is an important factor that determines the performance of a model. Therefore, we can deepen our understanding of learning algorithms if we can visualize changes in the structure of feature space during the learning process. However, visualizing such changes is difficult because it requires dimensionality reduction while maintaining consistency with the data structure in high-dimensional space and in the temporal direction. In this study, we visualized feature changes during the learning process by capturing them as changes in the positional relationship between target features and time-invariant reference coordinates with a log-bilinear model.
An agent in pursuit of a task repeatedly perceives its environment through sensors, updates its state based on observations, and then decides which action to take, given the current state of the environment. Observations have in common that they are made at a given time point and thus referred to as temporal data. Usually, such temporal data is provided as stream data if the agent continuously receives the data, or it is provided as historic data if the stream data is stored in, for instance, a database the agent has access to. DBMSs are especially designed to process static data (i.e. non-temporal data) given a declarative query language such as SQL. However, if the aim is to exploit temporal data as required in time series analysis, SQL has its limits because it does not provide useful abstractions such as a window operator. Hence high-level declarative stream query languages, equipped with time-based window operators were designed. A challenge of those abstractions is the additional overhead of the algorithms that automatically transform high-level queries into low-level queries executable over DBMSs. If not handled properly those transformation algorithms may result in low-level queries with processing times too long for agents to make decisions. We describe a robust and optimized transformation algorithm for a high-level declarative stream query language and show that it leads to low-level queries with feasible processing times on real-world data.
The field of Natural Language Processing is gaining increased attention for the Aspect Based Sentiment Analysis task due to its ability to provide fine-grained information. This paper introduces SBERTiment, a novel approach to perform Aspect Based Sentiment Analysis. The method extracts relevant topics along with their sentiments from the input text by using a 2-step pipeline. In the first step, a token classification model is used to identify the relevant aspect terms and their sentiments. In the second step, a Sentence-BERT embedding model maps each aspect term to a predefined aspect category. Our approach has been tested on benchmark datasets and has achieved scores that are comparable to the best-performing methods. The pipeline is also able to perform zero-shot classification, which means it can extract information in unseen domains without additional training. When evaluated on a dataset with unseen aspect categories, SBERTiment achieved the best score among benchmark approaches.
Migratory waterfowl (i.e., ducks, geese, and swans) management relies on landscape bioenergetic models to inform on-the-ground habitat conditions and conservation practices. Therefore, conservation planners rely on accurate predictions of wetland habitats for waterfowl at regional scales. Unharvested flooded corn is a popular management tool on public and private lands that greatly increases landscape-level energy compared to other wetlands; thus, landscape bioenergetic models are particularly sensitive to these habitat features. Despite their importance to conservation planning and implementation, the abundance and distribution of unharvested flooded corn fields across North America is unknown. Furthermore, training data is difficult to collect and accurate predictions are challenging given their unique attributes and discreteness at landscape-level lens. Advances in multispectral imagery and deep learning algorithms may enable continuous and autonomous detection of these habitat features. Therefore, we conducted modeling experiments using training data of unharvested flooded corn fields in West Tennessee and multispectral imagery collected from Sentinel-2 satellite missions. We performed several experiments using individual band combination composites and/or vegetation indices to identify optimal bands using MRUNET architectures. We subsequently used 3 ensemble models of important individual networks. We found the use of multispectral bands was necessary and although the CIR composite and OSAVI index improved precision, the 12-band composite increased recall, the metric we were most interested in. Moreover, all ensembles exhibited poor performance. Here, we present results of our initial modeling experiments and suggest future modeling exercises including temporal image and vegetation index stacking using multi-modal and/or recurrent neural network architectures.
In this paper, we present a solution to the area coverage problem using a team of mobile robots with state estimation. A group of autonomous mobile robots is deployed in a two-dimensional area of interest, for example, where communication among robots is limited or noisy as expected in a real life scenario. Each robot estimates its own state (position and orientation) using noisy range and bearing information received from other robots in its operating range. The area of interest is then divided into multiple sub-area using a voronoi tessellation. Using the classical Lloyd's algorithm, each robot employs distributed action command to move towards the centroid of the voronoi cell that it belongs yielding the maximum coverage of the area. Here we emphasize that the network of autonomous robots deployed in the environment is unknown a priori. A set of computer experiments is conducted to validate the fact that the area coverage is still possible under noisy communication among robots deployed in a two-dimensional area.
Hierarchical Task Networks (HTNs) are a common formalism for automated planning, allowing to leverage the hierarchical structure of many activities. While HTNs have been used in many practical applications, building a complete and efficient HTN model remains a difficult and mostly manual task. In this paper, we present an algorithm for learning such hierarchical models from a set of demonstrations. Given an initial vocabulary of tasks and accompanying demonstrations of possible ways to achieve them, we present how each task can be associated with a set of methods capturing the knowledge of how to achieve it. We focus on the algorithms used to learn the structure of the model and to efficiently parameterize it, as well as an evaluation in terms of planning performance.
The contribution of this article is to bring the interesting features of SPARQL to the conceptual graph model while preserving its distinctive features and especially the graphical ones.Therefore this article presents a general conceptual graph query language, called CGQL.This article also provides an experimental comparison between CGQL and SPARQL engines: it shows the interest of our implementation for querying knowledge.
Machine learning has shown to be a crucial part of big data analytics; however, it lacks when the data is continuously streaming in from the system and changing too much from the original training data. Online learning is machine learning for streaming data that arrives in a sequential order where the model updates after every data point. While machine learning relies on well-established libraries such as PyTorch and Keras, the libraries for online learning are less well known, but they are here to serve similar purposes of reproducibility and reducing the time from research to production. Here, we compare different libraries for online learning research, specifically supervised learning. We compare them on the axes of developmental experience and benchmark testing as researchers. Our comparison as developers takes maintenance, documentation, and offerings of state-of-the-art algorithms into account. As this is not necessarily free of bias, we also use benchmarks known to online learning to gather power usage, RAM usage, speed, and accuracy of these libraries to get an objective view. Our findings show that Avalanche and River, including River-torch, are among the best libraries in terms of performance and applicability to the research in supervised online learning.
Most studies in graph theory assume that graphs are static, but in reality, graph structures and features change over time, leading to the concept of dynamic graphs, which is an under-researched area. Contemporary research in dynamic graph representation learning typically treats different snapshots of the graph as separate entities, disregarding the benefits of incorporating temporal information. While some techniques try to solve this problem using recurrent neural network-based solutions, these approaches still face the challenge of the vanishing or exploding gradient problem and complicated training procedures. To address these issues, we propose DynamicG2B, a BiLSTM-based graph neural architecture that computes node representations guided by attention using neighborhood aggregation. Our method applies relevant attention weights at different time steps to classify nodes in a supervised manner, utilizing dynamic edges and node feature information. Our evaluation of two benchmark datasets shows that DynamicG2B outperforms seven state-of-the-art baseline models in node classification in dynamic graphs. Additionally, our analysis of attention weights opens up opportunities for further research into exploring the importance of relationships among graph nodes.
This paper introduces the Smart Passenger Center (SPaCe), MerMec's solution for real-time optimization of urban public transport. The proposed system, based on cutting-edge artificial intelligence technologies, provides real-time vehicle status information and passenger activity monitoring. All of the information acquired from the vehicles is used to optimize the performance of the vehicle fleet for real-time traffic management.The purpose of the paper is to explain the reasons and benefits of such Intelligent Transport System (ITS) with a deep understanding of its architecture and advanced functionalities.Additionally, a practical application of the data collected by SPaCe is proposed for the task of bus timetable optimization.The goal is to schedule bus trips in a way that maximizes multiple conflicting goals such as service quality and operating costs, based on real time collected data.The reported results show how the developed optimization system can support the decision-making process to balance the interests between passengers and public transport agencies.
Social Media provide voluminous posts about current topics and events. When a user desires to investigate a popular topic, it is not feasible as there are many posts. Besides, posts show different biases, viewpoints, perspectives, and emotions. Thus, providing summaries of large post sets with different viewpoints is necessary. We develop a multiple view summa-rization framework to generate different view-based summar-ies of Twitter posts. Users can apply different methods to generate summaries: 1) Entity-centered, 2) Social feature-based, 3) Event-based summarization, using all triple embed-dings and 4) Sentiment-based summarization to generate summaries of positive or negative views of tweets. These summarization methods are compared with BertSum, SBert, T5, and Bart-Large-CNN with a gold standard dataset. Our results, based on Rouge scores, were better than these pub-lished extractive and abstractive summarization models.
