: We wanl lo creale a working 1001 for malhematics leachers and a corresponding learning 1001 for sludenls, namely a graphical simulalor of malhemalical algorilhms (GraSMa). To achieve il we Iry Iwo differenl slralegies. We slarled by annolaling manually lhe original algorilhm wilh inspeclor functions. Now we are lesting a new approach Ihal aims ai aulomalically annolaling lhe original code wilh inspeclor funclions. To achieve Ihis we are developing a language Iranslalor module Ihal enables us lo commenl aulomatically on any code written in Oclave language. The run of lhe annolaled code galed by one of Ihese two ways, records in a XML (eXlensible Markup Language) file everylhing Ihal happened during lhe execution. Subsequentiy, lhe XML file is parsed by a Java applicalion Ihal graphically represenls lhe malhemalic objecls and Iheir behaviour during execulion. The final application will be accessed on-line Ihrough a websile (WebGraSMa) which is currentiy under developmenl. In Ihis paper we reporl and discuss aboul lhe procedures followed and presenl some intermediate results.
: The traditional pitch histogram and various features extracted from it play a pivotal role in music information retrieval. In the research on songs, especially applying pitch statistics to investigate the main melody, we found that the pitch histogram may not necessarily reflect the notes’ pitch characteristic of the whole song perfectly. Therefore, we took the note duration into account to propose two advanced versions of pitch histograms and validated their applicability. This paper introduces these two novel histograms: the merged pitch histogram by merging consecutively repeated pitches and the pitch-duration histogram by utilizing each pitch’s duration information. Complemented by the description of their calculation algorithms, the discussion of their advantages and limitations, the analysis of their application to songs from various languages and cultures, and the demonstration of their use cases in state-of-the-art research works, the proposed histograms’ characteristics and usefulness are intuitively revealed.
In this paper we propose spline models of audio as the first step toward a hierarchical system of level of detail (LOD) for audio rendering. We describe methods such as cycle interpolation which can produce spline models and approximations of audio data. These models can be used to render output in realtime, but can also be mixed prior to rendering. Examples of audio data simplified to spline models with cycle interpolation can reduce the data to less than 2% of the full resolution data size, with only minor impact to audio quality. We present a sequence of such examples with instrument models. We also introduce the idea of pre-rendered filtering and mixing, based on the B-spline coefficients of the models.
We present HERO, a Conversational Intelligent Assistant to support workers in industrial domains. The proposed system is able to interact with humans using natural language and observing the surrounding world in order to avoid the language ambiguity. HERO is composed of four modules: 1) the input module to process both text and visual signals, 2) the NLP module to predict user intent and extract relevant entities from text, 3) the object detection module to extract entities by analyzing images captured by the user and 4) the output module which is responsible for choosing the best answer to send to the user. To assess its usefulness in a real scenario, the proposed system is implemented and evaluated in an industrial laboratory setting. Preliminary experiments show that HERO achieves good performance in predicting intents and entities exploiting both text and visual signals.
In video error concealment, we estimate any missing information in the video frames as close to the actual data. In this paper, we present a video error concealment technique, named Convergent Error Concealment Neural Network (CECNN), based on Convolutional Neural Network (CNN). CECNN is a two-stage process where it first learns to predict voxel information from the training dataset. It then applies transfer learning using the pre-trained model from the first stage to produce intermediate outputs. CECNN consists of dedicated paths for the past and future frames to produce the intermediate outputs, which are then combined to fill the missing information in the errored frame. The quality of the outputs from CECNN is compared with other techniques, such as motion vector estimation, error concealment using neighboring motion vectors, and generative image inpainting. The evaluation results suggest that our CECNN approach would be a good candidate for error concealment in video decoders.
: Convolutional neural networks are often trained on RGB images because it is standard practice to use transfer learning using a pre-trained model. Satellite and aerial imagery, however, usually have additional bands, such as infrared or elevation channels. Especially when it comes to detection of small objects, like cars, this additional information could provide a significant benefit. We developed a semantic segmentation model trained on the combined optical and elevation data. Moreover, a post-processing routine using Markov Random Fields was developed and compared to a sequence of pixel-wise and object-wise filtering steps. The models are evaluated on the Potsdam dataset on the pixel and object-based level, whereby accuracies around 90% were obtained.
H.264 video compression has become the prevalent choice for devices which require live video streaming and include mobile phones, laptops and Micro Aerial Vehicles (MAV). H.264 utilizes motion estimation to predict the distance of pixels, grouped together as macroblocks, between two or more video frames. Live video compression using H.264 is ideal as each frame contains much of the information found in previous and future frames. By estimating the motion vector of each macroblock for every frame, significant compression can be obtained. Combined with Socket on Chip (SoC) encoders, high quality video with low power and bandwidth is now achievable. 3D scene reconstruction utilizing structure from motion (SfM) is a highly computational intensive process, typically performed offline with high computing devices. A significant portion of the computation required for SfM is in the feature detection, matching and correspondence tracking necessary for the 3D scene reconstruction. We present a SfM pipeline which uses H.264 motion vectors to replace much of the processing required to detect, match and track correspondences across video frames. Our pipeline results have shown a significant decrease in computation, while accurately reconstructing a 3D scene.
It is well known the existence of certain headaches in humans caused by the sympathetic hypofunction, either congenital or developed at birth. These pathologies, called cluster headaches, are physically manifested by the change in texture, color and/or intensity of the iris eye on the painful side. The automatic study of these variations would make it possible to provide quantitative measures of the existence of such pathology from color images of the left and right iris of a particular individual. In this context, this work analyzes the color of the left and right irises to identify chromatic differences between the irises belonging to the same individual by analyzing three color spaces. The iris color distribution in the same eye has been studied, as well as the degree of similarity and divergence between the chromatic distributions of irises in both eyes. Cross-correlation between color feature vectors exhibited low detection capabilities, whereas a relative measure based on the Kullback-Leibler divergence provided good performance to show color differences in the irises. No color space was identified as the most appropriate for evidencing color differences in all the scrutinized cases. The results obtained are promising on a dataset with eight patients, and can be considered a proof of concept on which it is necessary to extend the analysis with a larger database. From a practical viewpoint, this characterization could help to discriminate patients who attend the neurology department suffering from headache.
Deep learning Video Super-Resolution (VSR) methods rely on learning spatio-temporal correlations between a target frame and its neighbouring frames in a given temporal radius to generate a high-resolution output. Among recent VSR models, a sliding window mechanism is popularly adopted by picking a fixed number of consecutive frames as neighbouring frames for a given target frame. This results in a single frame being used multiple times in the input space during the super-resolution process. Moreover, the approach of adopting the fixed consecutive frames directly does not allow deep learning models to learn the full extent of spatio-temporal inter-dependencies between a target frame and its neighbours along a video sequence. To mitigate these issues, this paper proposes a Spatio-Temporal Input Frame Selection (STIFS) algorithm based on image analysis to adaptively select the neighbouring frame(s) based on the spatio-temporal context dynamics with respect to the target frame. STIFS is first-ever dynamic selection mechanism proposed for VSR methods. It aims to enable VSR models to better learn spatio-temporal correlations in a given temporal radius and consequently maximise the quality of the high-definition output. The proposed STIFS algorithm achieved remarkable PSNR improvements in the high-resolution output for VSR models on benchmark datasets.
The recognition of the emotions of humans is crucial for various applications related to human-computer interaction or for understanding the users' mood in several tasks. Typical machine learning approaches used towards this goal first extract a set of linguistic features from raw data, which are then used to train supervised learning models. Recently, Convolutional Neural Networks (CNNs), which unlike traditional approaches, learn to extract the appropriate features of their inputs, have also been applied as emotion recognition classifiers. In this work, we adopt a CNN architecture that uses spectrograms, extracted from audio signals as inputs and we propose data augmentation techniques to boost the classification performance. The proposed data augmentation approach includes noise addition, shifting of the audio signal, and changing its pitch or its speed. Experimental results indicate that the herein presented approach outperforms previous work which not use augmented data.
The problem of localizing objects exploiting RSSI signals has been tackled using both geometric and machine learning based methods. Solutions machine learning based have the advantage to better cope with noise, but require many radio signal observations associated to the correct position in the target space. This data collection and labeling process is not trivial and it typically requires building a grid of dense observations, which can be resource-intensive. To overcome this issue, we propose a pipeline which uses an autonomous robot to collect RSSI-image pairs and Structure from Motion to associate 2D positions to the RSSI values based on the inferred position of each image. This method, as we shown in the paper, allows to acquire large quantities of data in an inexpensive way. Using the collected data, we experiment with machine learning models based on RNNs and propose an optimized model composed of a set of LSTMs that specialize on the RSSI observations coming from different antennas. The proposed method shows promising results outperforming different baselines, suggesting that the proposed pipeline allowing to collect and automatically label observations is useful in real scenarios. Furthermore, to aid research in this area, we publicly release the collected dataset comprising 57158 RSSI observations paired with RGB images.
This paper proposes a lightweight methodology to attract users and increase views of videos through personalized artistic media i.e., static thumbnails and animated Graphics Interchange Format (GIF) images. The proposed method analyzes lightweight thumbnail containers (LTC) using computational resources of the client device to recognize personalized events from feature-length sports videos. In addition, instead of processing the entire video, small video segments are used in order to generate artistic media. This makes our approach more computationally efficient compared to existing methods that use the entire video data. Further, the proposed method retrieves and uses thumbnail containers and video segments, which reduces the required transmission bandwidth as well as the amount of locally stored data that are used during artistic media generation. After conducting experiments on the NVIDIA Jetson TX2, the computational complexity of our method was 3.78 times lower than that of the state-of-the-art method. To the best of our knowledge, this is the first technique that uses LTC to generate artistic media while providing lightweight and high-performance services on resource-constrained devices.
: Owing to the low equipment cost, the number of surveillance cameras installed has increased significantly; however, most of them are not being used effectively. These cameras can be used for various purposes, such as marketing, if behavior tracking is possible from the obtained images. Previously, we proposed a method of tracking by embedding information in “Artificial Fiber Pattern.” However, the body shape of the wearer and wrinkles of the clothes affect the accuracy of the results. To overcome this drawback, in this study, we combined PIFu HD, a technology that generates a full three-dimensional model from a single image of a person, with the modeling and calculation of the body shape of the subject to verify the conditions under which the body shape of the wearer and wrinkles in the clothes affect the accuracy. Consequently, we achieved precision improvement by removing data that met unsuitable conditions.
Early stopping is an extremely common tool to minimize overfitting, which would otherwise be a cause of poor generalization of the model to novel data. However, early stopping is a heuristic that, while effective, primarily relies on ad hoc parameters and metrics. Optimizing when to stop remains a challenge. In this paper, we suggest that for some biomedical applications, a natural dichotomy of invasive/non-invasive measurements of a biological system can be exploited to provide objective advice on early stopping. We discuss the conditions where invasive measurements of a biological process should provide better predictions than non-invasive measurements, or at best offer parity. Hence, if data from an invasive measurement is available locally, or from the literature, that information can be leveraged to know with high certainty whether a model of non-invasive data is overfitted. We present paired invasive/non-invasive cardiac and coronary artery measurements from two mouse strains, one of which spontaneously develops type 2 diabetes, posed as a classification problem. Examination of the various stopping rules shows that generalization is reduced with more training epochs and commonly applied stopping rules give widely different generalization error estimates. The use of an empirically derived training ceiling is demonstrated to be helpful as added information to leverage early stopping in order to reduce overfitting.
An accurate evaluation of computed tomography (CT) chest images is crucial in the early-stage detection of Covid-19. The accuracy of a diagnosis is determined by the imaging modality used and the images' consistency. This paper describes a gradient-based enhancement algorithm (GCE) for CT images that can increase the visibility of the infected region. Using a multi-scale dependent dark pass filter aims to increase contrast while preserving information and edge details of the infected area. Joint occurrence between the edge details and pixel intensities of the input image is calculated to construct a cumulative distribution function (CDF). To obtain the contrast improved image, the CDF is mapped to the uniform distribution. The GCE approach is tested on the CT Covid database, and performance metrics like the contrast improvement index (CII), discrete entropy (DE), and Kullback-Leibler distance (KL-Distance) are used to evaluate the results. Compared to other techniques available in the literature, the GCE approach produces the highest CII and DE values and has more uniformity. To check the suitability of the enhancement algorithm in terms of pre-processing, a pre-trained AlexNet is employed for the classification of Covid-19 images. The finding shows an improvement of 7% in classification accuracy after enhancing the Covid-19 images using the GCE technique. Copyright © 2021 by SCITEPRESS - Science and Technology Publications, Lda. All rights reserved
Recent developments in machine and deep learning have made it possible to expand the realms of traditional audio pattern recognition to real-time and practical applications. This work proposes a novel framework for robust bird species identification using the neural network (RoBINN) based on their unique vocal signatures. To make the network robust and efficient, data augmentation is performed to create synthetic training samples for bird species with less available recordings. Further, inherent properties of audio signals are suitably leveraged via effective speech recognition-based feature engineering techniques to develop an end-to-end convolutional neural network (CNN). Additionally, the proposed model architecture for the CNN framework employs residual learning and attention mechanism to generate attention-aware features, which enhances the overall accuracy of birdcall identification. The proposed architecture employs an exhaustive dataset with 21375 recordings corresponding to 264 bird species. Experimental results validate the proposed bird species classification technique in terms of accuracy, F1-score, and binary cross-entropy loss.
In this work, we analyze the performance of several tracking methods in the scenarios of low temporal sampling acquisition setup in fluorescence microscopy. Machine Learning methods were applied to classify and analyze the extracted trajectories of sperm samples and their motion parameters. The results were compared with the most widely used sperm motility classification methods. Analyzed image sequences include real sequences acquired by confocal fluorescence microscopy and synthetic sequences generated by in-house software. The complete framework runs as a standalone application and can be used with minimal training by users with no programming skills.
Automatic identification of heart irregularities based on the respective acoustic emissions is a relevant research field which receives ever-increasing attention over the last years. Devices such as digital stethoscope and smartphones can record the heartbeat sounds and are easily accessible, making this method more appealing. This paper presents different automatic procedures to classify heartbeat sounds coming from such devices into five different labels: normal, murmur, extra heart sound, extrasystole and artifact so that even people without medical knowledge can detect heart irregularities. The data used in this paper come from two different datasets. The first dataset is collected through an iPhone application whereas the second one is collected from a digital stethoscope. To be able to classify heartbeat sounds, time and frequency domain features are extracted and modeled by different machine learning algorithms, i.e. k-NN, random forest, SVM and ANNs. We report the achieved performances and a thorough comparison.
