Communication is part of our everyday life and our ability to communicate can have a significant role in a variety of contexts in our personal, academic, and professional lives. For long, the characterization of what is a good communicator has been subject to research and debate by several areas, particularly in Education, with a focus on improving the performance of teachers. In this context, the literature suggests that the ability to communicate is not only defined by the verbal component, but also by a plethora of non-verbal contributions providing redundant or complementary information, and, sometimes, being the message itself. However, even though we can recognize a good or bad communicator, objectively, little is known about what aspects – and to what extent—define the quality of a presentation. The goal of this work is to create the grounds to support the study of the defining characteristics of a good communicator in a more systematic and objective form. To this end, we conceptualize and provide a first prototype for a computational approach to characterize the different elements that are involved in communication, from audiovisual data, illustrating the outcomes and applicability of the proposed methods on a video database of public speakers.
This short paper describes our contribution to the LNDb - Grand Challenge on automatic lung cancer patient management [1]. We only participated in Sub-Challenge A: Nodule Detection. The officially stated goal of this challenge is From chest CT scans, participants must detect pulmonary nodules. We developed a computer-aided detection (CAD) system for the identification of small pulmonary nodules in screening CT scans. The two main modules of our system consist of a CNN based nodule candidate detection, and a neural classifier for false positive reduction. The preliminary results obtained on the challenge database is discussed. In this work, we developed an Ensemble learning pipeline using state of the art convolutional neural networks (CNNs) as base detectors. In particular, we utilize the 3D versions of SE-ResNet18 and DPN68. Much like classical bagging, base learners were trained on 10 stratified data-set folds (the LUNA16 patient-level dataset splits) generated by bootstrapping both our training set (LUNA16) and the challenge provided training set. Furthermore, additional variation was introduced by using different CNN architectures. Particularly, we opted for an exhaustive search of the best detectors, consisting mostly of DPN68 [2] and SE-ResNet18 [3] architectures. We unfortunately joined the competition late, and we did not train our system on the corpus provided by the organizers and therefore we only run inference using our LIDC-IDRI trained model. We do realize this is not the best approach.
This paper proposes representing and detecting manufacturing defects at the micrometre scale using deep convolutional neural networks. The information theoretic notion of entropy is used to quantify the information gain or mutual information of filters throughout the network, where the deepest network layers are generally shown to exhibit the highest mutual information between filter responses and defects, and thus serve as the most discriminative features. Quantitative detection experiments based on the AlexNet architecture investigate a variety of design parameters pertaining to data preprocessing and network architecture, where the optimal architectures achieve an average accuracy of 98.54%. CNNs are relatively easy to perform and give impressive achievements in classification tasks. However, the informational complexity coming from the depth of networks represents a limit to improve their capabilities.
In this work, we have carried out a performance analysis of chess players comparing a standard ranking measure with a novel one proposed here. Using the idea of treating participants eye movements, when answering several on-screen valid chess questions of distinguished complexities, as high-dimensional spatial attention patterns we have shown that expertise is consistently associated with the ability to process visual information holistically using fewer fixations rather than locally focusing on individual pieces. These findings might disclose new insights for predicting chess skills.
Identification and extraction of individual modes in non-stationary multicomponent signals is a challenging task which is shared by several applications, like micro-doppler human gait analysis, surveillance or medical data analysis. State-of-the-art methods are not capable yet to correctly estimate individual modes if their instantaneous frequencies laws are not separable. The knowledge of time instants where modes interference occurs could represent a useful information to use in separation strategies. To this aim, a novel time-domain method that is capable of locating interferences is investigated in this paper. Its main property is the use of multiscale energy for selecting the best analysis scale without requiring either the use of time-frequency representations or imaging methods. The performance of the proposed method is evaluated through several numerical simulations and comparative studies with the state of the art Rényi entropy based method. Finally, an example concerning a potential application to simulated micro-doppler human gait data is provided.
Focal Cortical Dysplasia (FCD) is one of the most common causes of paediatric medically intractable focal epilepsy. In cases of medically resistant epilepsy, surgery is the best option to achieve a seizure-free condition. Pre-surgery lesion localization affects the surgery outcome. Lesion localization is done through examining the MRI for FCD features, but the MRI features of FCD can be subtle and may not be detected by visual inspection. Patients with epilepsy who have normal MRI are considered to have MRI-negative epilepsy. Recent advances in machine learning and deep learning hold the potential to improve the detection and localization of FCD without the need to conduct extensive pre-processing and FCD feature extraction. In this research, we apply Convolutional Neural Networks (CNNs) to classify FCD in children with focal epilepsy and localize the lesion. Two networks are presented here, the first network is applied on the whole-slice of the MR images, and the second network is taking smaller patches extracted from the slices of each MRI as input. The patch-wise model successfully classifies all healthy patients (13 out of 13), while 12 out of 13 cases are correctly identified by the whole-slice model. Using the patch-wise model, we identified the lesion in 17 out of 17 MR-positive subjects with coverage of 85% and for MR-negative subjects, we identify 11 out of 13 FCD subjects with lesion coverage of 66%. The findings indicate that convolutional neural network is a promising tool to objectively identify subtle lesions such as FCD in children with focal epilepsy.
Studying human motion from images and videos has turned into an interesting topic of research given the recent advances in computer vision and deep learning algorithms. When focusing on the automatic procedure of tracking physical exercises, cameras can be used for full human pose estimation in relation to worn sensors. In this work, we propose a method for workout repetition counting and validation based on a set of skeleton-based and deep semantic features that are obtained from a 2D human pose estimation network. Given that some of the individuals’ body parts might be occluded throughout physical exercises, we also perform a multi-view analysis on supporting cameras to improve our recognition rates. Nevertheless, the obtained results for a single-view approach show that we are able to count valid repetitions with over 90% precision scores for 4 out of 5 considered exercises, while recognizing more than 50% of the invalid ones.
Lung cancer is the most common form of cancer in the world affecting millions yearly. Early detection and treatment is critical in driving down mortality rates for this disease. A traditional form of early detection involves radiologists manually screening low dose computed tomography scans which can be tedious and time consuming. We propose an automatic system of deep learning methods for the detection, segmentation, and classification of pulmonary nodules. The system is composed of 3D convolutional neural networks based on VGG and U-Net architectures. Chest scans are received as input and, through a series of patch-wise predictions, patient follow-up recommendations are predicted based on the 2017 Fleischner society pulmonary nodule guidelines. The system was developed as part of the LNDb challenge and participated in the main challenge as well as all sub-challenges. While the proposed method struggled with false positives for the detection task and a class imbalance for the texture characterization task, it presents a baseline for future work.
Gaussian Mixture Models are widely used nowadays, thanks to the simplicity and efficiency of the Expectation-Maximization algorithm. However, determining the optimal number of components is tricky and, in the context of data partitioning, may differ from the actual number of clusters. We propose to apply a post-processing step by means of Spectral Clustering: it allows a clever merging of similar Gaussians thanks to the Bhattacharyya distance so that clusters of any shape are automatically discovered. The proposed method shows a significant improvement compared to the classical Gaussian Mixture clustering approach and promising results against well-known partitioning algorithms with respect to the number of parameters.
Studying human motion from images and videos has turned into an interesting topic of research given the recent advances in computer vision and deep learning algorithms. When focusing on the automatic procedure of tracking physical exercises, cameras can be used for full human pose estimation in relation to worn sensors. In this work, we propose a method for workout repetition counting and validation based on a set of skeleton-based and deep semantic features that are obtained from a 2D human pose estimation network. Given that some of the individuals’ body parts might be occluded throughout physical exercises, we also perform a multi-view analysis on supporting cameras to improve our recognition rates. Nevertheless, the obtained results for a single-view approach show that we are able to count valid repetitions with over 90% precision scores for 4 out of 5 considered exercises, while recognizing more than 50% of the invalid ones.
We present a study of minimal-case motion estimation with affine correspondences and introduce a new solution for multi-camera motion estimation with affine correspondences. Ego-motion estimation using one or more cameras is a well-studied topic with applications in 3D reconstruction and mobile robotics. Most feature-based motion estimation techniques use point correspondences. Recently, several researchers have developed novel epipolar constraints using affine correspondences. In this paper, we extend the epipolar constraint on affine correspondences to the multi-camera setting and develop and evaluate a novel minimal solver using this new constraint. Our solver uses six affine correspondences in the minimal case, which is a significant improvement over the point-based version that requires seventeen point correspondences. Experiments on synthetic and real data show that, in comparison to the point-based solver, our affine solver effectively reduces the number of RANSAC iterations needed for motion estimation while maintaining comparable accuracy.
The optic disc (OD) and the fovea are relevant landmarks in fundus images. Their localization and segmentation can facilitate the detection of some retinal lesions and the assessment of their importance to the severity and progression of several eye disorders. Distinct methodologies have been developed for detecting these structures, mainly based on color and vascular information. The methodology herein described combines the entropy of the vessel directions with the image intensities for finding the OD center and uses a sliding band filter for segmenting the OD. The fovea center corresponds to the darkest point inside a region defined from the OD position and radius. Both the Messidor and the IDRiD datasets are used for evaluating the performance of the developed methods. In the first one, a success rate of 99.56% and 100.00% are achieved for OD and fovea localization. Regarding the OD segmentation, the mean Jaccard index and Dice’s coefficient obtained are 0.87 and 0.94, respectively. The proposed methods are also amongst the top-3 performing solutions submitted to the IDRiD online challenge.
Recent years have witnessed a growing popularity of 4K or ultra high definition (UHD) content. However, the acquisition, production, post-production, and distribution pipelines of such content often go through stages where the actual video resolution goes below 4K/UHD level and is then upscaled to 4K/UHD resolution at later stages. As a result, the claimed 4K content in the real world often drops below the intended 4K quality, while final consumers are not well informed about such quality degradation. Here, we present our recent research progress on automatic image resolution assessment methods that determine whether a given image has true 4K resolution or not. Specifically, we developed a largest of its kind database of more than 10,000 true and fake 4K/UHD images with ground-truth labels. We have also made some initial attempts on constructing edge feature, Fourier transform feature, and deep learning based methods for the classification task. We believe that the built database and the attempted methods will help accelerate the research progress on automatic image resolution assessment.
The automated analysis of Computed Tomography scans of the lung holds great potential to enhance current clinical workflows for the screening of lung cancer. Among the tasks of interest in such analysis this paper is concerned with the segmentation of lung nodules and their characterization in terms of texture. This paper describes our solution for these two problems in the context of the LNdB challenge, held jointly with ICIAR 2020. We propose a) the optimization of a standard 2D Residual Network, but with a regularization technique adapted for the particular problem of texture classification, and b) a 3D U-Net architecture endowed with residual connections within each block and also connecting the downsampling and the upsampling paths. Cross-validation results indicate that our approach is specially effective for the task of texture classification. In the test set withheld by the organization, the presented method ranked 4th in texture classification and 3rd in the nodule segmentation tasks. Code to reproduce our results is made available at http://www.github.com/agaldran/lndb .
Breast Ultrasound has long been used to support diagnostic and exploratory procedures concerning breast cancer, with an interesting success rate, specially when complemented with other radiology information. This usability can further enhance visualization tasks during pre-treatment clinical analysis by coupling the B-Mode images to 3D space, as found in Magnetic Resonance Imaging (MRI) per instance. In fact, Lesions in B-mode are visible and present high detail when comparing with other 3D sequences. This coupling, however, would be largely benefited from the ability to match the various structures present in the B-Mode, apart from the broadly studied lesion. In this work we focus on structures such as skin, subcutaneous fat, mammary gland and thoracic region. We provide a preliminary insight to several structure segmentation approaches in the hopes of obtaining a functional and dependable pipeline for delineating these potential reference regions that will assist in multi-modal radiological data alignment. For this, we experiment with pre-processing stages that include Anisotropic Diffusion guided by Log-Gabor filters (ADLG) and main segmentation steps using K-Means, Meanshift and Watershed. Among the pipeline configurations tested, the best results were found using the ADLG filter that ran for 50 iterations and H-Maxima suppression of 20
Two-Stream framework with spatial information and optical flow information have reached the great performance for action recognition task in video. The optical flow information captures the low-level motion characteristics via a fixed quantity of consecutive video frames, which however contains noise information and is incompetent to characterize different actions with varying posture and duration. Usually ten frames before and after a frame are used as optical flow information, which may be too long or too short to capture the useful motion features for different actions. Moreover, the cost of calculating optical flow information from several consecutive video frames is high. To solve these issues, we propose a novel framework to recognize actions by capturing a high-level motion feature, human pose estimation, instead of the optical flow. Our framework uses 2D human pose estimation as the motion feature, and fuses it with the spatial information using attention mechanisms. We handle extensive experiments on two excellent and challenging datasets of realistic human action, HMDB-51 and UCF-101. The experimental results illustrate that our two-stream framework outperforms state-of-the-art approaches in terms of accuracy.
This research introduces a novel multiple object tracking algorithm called SMAT (Smart Multiple Affinity Metric Tracking) that works as an online tracking-by-detection approach. The use of various characteristics from observation is established as a critical factor for improving tracking performance. By using the position, motion, appearance, and a correction component, our approach achieves an accuracy comparable to state of the art trackers. We use the optical flow to track the motion of the objects, we show that tracking accuracy can be improved by using a neural network to select key points to be tracked by the optical flow. The proposed algorithm is evaluated by using the KITTI Tracking Benchmark for the class CAR.
Anomaly detection is a pattern recognition task that aims at distinguishing abnormal patterns from normal ones. In this paper, we propose a convolutional auto-encoder based model to detect anomaly images by producing a sparse representation in the latent space. The proposed approach is able to represent the normal images using sparse encoding and the encoding can be well reconstructed by the decoder. However, the learned convolutional filters are not able to represent the abnormal images in a sparse way. Therefore, the decoder can not reconstruct the abnormal images with high quality. By assessing the reconstruction performance, we can distinguish the abnormal images from the normal ones. The experimental results show the superiority of our proposed model over other variants of auto-encoder based anomaly detection models in terms of AUC. In addition, the results show that the sparse representation based anomaly detection method could apply to different scenarios.
The Distance-Regularized Level Set Evolution (DRLSE) algorithm solves many problems that plague the class of Level Set algorithms, but has a significant computational cost and is sensitive to its many parameters. Configuring these parameters is a time-intensive trial-and-error task that limits the usability of the algorithm. This is especially true in the field of Medical Imaging, where it would be otherwise highly suitable. The aim of this work is to develop a parallel implementation of the algorithm using the Compute-Unified Device Architecture (CUDA) for Graphics Processing Units (GPU), which would reduce the computational cost of the algorithm, bringing it to the interactive regime. This would lessen the burden of configuring its parameters and broaden its application. Using consumer-grade, hardware, we observed performance gains between roughly 800% and 1700% when comparing against a purely serial C++ implementation we developed, and gains between roughly 180% and 500%, when comparing against the MATLAB reference implementation of DRLSE, both depending on input image resolution.
Sexual dimorphism in the human retina has recently been connected to gonadal hormones. In the study herein presented, texture analysis was applied to computed mean value fundus (MVF) images from optical coherence tomography data of female and male healthy adult controls. Two separate age-group analyses that excluded the probable perimenopause period of the women in the present study were performed, using a modified MVF image computation method that further highlights texture differences present in the retina. While distinct texture characteristics were found between premenopausal females and age-matched males, these differences almost disappeared in the older groups (postmenopausal women vs age-matched men), suggesting that sex-related texture differences in the retina can be correlated to the hormonal changes that women go through during the menopausal transition. These findings suggest that texture-based metrics may be used as biomarkers of physiology and pathophysiology of the retina and the central nervous system.
