Audio classification, including speech emotion, has been a topic of extensive research and applies to various virtual assistants and intelligent systems. Previous methods relied on handcrafted features such as spectrograms, but these features often have limitations due to their manual nature. Recently, hybrid models that use both end-to-end learning from raw audio with CNNs and Transformers have been developed to address this issue. However, when raw audio features are compressed through convolutional neural networks (CNNs), numerous channels are created, leading to redundancy or irrelevant information, while Transformers also have their limitations. Therefore, we propose Channel Attention Shuffle and Temporal Jigsaw (CAS-TJ) to generate more effective features and improve robustness. CAS divides channels into groups, multiplies them by attention weights, aggregates, and shuffles them. This process allows information to be exchanged among various channels, creating more discriminative channels. TJ generates frame patches of a specific size and uses mixing and matching during the learning process. This helps to better understand temporal relationships and detect discriminative patterns. Finally, we conduct experiments on the ESC-50 and Urban-8k datasets and find that the overall performance of CAS-TJ is better than the baseline models.

In this study, we propose a novel framework for time-series representation learning that integrates a learnable masking-augmentation strategy into a contrastive learning framework. Time-series data pose challenges due to their temporal dependencies and feature-extraction complexities. To address these challenges, we introduce a masking-based reconstruction approach within a contrastive learning context, aiming to enhance the model’s ability to learn discriminative temporal features. Our method leverages self-supervised learning to effectively capture both global and local patterns by strategically masking segments of the time-series data and reconstructing them, which aids in revealing nuanced temporal dependencies. We utilize learnable masking as a dynamic augmentation technique, which enables the model to optimize contextual relationships in the data and extract meaningful representations that are both context-aware and robust. Extensive experiments were conducted on multiple time-series datasets, including SleepEDF-78, 20, UCI-HAR, achieving improvements of 2%, 2.55%, and 3.89% each and similar performance on Epilepsy in accuracy over baseline methods. Our results show significant performance gains compared to existing methods, highlighting the potential of our framework to advance the field of time-series analysis by improving the quality of learned representations and enhancing downstream task performance.

In this paper, a deep learning-based model for 3D human motion generation from the text is proposed via gesture action classification and an autoregressive model. The model focuses on generating special gestures that express human thinking, such as waving and nodding. To achieve the goal, the proposed method predicts expression from the sentences using a text classification model based on a pretrained language model and generates gestures using the gate recurrent unit-based autoregressive model. Especially, we proposed the loss for the embedding space for restoring raw motions and generating intermediate motions well. Moreover, the novel data augmentation method and stop token are proposed to generate variable length motions. To evaluate the text classification model and 3D human motion generation model, a gesture action classification dataset and action-based gesture dataset are collected. With several experiments, the proposed method successfully generates perceptually natural and realistic 3D human motion from the text. Moreover, we verified the effectiveness of the proposed method using a public-available action recognition dataset to evaluate cross-dataset generalization performance.

This paper deals with a super-resolution that improves the resolution of side scan sonar images using learning-based compressive sensing. Learning-based compressive sensing combined with deep learning and compressive sensing takes a structure of a feed-forward network and parameters are set automatically through learning. In particular, we propose a method that can effectively extract additional information required in the super-resolution process through various initialization methods. Representative experimental results show that the proposed method provides improved performance in terms of Peak Signal-to-Noise Ratio (PSNR) and Structure Similarity Index Measure (SSIM) than conventional methods.

This paper proposes an unsupervised single-image Super-Resolution(SR) model using cycleGAN and domain discriminator to solve the problem of SR with unknown degradation using unpaired dataset. In previous approaches, paired dataset is required for training with assumed levels of image degradation. In real world SR applications, however, training sets are typically not of low and high resolution image pairs, but only low resolution images with unknown degradation are provided as inputs. To address the problem, we introduce a cycle-in-cycle GAN based unsupervised learning model using an unpaired dataset. In addition, we combine several losses attributed to image contents, such as pixel-wise loss, VGG feature loss and SSIM loss, for stable learning and performance improvement. We also propose a domain discriminator, which consists of noise discriminator, texture discriminator and color discriminator, to guide generated images to follow target domain distribution rather than source domain. We validate effectiveness of our model in quantitative and qualitative experiments using NTIRE2020 real-world SR challenge dataset.

Single image extreme Super Resolution (SR) is a difficult task as scale factor in the order of 10X or greater is typically attempted. For instance, in the case of 16x upscale of an image, a single pixel from a low resolution image gets expanded to a 16x16 image patch. Such attempts often result fuzzy quality and loss in details in reconstructed images. To handle these difficulties, we propose a network architecture composed of a series of connected blocks in recurrent and feedback fashions for enhanced SR reconstruction. By use of recurrent network, an SR image is refined over a sequence of enhancement stages in coarse to fine manner. Additionally, each stage involves back projection of SR image to LR images for continuously being refined during the sequence. According to the preliminary results of NTIRE 2020 Perceptual Extreme SR challenge, our team (KU_ISPLB) secured 6th place by PSNR and 7th place by SSIM among all participants.

This paper reviews the NTIRE 2020 challenge on perceptual extreme super-resolution with focus on proposed solutions and results. The challenge task was to super-resolve an input image with a magnification factor ×16 based on a set of prior examples of low and corresponding high resolution images. The goal is to obtain a network design capable to produce high resolution results with the best perceptual quality and similar to the ground truth. The track had 280 registered participants, and19 teams submitted the final results. They gauge the state-of-the-art in single image super-resolution.

Rising rate of vandalism against Automatic Teller Machines (ATMs) is a serious issue within banking industries, prompting needs of a technology to autonomously recognize such events. A vision based fusion method proposed here for classifying these incidents is rooted on visually recognizing heavy or sharp objects potentially used for detecting vandalism actions inferred from optical flow. The recognition performance has been improved chiefly by a novel employment of influence functions in selecting data points of each class useful in learning. We show that the tool recognition performance can be improved when the training data is selected from the ImageNet data set as guided by the influence function.

In this paper, a multi-modal classification is proposed for recognizing vandalism against Automatic Teller Machines (ATMs). The visual and textual information base model is developed here to identify external threats on ATMs. The model discriminates threatening behaviors from those that are benign in the image. It provides a level of confidence in the threat recognition by visual object classification coupled with word vector distance measure. To achieve our goal, real-time object detection based on a Region Convolutional Neural Network (R-CNN) first detects objects in the scene and word embedding technique allows to measure distance between the detected object label with predefined tools assumed to be used for vandalizing ATMs. Similarity measure from word embedding not only determines whether the scene may lead to any nefarious activities, but also would provide the level of confidence in occurrence of such incidents. From the experimental evaluation, it is shown that the method is effective and delivers a quantitative measure on decisions it makes.