Pedestrian re-identification is a technique to locate the same individual in data from different cameras. Currently, the method of multi-source domain adaptation for pedestrian re-identification is becoming popular. In this approach, the core challenge lies in the issue of distribution differences between domains. In addition, many scholars are also dedicated to solving the noise problem existing in pseudo labels. This paper takes a computational cost perspective and aims to solve the domain adaptation challenges of large model sizes and slow image processing speeds. We have performed a lightening process on the base model ViT. Firstly, a hybrid architecture, MixNet, is proposed, introducing the CNN basic module into ViT, extracting local features via shallow CNN, and capturing long-distance dependency patterns using deep Transformers, thereby reducing the parameters and computations. Additionally, an optimization is further conducted on the MixNet model based on the pruning algorithm. By using hierarchical pruning to compress multi-head attention and joint pruning for the mapping matrix in self-attention, the computational costs of the model have been effectively optimized. Relevant experiments carried out on several different tasks strongly verify that the proposed method has achieved significant optimization in computational costs.

As a pivotal computer vision technique, person re-identification (re-ID) assumes a paramount role in bolstering public security. During the process of computing feature similarities among person images, misaligned and occluded body parts may impede accurate identity retrieval. To mitigate these challenges, we introduce a Multi-Branch Feature Alignment Network (MBFA) comprising three distinct deep neural network branches. Primarily, the global feature branch is tailored to extract comprehensive features. Subsequently, the pose alignment branch is formulated to acquire segmented features via a specific feature-weighted fusion strategy. Finally, the semantic alignment branch is devised to derive high-order semantic features at a pixel level, enabling precise localization of visible parts in occluded pedestrians and focusing similarity computations on these regions. The integration of multi-scale feature information synergistically complements one another, resulting in feature alignment that augments the robustness and discrimination capabilities of the entire network. Consequently, MBFA adeptly mitigates the interferences caused by misalignment and occlusion. Across three prominent re-ID datasets and an occluded re-ID dataset, experimental results unequivocally affirm the superiority of our proposed methodology over existing state-of-the-art methods.

In the study of unsupervised domain adaptation for pedestrian re-identification, one prevalent approach during training on unlabeled target domains involves generating pseudo-labels. However, due to the inability to guarantee the correctness of pseudo-labels, they inevitably carry noise, subsequently affecting the network's training process. In this paper, we propose a novel approach that better constrains noise and demonstrates strong performance. First, building upon the DBSCAN clustering method, we generate soft labels for samples, thereby providing fine-grained category loss supervision. Specifically, we introduce an adaptive reverse cross-entropy loss to impose adaptive constraints on the pseudo-label generation process. We conduct various experiments on four datasets across three transfer tasks, and the results consistently highlight the superiority of our method, providing strong evidence for its effectiveness.

Speech emotion recognition has received growing attention to improving the user-friendly experience, but it is still a difficult challenge. That is mainly due to the problem of uneven distribution of emotional information in speech samples. The overall state label of a sample is not sufficient to fully describe each part of the sample. In other words, each part contributes differently to the overall recognition. In this paper, we build unidirectional and bidirectional temporal attention networks to enhance the ability to extract essential features, and build the key-frame loss function to make the attention weight closer to 0 and 1. The purpose of using the attention module and key-frame loss is to extract the information that is effective for classification and filter out the unimportant information. A large number of experiments were conducted on the IEMOCAP dataset to verify the effectiveness of the model and loss function. The experimental results show that under the combined effect of the attention model and the loss function, our method has achieved better performance than the existing methods, with a weighted accuracy of 59.15% and an unweighted accuracy of 60.87%.

Unsupervised multi-source domain adaptation facilitates the transfer of knowledge from multiple source domains, which possess labeled data, to an unlabeled target domain. Pedestrian re-identification is a technique for cross-camera pedestrian retrieval in surveillance data. The utilization of multiple source domains holds significant research implications, particularly in scenarios involving a substantial volume of data. Recently, efforts have been made to eliminate distributional differences between data from different domains. However, these approaches do not take into account the specificity of the target domain. Furthermore, when using graph convolutional networks for domain fusion, few studies have explored the utilization of deep correlations between nodes, which is crucial for node updates in GCNs. In this paper, we present a novel methodology that enhances domain fusion and showcases robust performance. In particular, we first propose a Domain Fusion Module based on the prioritization of target domains, which enables the fusion of domain information. Second, in order to mine the deep correlation between nodes, we process it by introducing multidimensional edge features. Our multiple experimental results using four datasets on three migration tasks demonstrate the superior performance of the DFTM, thus providing strong support for the effectiveness of our approach.

The critical goal of gait recognition is to acquire the inter-frame walking habit representation from the gait sequences. The relations between frames, however, have not received adequate attention in comparison to the intra-frame features. In this paper, motivated by optical flow, the bilateral motion-oriented block is proposed to explore motion description at the feature level. It can allow the classic 2D convolutional structure to have the capability to directly portray gait movement patterns while preventing costly computations on the estimation of optical flow. Based on such features, we develop a set of multi-scale temporal representations that force the motion context to be richly described at various levels of temporal resolution. Furthermore, the dynamic information is sensitive to inaccurate segmentation on the edge, so a correction block is devised to eliminate the segmentation noise of silhouettes for getting more precise gait modality. Subsequently, the temporal feature set and the spatial features are combined to comprehensively characterize gait processes. Extensive experiments are conducted on CASIA-B and OU-MVLP datasets, and the results achieve an outstanding identification performance, which has demonstrated the effectiveness of the proposed approach.

The research on multi-target recognition of mixed underwater acoustic signals is of great significance for military missions, ocean development, and navigation safety assurance. Due to the limited availability of information and the significant impact of the seawater medium and marine environmental noise on mono-channel underwater acoustic signals, achieving reliable and accurate multi-target recognition remains challenging. To overcome the challenge of reliability in multi-target recognition of mixed underwater acoustic signals, this paper focuses on investigating a deep learning-based method for recognizing and classifying underwater acoustic signals. A recurrent neural network fused with deep convolutional networks is proposed for multi-target recognition of mixed underwater acoustic signals. The deep convolutional network, DenseNet, is employed to extract frequency domain features, and an attention mechanism is introduced to capture the most salient features. Finally, hybrid underwater acoustic signal recognition is achieved through a recurrent neural network LSTM. The analysis of experimental results demonstrates that the DenseNet-LSTM model can enhance the accuracy of mixed underwater acoustic signal recognition based on frequency domain features. Furthermore, by incorporating the attention mechanism, the recognition rate is further improved.

Multi-source Unsupervised Domain Adaptation (MUDA) transfers knowledge from multiple source domains with labeled data to an unlabeled target domain. Recently, endeavours have been made in establishing connections among different domains to enable feature interaction. However, these approaches essentially enhance category information and thus lack the transfer of the domain-specific information. Moreover, few research has explored the connection between pseudo-label generation and the framework’s learning capabilities, crucial for ensuring robust MUDA. In this paper, we propose a novel framework, which significantly reduces the domain discrepancy and demonstrates new state-of-the-art performance. In particular, we first propose a Contrary Attention-based Domain Merge (CADM) module to enable the interaction among the features so as to achieve the mixture of domain-specific information instead of focusing on the category information. Secondly, to enable the network to correct the pseudo labels during training, we propose an adaptive and reverse cross-entropy loss, which can adaptively impose constraints on the pseudo-label generation process. We conduct experiments on four benchmark datasets, showing that our approach can efficiently fuse all domains for MUDA while showing much better performance than the prior methods.

Person re-identification plays a critical role in video surveillance and has a variety of applications. However, the body misalignment caused by detectors or pose changes sometimes makes it challenging to match features extracted from different images. To address the issues above, we propose a multi-branch attention and alignment network (MAAN). This approach is based on a deep network with three main branches. One branch is used for global feature representations. Another branch implements a multi-attention process based on keypoints, filters the practical information in the image, and then horizontally partitions the image to extract local features. For the last branch, we create a method based on part feature alignment. We obtain 17 keypoints from a pretrained pose estimation model, and nine local regions from the corresponding feature map are extracted for alignment. Experimental results on various popular datasets demonstrate that our method can produce competitive results under posture changes and body misalignment.

The separation of single-channel underwater acoustic signals is a challenging problem with practical significance. Few existing studies focus on the source separation problem with unknown numbers of signals, and how to evaluate the performances of the systems is not yet clear. We propose a solution with a fixed number of output channels to address these two problems, enabling it to avoid the dimensional disaster caused by the permutation problem induced by the alignment of outputs to targets. Specifically, we propose a two-step algorithm based on autoencoders and a new performance evaluation method for situations with mute channels. Experiments conducted on simulated mixtures of radiated ship noise show that the proposed solution can achieve similar separation performance to that attained with a known number of signals. The proposed algorithm achieved competitive performance as two algorithms developed for known numbers of signals, which is highly explainable and extensible and get the state of the art under this framework.

The radiated noise from ships is of great significance to target recognition, and several deep learning methods have been developed for the recognition of underwater acoustic signals. Previous studies have focused on single-target recognition, with relatively few reports on multitarget recognition. This paper proposes a deep learning-based single-channel multitarget underwater acoustic signal recognition method for an unknown number of targets in the specified category. The proposed method allows the two subproblems of recognizing the unique class and duplicate categories of multiple targets to be solved. These two tasks are essentially multilabel binary classification and multilabel multiple value classification, respectively. In this paper, we describe the use of real-valued and complex-valued ResNet and DenseNet convolutional networks to recognize synthetic mixed multitarget signals, which was superimposed from individual target signals. We compare the performance of various features, including the original audio signal, complex-valued short-time Fourier transform (STFT) spectrum, magnitude STFT spectrum, logarithmic mel spectrum, and mel frequency cepstral coefficients. The experimental results show that our method can effectively recognize synthetic multitarget ship signals when the magnitude STFT spectrum, complex-valued STFT spectrum, and log-mel spectrum are used as network inputs.

As an effective method to assist intelligent management of public areas, person re-identification (re-ID) technology has been developed rapidly in recent years. However, there are still some insurmountable obstacles in practical applications, among which misalignment caused by some factors is a challenging problem. Unlike previous approaches that only superficially mine pedestrian information to solve this problem, we propose an aligned person re-ID method based on human semantic parsing and message passing. Our method achieves pixel-level alignment through the incorporation of semantic parsing and also utilizes the results of semantic parsing. It constructs a graph neural network based on the structure of the human body to achieve information interaction between various part features. Additionally, various semantic features and a global feature are considered and used in the loss function for the ensemble of features, thereby ensuring the discrimination and robustness. Such ensemble learning allows our method to perform well not only for the unaligned case, but also have the ability to handle occlusion. Thus, the proposed SPMP method achieves better performance than most existing methods on multiple popular datasets.

The sound signal can be transmitted over a long distance in the water environment, but there is often interference from other signal sources in the real environment, which will seriously reduce the sensitivity and recognizability of the underwater acoustic signal. At this time, it is necessary to use underwater acoustic signal separation technology to separate mixed underwater acoustic signals. Due to the time sequence of the audio signal, the feature extraction ability of the separation model for the input sound signal largely determines the performance of the model. We propose a C-RNN network model that combines convolutional and recurrent neural network to achieve the improvement of separation performance. The advantages and disadvantages of separation based on time domain and frequency domain are compared, and a hybrid coding module is proposed to achieve a new state-of-the-art for underwater acoustic signal separation.

The separation of single-channel underwater acoustic signals is a challenging problem with practical significance. Few existing studies focus on the source separation problem with unknown numbers of signals, and how to evaluate the performance of the systems is not yet clear. In this paper, a deep learning-based simultaneous separating solution with a fixed number of output channels equal to the maximum number of possible targets is proposed to address these two problems. This solution avoids the dimensional disaster caused by the permutation problem induced by the alignment of outputs to targets. Specifically, we propose a two-step learning-based separation model with a separator-decoder structure. A performance evaluation method with two quantitative metrics of the separation system for situations with mute channels in the output channels that do not contain target signals is also proposed. Experiments conducted on simulated mixtures of radiated ship noise show that the proposed solution can achieve similar separation performance to that attained with a known number of signals. The proposed separation model with separator-decoder structure achieved competitive performance as two models developed for known numbers of signals, which is highly explainable and extensible and gets the state of the art under this framework.

Traditional emotion recognition technology often focuses on recognizing human biometrics such as facial expressions or body postures. However, psychological research shows that the context (contextual information) also plays an important role in perceiving the emotions of others. Existing research methods, that are based on contextual information, have heavily relied on the semantic features of images. They do not take into account the interrelationships between objects and fail to use external knowledge. Meanwhile, external knowledge is likely to be very helpful in perceiving emotion. In this paper, by incorporating external structured emotion commonsense knowledge, two methods are proposed for constructing emotion knowledge graphs based on the objective text of images, and a multi-modal emotion recognition model is designed. The model has three branches, one of which focuses on human biometrics, and another two branches employ emotion knowledge graphs to perceive emotion from contextual information. Before constructing the emotion knowledge graphs, we convert the visual content into the text information to obtain the prime and ample contextual information from the object, scene, and the relationship between the objects. This approach can reduce redundant and invalid information. After that, the structured emotion commonsense knowledge is integrated into the objective text in word sharing. A large-scale emotion knowledge graph based on all valid words (LEKG) and a small-scale emotion knowledge graph based on the document itself (TEKG) are constructed, respectively. We propose two fusion modules, one of which is attention-based, and the other is a deep reasoning module that incorporates interpersonal relation. We conduct extensive experiments on the benchmark dataset EMOTIC. The experimental results prove that our method is superior to the most advanced methods, and it has obvious advantages in global context-aware tasks.

As a study on the efficient usage of data, Multi-source Unsupervised Domain Adaptation transfers knowledge from multiple source domains with labeled data to an unlabeled target domain. However, the distribution discrepancy between different domains and the noisy pseudo-labels in the target domain both lead to performance bottlenecks of the Multi-source Unsupervised Domain Adaptation methods. In light of this, we propose an approach that integrates Attention-driven Domain fusion and Noise-Tolerant learning (ADNT) to address the two issues mentioned above. Firstly, we establish a contrary attention structure to perform message passing between features and to induce domain movement. Through this approach, the discriminability of the features can also be significantly improved while the domain discrepancy is reduced. Secondly, based on the characteristics of the unsupervised domain adaptation training, we design an Adaptive Reverse Cross Entropy loss, which can directly impose constraints on the generation of pseudo-labels. Finally, combining these two approaches, experimental results on several benchmarks further validate the effectiveness of our proposed ADNT and demonstrate superior performance over the state-of-the-art methods.

As an important part of intelligent surveillance systems, person re-identification (re-ID) has a wide range of application prospects in smart cities. However, due to occlusion, viewpoint variation, and background shift, the misalignment problem always decreases the re-ID systems’ effects. To solve this problem, a pose alignment network with information interaction (PAII) is proposed. This approach consists of three cascaded modules. First, guided by a pretrained pose estimator, the backbone with a dual attention block is used to obtain local features corresponding to different pose keypoints along with the global feature. Then, a pose alignment module is constructed to group these local features into different parts and fuse them with a hyperparameter λ , which provides the possibility to achieve semantic alignment. Finally, since different semantic features are extracted, an information interaction module consisting of graph attention layers is made to conduct message passing between different semantic features. All semantic features and the global feature are used to calculate the loss functions. Our approach considers multi-scale representations and information interaction of semantic features, which makes it more robust to misalignment problems. Thus, the proposed PAII method achieves better performance than most existing methods on multiple popular re-ID datasets.

Gait recognition is an emerging biometric recognition technology. Gait features have the advantages of non-contact, long collection distance and so on. It has received extensive attention from researchers in the field of biometric identification. We propose a novel model-based gait recognition method. Early methods were mainly based on appearance. Appearance-based features usually use gait contour maps as input. The gait contour map is easy to obtain and proved to be effective for recognition tasks. However, its appearance will be affected by the changes of clothing and carrying items. Contrast to the contour-based method is the model-based method. We use the human pose estimation algorithm to obtain 3D key points, use the key points coordinates as graph nodes feature to build a spatial-temporal graph, and use graph neural network to extract features for gait recognition tasks. This method is experimented on the large-scale dataset CSAIA-B dataset. The experimental results show that the proposed method can achieve advanced performance. It is also robust to covariate changes.

Facial action unit (AU) detection has been applied in a wild range of fields, and has attracted great attention over the last decades. Most existing methods employ the predefined regions of interest with same number and range for all samples. However, we find that the flexibility of predefined regions of interest is finite, as the occurrence of different AUs may not be simultaneous and their ranges change with intensity changes. In addition, many AU detection works try to independently design feature extraction modules and classifiers for each AU, which is of high computation cost and ignores the dependency among different AUs. In view of the limited flexibility of predefined regions of interest, we propose difference saliency maps that do not depend on facial landmarks. They are the spatial pixel-wise attentions, where each element represents the importance of the corresponding pixel on the entire image. Therefore, all the regions of interest can be irregular. In addition, in order to solve the problem of high computation cost, we combine group convolution with skip connection to propose a lightweight network that is more suitable for AU detection. All AUs share features and there is only one classifier, so the computation cost and the number of parameters are greatly reduced. In particular, the difference saliency maps and the global feature maps are combined to obtain the regional enhancement features. To maximize the enhancement effect, the down-sampled difference saliency maps are added to multiple blocks of the lightweight network. The enhanced global features are directly sent to the classifier for AU detection. By changing the number of neurons in the classifier, our framework can easily adapt to different datasets. Extensive experimental results show that the proposed framework soundly outperforms the classic deep learning method when evaluated on the DISFA+ and CK+ datasets. After adding the difference saliency maps, the detection result is better than the state-of-the-art AU detection methods. Further experiments demonstrate that our network is more efficient in using parameters, computation complexity and inference time.

针对步态周期检测直接影响到步态识别的计算量和精度的问题,本文基于深卷积神经网络实现步态周期检测,分别通过将步态序列根据周期性进行分类,和将步态序列拟合为正弦函数进行步态周期性检测的方法,对步态周期进行建模.其关键思想是根据步态周期的规律性,将步态起伏作为分类问题或一个正弦函数来建模,步态视频中的每个帧对应一个可以表示其周期特征的类别或函数值.而卷积神经网络被用于提取步态帧的周期性特征,以定位该帧在周期中的位置,得到分类或回归结果,最终实现步态周期检测.在CASIA-B数据集的不同视角下采用了多种网络结构进行了周期检测效果的验证.实验结果表明:该方法对步态周期性检测具有良好的精度和鲁棒性.