Gait recognition is a promising biometric recognition technique that uses walking patterns for authentication. It is known that motion representation stands as a long-term challenge for the task of gait recognition. To address this issue, most recent methods have conducted intensive studies on multi-scale temporal modeling and fine-grained spatial information aggregation, which generally characterize motion information in an implicit manner. How to quantitatively represent the change process of human body contours and dynamic motion differences remains an open problem. In this paper, we propose a novel motion representation for gait recognition stemming from deformation fields produced by the classical non-rigid point-set registration. Deformation fields are seamlessly integrated into the proposed gait recognition framework GaitDFG to yield discriminative motion features. GaitDFG mainly consists of three key components including Silhouette Feature extraction Network (SFNet), Deformation field Feature extraction Network (DFNet), and Knowledge Distillation Module (KDM). SFNet is employed to capture dynamic appearance motion difference and aggregate contextual information between neighboring frames from the input silhouette sequence. Furthermore, a multi-scale spatial perception module in DFNet is developed to extract the motion features of deformation fields to explore more useful motion clues. Besides, since real-time computation of deformation fields is infeasible in real-world scenarios, we design a deformation field feature simulation module to mimic the features of deformation fields for inference, which is learned from DFNet via knowledge distillation. Consequently, in the inference stage, we can fuse silhouette features and simulated deformation field features to perform gait recognition. Extensive experiments are conducted to validate the effectiveness of GaitDFG, demonstrating state-of-the-art performance on the standard gait recognition benchmarks, including CASIA-B (in-the-lab), GREW (in-the-wild) and CCPG (cloth-changing).

Gait recognition aims to identify target subjects across non-overlapping camera viewpoints according to their unique walking patterns. Motion representation is the core task in constructing an applicable gait recognition system, which is required to characterize fine-grained dynamic posture changes. In current gait recognition research, multi-scale temporal modeling in conjunction with spatial representation learning is the mainstream line. However, such ideas describe walking patterns in an implicit manner, which often leads to missing important motion information. To address these challenges, we model continuous human body movement as motion deformation field sequences with more physical interpretability. And the learned deformation fields are seamlessly integrated into the proposed gait recognition framework GaitMDF. Specifically, we first learn the multi-scale deformation fields from silhouettes using the designed Deformation Field Generation Network (DFGNet) in a self-supervised manner. Then, we develop two powerful feature extraction network, i.e., Silhouette Feature Extractor (SFE) and Deformation Field Feature Extractor (DFFE), for the silhouette and deformation field sequences to obtain discriminative spatial-temporal representations. Furthermore, a two-stage knowledge distillation strategy is developed to transfer the motion features learned from DFFE to the mimetic deformation field features. By applying this strategy, we can not only preserve the motion information of the deformation fields but also significantly reduce computational cost in inference with no need for DFGNet and DFFE. Finally, the silhouette and the mimetic deformation field features are fused for identity recognition. Extensive experiments on three popular gait datasets demonstrate the effectiveness and superiority of the proposed method.

As one of the classical strategies for positive-unlabeled (PU) learning, the cost-sensitive methods achieve binary classification by minimizing the overall risk. To achieve this goal, the class prior is usually exploited to guide model learning in many state-of-the-art cost-sensitive approaches. However, how to effectively mine the underlying supervision information using the class prior remains a major challenging problem. It is known that the distribution of positive and negative samples is fixed in unlabeled samples when the class prior is available. Based on this knowledge, we derive a novel supervision formulation from a risk perspective: if the class prior is known, the ratio between the positive risk distribution of negative samples and the negative risk distribution of positive samples converges to a fixed value in unlabeled samples. By fully exploiting this brand-new constraint, we construct a consistent risk estimator to impose consistency between the negative expected risk of the positive samples and the positive expected risk of the negative samples. Moreover, we introduce Mixup regularization to mitigate the confirmation bias of the learned model, and meanwhile employ the entropy minimization technique to improve the separability of samples, which can further enhance classification performance. Experimental results on four benchmark datasets demonstrate the advantage of our model over several baseline methods. The code is available at https://github.com/yunbao520/cPU.

Infrared small-target detection (IRSTD) has achieved considerable progress with the adoption of deep learning approaches. However, feature coupling during downsampling and modeling target saliency stands as a long-term challenge for this task. To address the aforementioned challenges, we propose a novel framework, HaarTransNet, that leverages frequency-domain decoupling and target-background relative saliency modeling to achieve high-precision detection. The proposed approach consists of two key components. The Haar decoupled downsampling block (HDDB) decouples input features along the channel dimension via the Haar wavelet transform (WT). By explicitly separating high-frequency feature components from low-frequency feature components, HDDB successfully avoids feature confusion and alleviates small-target attenuation during downsampling. The Haar channel cross Transformer (HCCT) introduces a high-dimensional, suppressed channel cross-attention mechanism to model cross-level global correlations, thereby bridging the semantic gap between the encoder and the decoder. Within HCCT, the low-frequency background cue guides attention to dynamically model the relative salience of targets in complex backgrounds, consequently enhancing the discrimination of high-frequency target features and suppressing background interference. Experimental results on two public benchmarks, NUDT-SIRST and IRSTD-1k, as well as a synthetic dataset with challenging scenes, demonstrate the superior performance and robustness of HaarTransNet. Our code will be made public at https://github.com/Big-Orange-Tiger/HaarTransNet

Gait recognition aims to learn the unique walking patterns of different subjects for identity retrieval. Most methods focus on exploiting robust spatio-temporal representations via global-local feature learning or multi-scale temporal modeling. However, they may neglect the dynamic posture change within consecutive frames. In addition, few methods explore useful motion cues by data self-mining manner. In this paper, we propose a novel Motion Pattern Augmentation assisted gait recognition framework named GaitMPA, which explores diverse behavior characteristics from the augmented movement sequences. GaitMPA consists of three components: Motion perception and Fine-grained variation extraction Network (MFNet), Motion Pattern Augmentation (MPA), and Multi-stage Feature Aggregation (MFA). Specifically, we present MFNet to capture dynamic motion difference between neighboring frames by Motion Perception Module (MPM), and extract multi-grained body representation via Fine-grained Variation Extractor (FVE). In MPA, we transform raw sequences into four novel motion patterns to provide distinctive movement traits. Furthermore, MFA is designed to merge the multi-source features of the raw and augmented sequences, and perform multi-stage motion information aggregation. The outputs of MFNet and MFA are fused for gait recognition. Experimental results demonstrate the effectiveness of GaitMPA on five public datasets, including the CASIA-B (in-the-lab), OU-MVLP (in-the-lab), CCPG (cloth-changing), GREW (in-the-wild), and Gait3D (in-the-wild).

As a periodic movement, gait contains informative biometric traits formed by individual body structures, motion patterns, and behavioral habits. Previous gait recognition methods mainly focus on mining the appearance cues from gait sequences, while neglecting the dynamic motion characteristics. Motion cues are important complementary information for generating high-quality gait representations that can help models accurately recognize individuals. In this article, we propose a novel gait recognition framework named GaitDS to model dynamic motion information and construct salient gait representations. Specifically, we develop a motion information perception module that can directly represent dynamic regions during walking and extract fine-grained motion features based on the appearance of body parts over time. In addition, since some frames in gait sequences share partial similarities, we present saliency identity representation learning to focus on key frames along the temporal dimension, and integrate salient identity features to enhance sequence-level representations. Furthermore, a channel enhanced module is designed to generate more discriminative gait representations, where motion and temporal salient features can be complemented with global representations. Compared with existing state-of-the-art methods, our model achieves superior average rank-1 recognition accuracy on three benchmark datasets, i.e., 93.7% on CASIA-B, 92.4% on OU-MVLP, and 50.7% on Gait3D.

Knee-less bipedal robots like SLIDER have the advantage of ultra-lightweight legs and improved walking energy efficiency compared to traditional humanoid robots. In this paper, we firstly introduce an improved hardware design of the SLIDER bipedal robot with new line-feet and more optimized mass distribution that enables higher locomotion speeds. Secondly, we propose an extended Hybrid Zero Dynamics (eHZD) method, which can be applied to prismatic joint robots like SLIDER. The eHZD method is then used to generate a library of gaits with varying reference velocities in an offline way. Thirdly, a Guided Deep Reinforcement Learning (DRL) algorithm is proposed to use the pre-generated library to create walking control policies in real-time. This approach allows us to combine the advantages of both HZD (for generating stable gaits with a full-dynamics model) and DRL (for real-time adaptive gait generation). The experimental results show that this approach achieves 150% higher walking velocity than the previous MPC-based approach.

Facial Expression Recognition (FER) with occlusion presents a challenging task in computer vision because facial occlusions result in poor visual data features. Recently, the region attention technique has been introduced to address this problem by researchers, which make the model perceive occluded regions of the face and prioritize the most discriminative non-occluded regions. However, in real-world scenarios, facial images are influenced by various factors, including hair, masks and sunglasses, making it difficult to extract high-quality features from these occluded facial images. This inevitably limits the effectiveness of attention mechanisms. In this paper, we observe a correlation in facial emotion features from the same image, both with and without occlusion. This correlation contributes to addressing the issue of facial occlusions. To this end, we propose a Information Gap based Knowledge Distillation (IGKD) to explore the latent relationship. Specifically, our approach involves feeding non-occluded and masked images into separate teacher and student networks. Due to the incomplete emotion information in the masked images, there exists an information gap between the teacher and student networks. During training, we aim to minimize this gap to enable the student network to learn this relationship. To enhance the teacher's guidance, we introduce a joint learning strategy where the teacher conducts knowledge distillation on the student during the training of the teacher. Additionally, we introduce two novel constraints, called knowledge learn and knowledge feedback loss, to supervise and optimize both the teacher and student networks. The reported experimental results show that IGKD outperforms other algorithms on four benchmark datasets. Specifically, our IGKD achieves 87.57% on Occlusion-RAF-DB, 87.33% on Occlusion-FERPlus, 64.86% on Occlusion-AffectNet, and 73.25% on FED-RO, clearly demonstrating its effectiveness and robustness. Source code is released at: https://github.com/Lzh-Code-lab/IGKD.

By decoupling the software function on hardware devices, Network Function Virtualization(NFV) provides a new service architecture named Service Function Chain(SFC), which combines multiple Virtual Network Functions(VNFs) in a specific order. In order to reduce network resources consumption and improve the resource utilization, VNF sharing provides an effective solution for this requirement. However, traditional sharing methods lack a dynamic processing mechanism to select the deployment and shared node location according to the network state dynamically. Moreover, how to further optimize the utilization of network resources is challenging. This paper proposed a VNF sharing evaluation mechanism to evaluate and decide whether to share a VNF, then a node priority calculation mechanism was designed and mapped on node selection probability, which can select appropriate VNF to deploy or share VNF according to network state and resource requirements of SFC, finally, a reinforcement learning approach was utilized to update the selection probability of nodes and complete the VNF sharing process in air-ground network. The experimental results indicate that compared with other five benchmark algorithms, the proposed algorithm can reduce the transmission delay effectively, at the same time, it can improve node and link load resource utilization and acceptance rate of SFC after the VNF sharing.

Deep supervised hashing is more popular among researchers due to its satisfactory computational efficiency and retrieval performance. Most existing models learn hash codes for data by constructing inter-sample pair-wise or triplet losses, allowing for consideration of the topological information from the label space. However, the topological relationships among samples in the feature space are not fully explored, which may result in less discriminative hash codes. To address this issue, we propose a novel graph hashing network (GHash) for image retrieval. Our GHash explores positional relationships among samples under a large receptive field through alternating updates of graph nodes and edges, generating high-quality image descriptors based on optimized positional relationships and neighborhood information. Subsequently, graph-level descriptors are mapped into highly discriminative hash codes. Additionally, we introduce an extra classification loss to enhance the accuracy of the topological relationships among samples in the graph by supervising the learning of edge features. Finally, we conduct extensive comparison and ablation experiments on three benchmark datasets, with results demonstrating that our method achieves superior retrieval performance compared to state-of-the-art deep hashing methods.

Wheat lodging is a usual agricultural disaster in wheat growth. It reduces the grain yield and harvesting efficiency. Existing segmentation methods cannot achieve satisfactory performance and trade-offs between accuracy, inference time, and lightweight when facing the challenge of multiple lodging scenes. Therefore, developing an innovative segmentation algorithm that is real-time and low-complexity to identify lodging situations is of great value for improving agricultural production. To achieve these goals, we propose a lightweight and efficient lodging semantic segmentation model, WLUSNet, to separate the lodging area of unmanned aerial vehicle (UAV) images. Inspired by the mixed depth-wise grouping convolution (MC) and the channel feature pyramid (CFP) modules, a multiscale backbone (MC-CFP) is designed to reduce information loss in feature extraction. Then, drawing on the characteristics of MC and the channel attention (CA) mechanism, a space pyramid module (MC-SP) is designed to enhance feature representation by obtaining the global information on the channel feature and the local information on the space feature. To reconstruct a high-resolution feature map, a feature fusion module (EDFF) between the shallow and deep features is introduced to improve segmentation accuracy. The comprehensive experimental results demonstrate that WLUSNet performs excellently well compared with 11 other state-of-the-art (SOTA) segmentation algorithms. WLUSNet achieves a mean intersection over union (mIoU) of 86.9, a mean pixel accuracy (mPA) of 93.26, a model size of 4.1 M, and an inference speed of 26.94 FPS on the self-built UAV remote sensing dataset in this paper. The generation experiment indicates that WLUSNet has the potential to segment other lodging crops, and can provide technical support for segmentation tasks in crop lodging.

The abstract Deep learning-based image recognition methods have been adopted to recognize tea leaf diseases, which can help people make appropriate disease prevention and control plans. To achieve satisfactory recognition accuracy, most existing methods rely on a fully supervised learning paradigm, where all training data are assigned their own labels. Unfortunately, the annotation process for them is usually time-consuming. To address this issue, this article proposes a semi-supervised learning approach for recognizing tea leaf diseases where both labeled and unlabeled data are employed to train our model. In our method, both the teacher network and the student network are composed of segmentation-guided classification modules (SGCM). SGCM includes classification and segmentation branches that acquire image features from a common encoder module and perform classification and segmentation tasks separately. The classification branch receives guidance from the segmentation branch and constrains its results to improve recognition by combining cross-entropy and IoU losses. The teacher network is initially trained using annotated samples and generates pseudo-labels for unlabeled ones. We constructed a confidence calculation function using the maximum confidence of the classification prediction results and the entropy of the probability distribution to screen for high-confidence pseudo-labels. Then, a student network is trained using data with real labels or pseudo-labels. Moreover, the exponential moving average (EMA) strategy is adopted to continuously update weights of the teacher network along with the training of the student network. In order to test the effectiveness of the model, relevant experiments were conducted on self-built tea leaf disease data. Experimental results demonstrate the effectiveness of our method for identifying tea leaf diseases, achieving 98.13% recognition accuracy with only 50% of the training data annotated.

Gait recognition is a promising biometric technology that aims to identify the target subject via walking pattern. Most existing appearance-based methods focus on learning discriminative spatio-temporal representations from gait silhouettes. However, these methods pay less attention to probing the causality between identity factors and identity labels, which often mislead the model to learn gait representations that are susceptible to identity-irrelevant factors. In this paper, we attribute the cause that leads to the decline of model generalization under different external conditions to identity-irrelevant factors. We formulate the causalities among the identity factors, identity-irrelevant factors, and identity labels as a structural causal model (SCM). We accordingly propose a novel gait recognition framework named GaitSCM to learn covariate invariant gait representations, which is mainly composed of three components, including feature extraction module, feature disentanglement module, and backdoor adjustment. Specifically, we design a feature extractor with regard to the movement patterns of different body parts to learn fine-grained gait motion features, and then present a two-branch feature decoupling module to disentangle identity features and identity-irrelevant features with the aid of the classification confusion loss. To relieve the negative effect of identity-irrelevant factors, we develop a backdoor adjustment strategy to eliminate spurious associations between identity and identity-irrelevant features, which further facilitates the proposed framework to generate more powerful identity representations. Extensive experiments conducted on two public datasets validate the effectiveness of our method. The average Rank-1 can reach 93.2% and 90.4% on CASIA-B and OU-MVLP datasets, respectively, which verifies the superiority of GaitSCM. Source code is released at: https://github.com/HuoweiCode/GaitSCM.

Multi-dimensional classification (MDC) assumes that each instance has multiple heterogeneous class spaces simultaneously, and each class variable describes the semantic information of instances from a specific dimension. Recent studies have proven that encoding heterogeneous class spaces into a special logical-label space and employing the label enhancement technique to learn latent real-number labels (i.e., label distributions) of instances is an effective strategy for MDC. However, the adopted label enhancement methods can result that data whose features are quite different to each other have similar label distributions. To tackle this problem, we propose a novel probability-based label enhancement approach for MDC. Specifically, manifold structures of the feature and label distribution spaces are transformed into two different probability distributions, and we expect them to be close. Subsequently, it makes label distributions of samples whose features have large differences be more differentiated. Moreover, the logical-label mapping and reconstruction terms are designed to preserve the intrinsic information from the logical-label space. Besides, an improved multi-output support vector regression is developed as the prediction model, where we introduce mean squared error to reduce the risk of model underfitting. Experimental results on ten benchmark datasets clearly validate the superiority of our method over state-of-the-art MDC baselines.

Planning multi-contact motions in a receding horizon fashion requires a value function to guide the planning with respect to the future, e.g., building momentum to traverse large obstacles. Traditionally, the value function is approximated by computing trajectories in a prediction horizon (never executed) that foresees the future beyond the execution horizon. However, given the non-convex dynamics of multi-contact motions, this approach is computationally expensive. To enable online Receding Horizon Planning (RHP) of multi-contact motions, we find efficient approximations of the value function. Specifically, we propose a trajectory-based and a learning-based approach. In the former, namely RHP with Multiple Levels of Model Fidelity, we approximate the value function by computing the prediction horizon with a convex relaxed model. In the latter, namely Locally-Guided RHP, we learn an oracle to predict local objectives for locomotion tasks, and we use these local objectives to construct local value functions for guiding a short-horizon RHP. We evaluate both approaches in simulation by planning centroidal trajectories of a humanoid robot walking on moderate slopes, and on large slopes where the robot cannot maintain static balance. Our results show that locally-guided RHP achieves the best computation efficiency (95%-98.6% cycles converge online). This computation advantage enables us to demonstrate online receding horizon planning of our real-world humanoid robot Talos walking in dynamic environments that change on-the-fly.

This article proposes an efficient 2D-FFT method based on the digital signal processing (DSP) platform, suitable for frequency-modulated continuous-wave (FMCW) radar systems. Given the real-time detection and imaging requirements of FMCW radar, enhancing system efficiency is of paramount importance. As a crucial step in fast algorithms for FMCW radar imaging, improving the calculation speed of 2D-FFT can significantly optimize system performance. As 2D-FFT is a data-intensive application with strided memory access patterns, traditional schemes cannot fully exploit the computing and storage performance of the DSP processor. Moreover, instruction and data parallelization often lead to memory access conflicts. This article introduces a multistage merged and conflict-free modulo addressing scheme called the CF2D-FFT algorithm, which reduces memory access operations and effectively resolves parallel memory conflicts. The proposed scheme is implemented and optimized on the HXDSP104x DSP platform, and the enhanced algorithm achieves a 1.24x improvement over the theoretical runtime. The performance improvement of the proposed algorithm was validated through comparative experiments and imaging experiments.

Image classification is a critical task in the field of computer vision, and its importance has significantly increased over the past few years. Machine learning and deep learning techniques have demonstrated immense potential in this field. However, traditional image classification models require a vast amount of training data, which can be challenging and expensive to obtain. To overcome this limitation, researchers are turning to few-shot learning, which aims to classify images with limited training samples. This paper presents a detailed analysis of the field of image classification using few-shot learning. First, it investigates the use of data augmentation, transfer learning, and meta-learning methods in this field. Then, it introduces several commonly used datasets and evaluation metrics in few-shot classification, compares several classical few-shot classification methods, and summarizes the experimental results obtained from public datasets. Finally, this paper analyzes the current challenges in few-shot image classification and suggests potential future directions.

Various daily behaviors usually exert pressure on the contact surface, such as lying, walking, and sitting. Obviously, the pressure data from the contact surface contain some important biological information for an individual. Recently, a computer vision task, i.e., pose estimation from contact pressure (PECP), has received more and more attention from researchers. Although several deep learning-based methods have been put forward in this field, they cannot achieve accurate prediction using the limited pressure information. To address this issue, we present a multi-task-based PECP model. Specifically, the autoencoder is introduced into our model for reconstructing input pressure data (i.e., the additional task), which can help our model generate high-quality features for the pressure data. Moreover, both the mean squared error and the spectral angle distance are adopted to construct the final loss function, whose aim is to eliminate the Euclidean distance and angle differences between the prediction and ground truth. Extensive experiments on the public dataset show that our method outperforms existing methods significantly in pose prediction from contact pressure. (c) 2024 SPIE and IS&T

Gait recognition is susceptible to external factors such as camera viewpoints, clothing, and carrying conditions, which could lead to performance degradation. To address these issues, the technique of non-rigid point set registration is introduced into gait recognition, which is used to improve the dynamic perception ability of human morphological changes by utilizing the deformation field between adjacent gait frames to represent the displacement of human contours during walking. Accordingly, a dual-flow convolutional neural network-GaitDef exploiting human contour deformation field is proposed in this paper, which consists of deformation field and gait silhouette extraction branches. Besides, a multi-scale feature extraction module is designed for the sparsity of deformation field data to obtain multi-level spatial structure information of the deformation field. A dynamic difference capture module and a context information augmentation module are proposed to capture the changing characteristics of dynamic regions in gait silhouettes and consequently enhance gait representation ability by utilizing context information. The output features of the dual-branch network structure are fused to obtain the final gait representation. Extensive experimental results verify the effectiveness of GaitDef. The average Rank-1 accuracy of GaitDef can achieve 93.5%(sic)68.3% on CASIA-B and CCPG datasets, respectively.