Sleep staging is essential for assessing sleep quality and diagnosing sleep disorders, yet current deep learning approaches face two major challenges: conventional fusion methods fail to account for the varying contributions of different modalities across sleep stages, and raw sleep data are often contaminated by artifacts that can interfere with model performance. We propose SleepGMUformer, a gated multimodal temporal neural network that processes multidomain sleep data including EEG (Fpz-Cz, Pz-Oz), EOG, and wearable biosignals (heart rate, motion, steps). Our architecture comprises: (1) a preprocessing module for signal alignment, artifact removal, and EEG detrending; (2) single-channel temporal feature extraction using transformer encoders; and (3) a novel Gated Multimodal Unit (GMU) for dynamic, instance-level modality weighting. Extensive experiments demonstrate that our model achieves state-of-the-art performance with accuracies of 85.7

Sleep staging is a key method for assessing sleep quality and diagnosing sleep disorders. However, current deep learning methods face challenges: 1) postfusion techniques ignore the varying contributions of different modalities; 2) unprocessed sleep data can interfere with frequency-domain information. To tackle these issues, this paper proposes a gated multimodal temporal neural network for multidomain sleep data, including heart rate, motion, steps, EEG (Fpz-Cz, Pz-Oz), and EOG from WristHR-Motion-Sleep and SleepEDF-78. The model integrates: 1) a pre-processing module for feature alignment, missing value handling, and EEG de-trending; 2) a feature extraction module for complex sleep features in the time dimension; and 3) a dynamic fusion module for real-time modality weighting.Experiments show classification accuracies of 85.03 SleepEDF-78 and 94.54 heterogeneous datasets and outperforms state-of-the-art models by 1.00

Human-object interaction (HOI) detection aims to achieve a fine-grained understanding of images by localizing humans and objects while interpreting the actions and spatial configurations between them. Existing approaches are heavily based on extensive annotated data, limiting their applicability in open scenarios. In order to address this, we propose a novel HOI detection framework that integrates generative feature augmentation supervised by contrastive learning to enhance the discriminability of generated features. Additionally, we introduce a hard negative mining strategy to distinguish semantically similar yet distinct interaction categories. The proposed approach employs a multibranch network that aggregates multiscale features during inference. Further, self-attention and cross-attention mechanisms are utilized to enrich semantic representations and strengthen comprehension of complex interactions. Extensive experiments on HICO-DET and V-COCO datasets demonstrate that the proposed approach substantially enhances detection performance for unseen interaction categories, achieving state-of-the-art generalization and robustness in zero-shot scenarios.

Action understanding, encompassing action detection and anticipation, plays a crucial role in numerous practical applications. However, untrimmed videos are often characterized by substantial redundant information and noise. Moreover, in modeling action understanding, the influence of the agent's intention on the action is often overlooked. Motivated by these issues, we propose a novel framework called the State-Specific Model (SSM), designed to unify and enhance both action detection and anticipation tasks. In the proposed framework, the Critical State-Based Memory Compression module compresses frame sequences into critical states, reducing information redundancy. The Action Pattern Learning module constructs a state-transition graph with multi-dimensional edges to model action dynamics in complex scenarios, on the basis of which potential future cues can be generated to represent intention. Furthermore, our Cross-Temporal Interaction module models the mutual influence between intentions and past as well as current information through cross-temporal interactions, thereby refining present and future features and ultimately realizing simultaneous action detection and anticipation. Extensive experiments on multiple benchmark datasets – including EPIC-Kitchens-100, THUMOS'14, TVSeries, and the introduced Parkinson's Disease Mouse Behaviour (PDMB) dataset – demonstrate the superior performance of our proposed framework compared to other state-of-the-art approaches. These results highlight the importance of action dynamics learning and cross-temporal interactions, laying a foundation for future action understanding research.

Cross-modal retrieval between video and motion capture (MoCap) data facilitates efficient reuse of human motion data in either skeletal or video format. For this purpose, we propose a deep cross-modal learning model for cross-modal retrieval between MoCap data and video data. First, we use a graph convolution-based network and a 3D convolution-based network to extract features from MoCap data and video data, respectively. In addition, we propose to use a pre-defined common subspace to maximize the inter-class variation and minimize the intra-class variation. Furthermore, we employ a similarity matrix to achieve the alignment between these two modalities and exploit their underlying correlations. For the purpose of experimental evaluation, due to the small amount of video data corresponding to the MoCap data in the public HDM05 dataset, we recorded a video dataset corresponding to the HDM05 motion capture dataset and performed cross-modal retrieval on it. The experimental results proved the effectiveness of the proposed scheme.

Estimating 3D human pose and shape from monocular videos is a challenging task due to inherent ambiguity and occlusion, which often lead to inaccurate predictions with high uncertainty. Despite significant progress in 3D pose and shape estimation from a single RGB image, achieving accurate and temporally coherent human mesh sequences from monocular videos remains a challenging endeavour. Inspired by the recent success of diffusion models in generating high-quality outputs with low uncertainty through progressively denoising noisy inputs, we propose a novel diffusion-based framework for 3D human pose and shape estimation. This framework formulates the human mesh recovery task as a reverse diffusion process. During training, it diffuses SMPL pose parameters from ground-truth distributions into input-specific distributions and learns to reverse this process. By leveraging the capacity of diffusion models to reduce noise progressively, our method effectively addresses the inherent ambiguity of this monocular task, producing accurate and smooth human mesh sequences from videos. Comprehensive experiments demonstrate that the proposed method significantly outperforms previous video-based methods in both per-frame 3D pose and shape accuracy and temporal coherence on widely used benchmarks, including 3DPW and Human3.6M datasets.

Human motion sequence segmentation plays a crucial role in understanding and applying human motion capture(MoCap) sequences. However, most of the traditional segmentation methods are designed to find the locations where the motion features have changed significantly. When dealing with complex motion scenes, such methods often lead to inefficiency, inaccuracy, and limitations. To address these challenges, we propose an end-to-end sequence segmentation method based on the Spatial Temporal Graph Convolutional Networks(ST-GCN). Our network effectively extracts motion features from MoCap sequences, reduces dimensions through convolutional operations, and identifies segmentation points between different motions. Under the constraints of excessive segmentation and clip length, the optimal segmentation is achieved by combining three carefully designed loss functions. The proposed framework was evaluated on two benchmark datasets, CMU MoCap database and HDM05 dataset, and achieved better accuracy and robustness compared with existing methods.

The grading and sorting process of the wood planks is a critical stage within the production line. However, in the real world, many factories still rely on humans to perform this task manually. This method is not only inefficient, but also time-consuming and laborintensive. To solve this problem, a lightweight wood board image classification algorithm based on a multichannel spatial attention mechanism is proposed in this paper, which can be used for the real-time classification of wood planks on the production line. This method is used to classify freshly rotated cut wood planks based on defects such as damage and voids on the production line. Specifically, the received images of the wood planks were processed by a feature extraction module to effectively separate the interfering background from the foreground of the wood planks. After fusing the edge information map with the foreground image of the wood planks, a multichannel convolutional neural network with spatial and channel attention ability was used to learn the features for correctly grading the wood planks. Experimental results show that the proposed method is superior to traditional methods and some existing deep learning algorithms in terms of performance and benefits.

Skeleton-based action recognition has achieved remarkable progress by employing graph convolutional neural networks (GCNs) to model correlations among body joints. However, GCNs have limitations in establishing long-term dependencies and are constrained by the natural connections of human body joints. To overcome these issues, we propose a Graph relative TRansformer (GTR) that captures temporal features through learnable topology and invariant joint adjacency graphs. The GTR provides a high-level representation of the structure of the spatial skeleton, seamlessly integrated into the time series. Moreover, we introduce a Multi-Stream Graph Transformer (MS-GTR) to integrate various dynamic information for an end-to-end human action recognition task. The MS-GTR applies a double-branch structure, where the GTR is implemented as the main branch to extract long-term dynamic features, and an auxiliary branch processes short-term kinematic content. Finally, we use cross-attention as an inter-branch interaction mediator. Experimental results on the HDM05, NTU RGB+D, and NTU RGB+D 120 datasets demonstrate the potential of the proposed MS-GTR model for improving action recognition.

Human motion prediction is the process of predicting future motion sequences based on past motion sequences. The graph convolution methods currently used for modelling human motion are effective in capturing the interrelationships between joints. However, these works lack skeleton constraints on the learning of graph filters, and DCT-based temporal modeling methods produce overly smooth motion representations and ignore the learning of human motion details. In this paper, we propose a network that uses adaptive spatial graph convolution and temporal self-attention to improve human motion prediction. The adaptive graph convolution effectively enhances cross-scale spatial interaction of joint movements based on different motion patterns. Meanwhile, temporal self-attention, combined with historical motion attention, improves the learning of motion temporal information. Our proposed network achieved state-of-the-art performance on two benchmark datasets, as demonstrated by extensive experiments.

In this work, we propose LiteMVGNet, a novel lightweight neural network for 3D shape classification. It is based on depth maps generated by multi-view rendering of the corresponding 3D model. LiteMVGNet is designed to be lightweight and effective in various aspects. First, the views and corresponding depth maps are partitioned into groups. Next, depth map features for each group are separately extracted by an adapted MobileNetV2 block. Finally, the extracted group features are fused by an adapted MobileViT block. The views are partitioned by good geometrical semantics and ECAnet is utilized to facilitate extraction of effective features. As demonstrated by experiments, in comparison with the state-of-the-art benchmark models, the proposed one cuts the network parameter count by a third and more and reduces the floating-point operation count by even one or two orders of magnitude. Still, the proposed model yields classification accuracies comparable with the benchmark models.

The observation and counting of red blood cells, white blood cells, and platelets in the blood are an important basis for clinical medical diagnosis. Abnormal blood cells mean that there may be blood-related problems such as clotting abnormalities, infections, and inflammation. As artificial blood cell detection is not only labor-intensive but also prone to false detection and misses, a novel blood cell detection algorithm YOLOv5-CBF is proposed to address the above problem. On the basis of the YOLOv5 framework, the algorithm improves detection accuracy by adding a coordinate attention（CA） mechanism to the backbone network. The FPN+PAN structure in the neck network is changed to the feature fusion structure combining the idea of the bidirectional feature pyramid network（BiFPN）, a cross-scale feature fusion method; in this way, the multi-scale features of the target can be effectively fused. In addition to the three-scale detection, a small target detection layer is added to improve the identification accuracy of small target platelets in the dataset. The results of a large number of experiments conducted on the dataset BCCD show that the algorithm presents an average accuracy improvement of 2.7% in the detection of the three blood cells compared to the conventional YOLOv5algorithm, demonstrating good performance. The algorithm is highly practical for blood cell detection.

Human Motion Capture (MoCap) has emerged as the most popular method for human animation production. However, due to joint occlusion, marker shedding, and equipment imprecision, the raw motion data is often corrupted, leading to missing motion data. To address this issue, a missing motion data recovery method utilizing attention-based transformers is proposed in this paper. The proposed model consists of two levels of transformers and a regression head. The first level of transformers extract the spatial features within each frame, and the second level of transformer integrates the per-frame features across time to capture temporal dependencies. The integrated features are then sent to the regression head to derive the complete motion. Extensive experiments on the CMU database demonstrate that the proposed model consistently outperforms the other state-of-the-art methods in recovery accuracy.

Most anchor-free object detectors suffer from intersample imbalance, underutilization of multiscale features and long training times in traffic object dataset. As a result, the efficiency and accuracy of the detector may be significantly reduced for samples with few categories and small sizes. To address these problems, we propose a novel anchor-free approach, namely, GSA-DLA34, which is based on Gaussian kernel, sample weights, and attention. Its features are as follows. First, pyramid squeeze attention (PSA) is added after the backbone network to enhance multiscale traffic object representations. Second, for better object positioning with few categories and small scales, we design active sample weights for regression loss to make better information use. In addition, an elliptical Gaussian sampling module (EGSM) with a controllable Gaussian kernel shape is incorporated into the classification and regression branches to accelerate network training. The results show that our GSA-DLA34 has a significant advantage in balancing training time, inference speed, and accuracy. With an average precision of 89% on the PASCAL VOC dataset and an inference speed of 55.2 FPS on the RTX 2080 Ti, the GSA-DLA34 method can significantly improve human-vehicle recognition accuracy.

Accurate segmentation and analysis for each animal in surveillance video images will help poultry farmers to monitor and promote animal welfare. However, it is challenging to accurately segment each animal due to the similar appearance, different scales, rapid growth and adhesive areas of group animals. Meanwhile, lacking of useful training data also limits the effectiveness of animal segmentation algorithms. To address these problems, we first construct a chicken image segmentation dataset to study the behavior of chickens for intelligent monitoring and analysis. Then, we propose an effective end-to-end framework for chicken image segmentation, which can also be used for other animal image segmentation. An end-to-end multi-scale based encoder-decoder network is first utilized to extract multi-scale features. Then, an attention-based module is employed to extract and intensify effective features, thus better segmentation results can be obtained. Finally, a multi-output combined loss function is proposed to make effective supervision for better segmentation. Experimental results demonstrate the promising performance of the proposed framework for chicken image segmentation.

In this work, we propose an efficient retrieval method for human motion capture (MoCap) data based on supervised deep hash code learning. Raw Mocap data is represented into three 2D images, which encode the trajectories, velocities and self-similarity of joints respectively. Such image-based representations are fed into a convolutional neural network (CNN) adapted from the pre-trained VGG16 network. Further, we add a hash layer to fine-tune the CNN and generate the hash codes. By minimizing the loss defined by classification error and constraints on hash codes, highly discriminative hash representations of the motion data can be generated. As experimentally demonstrated on the public HDM05 data set, our algorithm achieves high accuracy comparing with the state-of-the-art MoCap data retrieval algorithms. Besides, it achieves high efficiency due to the fast matching of hash codes.

With the rapid development of data-driven animation technologies, huge motion capture data has been accumulated. Motion capture data is a kind of spatio-temporal high dimensional data, which needs a lot of storage space. Efficient compression and transmission of motion capture data has become a hot topic in computer animation. In this paper, we induce the characteristics of motion capture data and the general processing pipeline of motion capture data compression algorithm. Then we review the research achievements in the field of motion capture data compression for the last twenty years. According to reduced dimensions, running platform, lossy or lossless, motion data format, environment contact processing and progressivity, we put these motion capture data compression algorithms into different categories. Finally, we look forward to the future research trend.

In the traditional teaching mode, the beginners generally feel that the concepts in the programming courses are abstract and the programming ability is difficult to improve. Analyzing the teaching contents and the key difficulties in the teaching activities, we put forward the mixed teaching mode which combines the MOOC teaching mode and the traditional classroom teaching mode. This teaching mode can make full use of the existing online resources to build courses, and focus on arousing the learners’ autonomous learning motivation and the mutual cooperation ability within study groups. The teaching practice shows that this mixed teaching mode can better improve the learning interest and practical programming ability of the students compared with the traditional teaching mode.

A single image based haze weather recognition is the fundamental operation of the applications of outdoor computer vision. Currently, the recognition results are remains undesirable and most existing methods are only for the fixed scene. In this paper, we propose multiple features and Random Forest based haze weather classification method for any scenario to improve the detection accuracy. First, through systematically investigation, multiple features are extracted and properly processed. Then, these features are combined into high dimension vectors and the Random Forest is adopted to lean an adaptive classifier for haze recognition. In the experiment, an outdoor image set which contains around 4000 images is collected. Form the experimental results is can be seen that the proposed method achieves 97.4% recognition accuracy of the haze weather on the collected dataset.