Human pose estimation is sensitive to the spatial resolution of feature representations in a single image. While multi-scale features are often exploited to improve keypoint localization, the effectiveness of parallel spatial learning across multiple uniform-sized spaces remains underexplored. To address this, To address this, we propose the Parallel Space Interaction Network (PSINet), which splits input features into multiple identical parallel spaces to facilitate efficient spatial feature learning for human pose estimation. Based on the multiple spaces, we refine the vanilla vision Transformer by proposing a Split-Head Transformer (SHT) to enhance global interaction within parallel spaces. Furthermore, we introduce Parallel Space Fusion instrument to strengthen the capacity of the PSINet by acquiring the relationship between multiple parallel spaces. Extensive experiments demonstrate that PSINet achieves state-of-the-art accuracy after data augmentation, running at 180 FPS on a single GTX2080Ti, offering a superior trade-off between accuracy and efficiency.

3-Dscanning has various applications in industrial design. However, the noisy point clouds being generated are difficult to be used in the applications directly. The ease of editing and lightweight nature of wireframes make it highly suitable for industrial design. To address this issue, WG-Net is proposed for generating wireframes from noisy point clouds. An edge primitive detection network that includes a multilevel feature extraction module and a feature fusion module is designed for point classification, primitive segmentation, and displacement vector prediction. Based on the classification and segmentation results of edge points, targeted fitting methods are applied to accurately fit each category of edge primitives, ensuring precise geometric representation. Finally, fitting results are integrated into a complete wireframe structure. Through extensive experiments on computer-aided design datasets, WG-Net achieved significant performance improvements compared to state-of-the-art methods. In addition, we have verified the feasibility and practical applicability of WG-Net on real scanned data.

Video shadow removal remains a challenging and underexplored task due to the lack of large-scale datasets and open-source benchmarks. To address this gap, we present Open Video Shadow Removal Dataset (Open-VSRD), the first publicly available dataset for video shadow removal, accompanied by a dedicated video diffusion framework designed specifically for multi-frame shadow removal. Open-VSRD is a large-scale synthetic dataset built using the Unity engine, consisting of 5931 training clips and 244 testing clips captured from diverse scenes. It features a wide range of dynamic scenarios involving various moving objects, complex camera motions, and heterogeneous lighting conditions, offering rich spatial–temporal and illumination variations essential for robust model training and evaluation. Building upon this dataset, we propose a video diffusion method based on alternating bidirectional diffusion guidance strategy that enhances temporal stability specifically within shadow regions while maintaining overall visual coherence. Furthermore, a temporal VAE (variational autoencoder) decoder is designed to leverage multi-scale latent features from a frozen encoder, significantly improving reconstruction fidelity and temporal consistency. Extensive experiments demonstrate that our method substantially outperforms existing single-frame shadow removal approaches, validating both the effectiveness of the proposed framework and the significance of the Open-VSRD dataset.

Most current methods use multi-scale feature fusion to improve human keypoint estimation accuracy. However, such methods are often computationally inefficient. To address this, we propose a Multi-Level Efficient Network (MLENet) that maintains single-scale features throughout the network and comprises three modules: Local Feature Extraction (LFE), Spatial Feature Extraction (SFE), and Global Feature Extraction (GFE). The objectives of this study are to improve keypoint estimation accuracy, maintain computational efficiency, and investigate the effect of combining these three modules. The LFE captures low-level features from the input image using convolutional blocks and incorporates Channel Squeeze Attention and Spatial Squeeze Attention to enhance overall performance. The GFE employs self-attention to capture global dependencies in high-level features and leverage position-wise information to improve Transformer robustness. In contrast, the SFE extracts middle-level features by separating feature channels into several identical spaces and applying multiple convolutions to obtain mixed features. By integrating low-middle-high multi-level feature extraction, MLENet facilitates complementary feature learning throughout the network. Extensive experiments confirm that MLENet outperforms most current state-of-the-art methods on COCO and MPII datasets while being more efficient than mainstream multi-scale fusion architectures, showing that the combination of the proposed three modules effectively enhances keypoint localization performance. The source code will be publicly available at: https://github.com/wangdong0556/MLENet.

In the realm of high dynamic range (HDR) imaging, tone mapping (TM) and multi-exposure fusion (MEF) techniques are essential for achieving visually appealing HDR-style low dynamic range (LDR) photographs. However, these methods tend to introduce undesirable artefacts, thus requiring robust perceptual quality assessment. This paper proposes a hierarchical framework, dubbed HQaN&A, for automatically evaluating the naturalness and aesthetics attributes of TM/MEF-processed LDR images. By extracting low-level naturalness features and high-level aesthetic attributes, the HQaN&A model simulates the human visual system's hierarchical perception process. Our approach is the first to jointly model naturalness and aesthetic in a hierarchical manner, bridging the gap between low-level image statistics and high-level perceptual appeal. Experimental results on benchmark databases demonstrate that our approach outperforms state-of-the-art methods, achieving SRCC = 0.812 and PLCC = 0.813 on the ESPL-LIVE HDR database. Here, we show that integrating local and regional naturalness with aesthetic evaluations leads to more reliable and comprehensive image quality predictions. This work not only advances the field of image quality assessment but also paves the way for applications in photo editing and automatic aesthetic judgement. Our code will be released at https://github.com/fjl0988/QmN-A.

Motion retargeting is a key task in computer character animation. Existing methods predominantly employ Transformers as backbones. However, the self-attention mechanism in these models scales quadratically with sequence length, which hinders the efficient generation of high-fidelity and temporally smooth motions. To address this, we propose HMT-ReNet, which incorporates State Space Model (SSM) to enable long-sequence modeling with linear complexity. Specifically, in the temporal dimension, we design a Bidirectional Mamba module to simultaneously capture forward and backward temporal dependencies, thereby mitigating the limitations imposed by the causal linearity of standard Mamba. In the spatial dimension, we introduce an attention-based pooling strategy for point cloud feature aggregation. This design addresses the limitations of PointNet, which primarily relies on max-pooling and often leads to local information loss, by enabling the model to adaptively attend to key geometric features that are significant for retargeting. We train and evaluate our model on a dataset downloaded from Mixamo. The Experiments we have done demonstrate that HMT-ReNet outperforms comparable Transformer baselines. In terms of generation quality, our method effectively reduces mesh interpenetration and contact error. In terms of efficiency, HMT-ReNet reduces the number of parameters by 30% and FLOPs by 7% compared to Transformer baseline with the same number of layers. These advantages highlight the potential of leveraging SSMs as a promising solution for motion retargeting tasks.

Image shadow removal has received increasing attention in recent years. Existing deep learning-based shadow removal methods usually rely on the assumption of globally uniform illumination or employ simplified multiplicative illumination models to estimate parameters related to shadow-free image reconstruction. However, the shading process often contains non-uniform, diverse, and complex shadow patterns, substantially reducing the robustness of current shadow removal techniques and limiting their performance. To address this issue, this paper revisits the traditional shadow degradation model and introduces a pixel-wise adaptive non-uniform illumination model. Building upon this model, a shadow adaptive normalization (SAN) module is designed to estimate the parameter maps of the illumination model and rectify the shadow features within the latent space. The proposed SAN dynamically performs attentive normalization on shadow region features, which can align the statistical distributions of shadow and non-shadow regions. To enhance the efficiency of the SAN module, this paper introduces an intra-module complexity reduction strategy to reduce computational complexity while improving the stability of the training process. In addition, to mitigate the color deviation between paired training data, this paper introduces a Poisson-function-based loss to achieve color robustness. Extensive experiments on image shadow triplet dataset (ISTD), adjusted image shadow triplet dataset (ISTD+), and shadow removal dataset (SRD) validate the superiority of the proposed method over other state-of-the-art (SOTA) approaches.

Generating 3-D wireframe from point cloud has been widely applied in 3-D measurement. The primary challenge lies in accurately detecting edge points and constructing complete wireframe structures. To address this, edge detection and wireframe generation (EDWG) is proposed to efficiently generate wireframe from point clouds. The approach begins with a deep learning-based edge point detection network, incorporating local point self-attention (LPSA) and a multilevel feature fusion (MLFF) module to enhance edge perception. These innovative mechanisms can effectively capture edge information in complex environments, thus laying the foundation for wireframe generation. Following edge point detection, a four-step process is employed, which includes wireframe initialization, outlier removal, edge point reconnection, and spline fitting to construct the wireframe structure. Through extensive experiments on several challenging publicly datasets and measurement point clouds, EDWG achieves 98.8% accuracy in edge detection and reduces the Chamfer distance (CD) error of the wireframe to 0.012. The wireframe generated by EDWG can be directly applied to geometric tasks such as surface reconstruction.

Convolutional neural networks (CNNs) have recently demonstrated impressive performance in complex machine learning tasks. However, the CNN requires a large quantity of annotated data to converge to a good solution, and the theoretical understanding of this network is still in its infancy. Towards this end, a variant of the CNN, dubbed the deep scattering network (DSN), has been proposed by employing the linear time–frequency transform. The DSN inherits the hierarchical structure of the CNN, but chooses predefined wavelet/Gabor filters as its convolutional kernels instead of data-driven linear filters. Unfortunately, the DSN suffers from a major drawback that it is suitable for stationary image textures but not for non-stationary image textures, since wavelet/Gabor filters are intrinsically linear translation-invariant filters. The aim of this paper is to overcome this deficiency based upon a generalized linear time–frequency transform–the short-time fractional Fourier transform (STFRFT) which can be interpreted as a bank of linear translation-variant filters and thus may be well suitable for non-stationary texture analysis. We first introduce a generalized time–frequency scattering transform using the STFRFT. By applying the derived result, we propose an interpretable CNN by cascading the STFRFTs and modulus operators. Moreover, several basic properties of the proposed interpretable CNN are derived, and an efficient implementation of this network is also presented. Finally, the applications of the derived results are discussed.

Human pose estimation has witnessed substantial advancements with the utilization of the Vision Transformer (ViT), which can accurately measure keypoint positions by leveraging ViT's ability to extract global features. While increasing the depth or width of the self-attention mechanism can improve the accuracy of joint measurement, it inevitably results in higher computational complexity. This article presents an efficient dual-scale cross-attention network (EDCNet) for the precise human keypoint measurement. EDCNet incorporates a space split module (SSM) and a dual-scale cross-attention module (DCM), striking a more favorable balance between efficiency and accuracy compared to the conventional stacking of vanilla ViT. At the core of DCM lies a novel structure termed the channel-spatial ViT (CSViT). By leveraging the cross-attention mechanism formed by dual-scale CSViT, DCM effectively captures both local and global spatial dependencies. Specifically, CSViT utilizes a spatial activation unit (SAU) to amalgamate independent spatial information from the query, key, and value, effectively integrating both short- and long-range dependencies. Concurrently, we introduce a channel activation unit (CAU) within CSViT to enhance channel awareness through successive convolutions. Moreover, the SSM facilitates access to multispatial features through cost-effective spatial splitting, in conjunction with DCM, enhancing the overall performance of EDCNet. Extensive experiments on the COCO and MPII benchmarks demonstrate the effectiveness of our proposed EDCNet, obtaining 74.6 and 91.9 AP on COCO val and MPII val, respectively, which achieves superior performance with fewer parameters and lower computational costs than state-of-the-art methods.

Some 3D computer vision techniques such as structure from motion (SFM) and augmented reality (AR) depend on a specific perspective-n-point (PnP) algorithm to estimate the absolute camera pose. However, existing PnP algorithms are difficult to achieve a good balance between accuracy and efficiency, and most of them do not make full use of the internal camera information such as focal length. In order to attack these drawbacks, we propose a fast and robust PnP (FRPnP) method to calculate the absolute camera pose for 3D compute vision. In the proposed FRPnP method, we firstly formulate the PnP problem as the optimization problem in the null space that can avoid the effects of the depth of each 3D point. Secondly, we can easily get the solution by the direct manner using singular value decomposition. Finally, the accurate information of camera pose can be obtained by optimization strategy. We explore four ways to evaluate the proposed FRPnP algorithm with synthetic dataset, real images, and apply it in the AR and SFM system. Experimental results show that the proposed FRPnP method can obtain the best balance between computational cost and precision, and clearly outperforms the state-of-the-art PnP methods.

Large-scale generative models have achieved remarkable advancements in various visual tasks, yet their application to shadow removal in images remains challenging. These models often generate diverse, realistic details without adequate focus on fidelity, failing to meet the crucial requirements of shadow removal, which necessitates precise preservation of image content. In contrast to prior approaches that aimed to regenerate shadow-free images from scratch, this paper utilizes diffusion models to generate and refine image residuals. This strategy fully uses the inherent detailed information within shadowed images, resulting in a more efficient and faithful reconstruction of shadow-free content. Additionally, to revent the accumulation of errors during the generation process, a crosstimestep self-enhancement training strategy is proposed. This strategy leverages the network itself to augment the training data, not only increasing the volume of data but also enabling the network to dynamically correct its generation trajectory, ensuring a more accurate and robust output. In addition, to address the loss of original details in the process of image encoding and decoding of large generative models, a content-preserved encoder-decoder structure is designed with a control mechanism and multi-scale skip connections to achieve high-fidelity shadow-free image reconstruction. Experimental results demonstrate that the proposed method can reproduce high-quality results based on a large latent diffusion prior and faithfully preserve the original contents in shadow regions.

In the past decade, visual odometry has made great progress due to the development of computer vision technology. However, the existing methods are mainly designed for vehicles rather than mobile devices because of their high computational cost. To address the problem, in this article, a fast visual odometry system is proposed for smartphones, the presented system called FastVO. The FastVO consists of feature tracking, relative pose estimation, and trajectory construction. Experiments conducted on both the benchmarking dataset and the real scenes are provided to demonstrate the effectiveness of the presented visual odometry system.

A match is considered as an incorrect match when the matched features in two views do not correspond to the same physical location. It is inevitable that generates mismatches at a local descriptor level. Differentiating true and false matches remains a challenge, especially in the case of ambiguities, wide baselines, and strong illumination variations, which might contain a large number of mismatches (even up to 90%). In this paper, we develop GlcMatch, an outlier rejection method that takes advantage of both global and local constraints to classify putative matches. Specifically, we use vector field consistency to form continuous global smoothness and use triangular mesh constraints to implement the local piecewise smoothness. Evaluation on benchmark datasets demonstrates GlcMatch can obtain large numbers of good quality correspondences and achieve significant performance.

Local feature matching plays a vital role in various computer vision tasks. In this work, we present a novel network that combines feature matching and outlier rejection for finding reliable correspondences between image pairs. The proposed method is a hybrid transformer-based graph neural network (GNN), termed HTMatch, which aims to achieve high accuracy and efficient feature matching. Specifically, we first propose a hybrid transformer that integrates self-and cross-attention together to condition the fea-ture descriptors between image pairs. By doing so, the intra/inter-graph attentional aggregation can be realized by a single transformer layer, which achieves more efficient message passing. Then, we introduce a new spatial embedding module to enhance the spatial constraints across images. The spatial informa-tion from one image is embedded into another, which can significantly improve matching performance. Finally, we adopt a seeded GNN architecture for establishing a sparse graph, which improves both ef-ficiency and effectiveness. Experiments show that HTMatch reaches state-of-the-art results on several public benchmarks.(c) 2022 Elsevier B.V. All rights reserved.

Most current human pose estimation methods pursue excellent performance via large models and intensive computational requirements, resulting in slower models. These methods cannot be effectively adopted for human pose estimation in real applications due to their high memory and computational costs. To achieve a trade-off between accuracy and efficiency, we propose TRPose, a Transformer-based network for human pose estimation rapidly. TRPose consists of an early convolutional stage and a later Transformer stage seamlessly. Concretely, the convolutional stage forms a Rapid Fusion Module (RFM), which efficiently acquires multi-scale features via three parallel convolution branches. The Transformer stage utilizes multi-resolution Transformers to construct a Dual scale Encoder Module (DEM), aiming at learning long-range dependencies from different scale features of the whole human skeletal keypoints. The experiments show that TRPose acquires 74.3 AP and 73.8 AP on COCO validation and testdev datasets with 170+ FPS on a GTX2080Ti, which achieves the better efficiency and effectiveness trade-offs than most state-of-the-art methods. Our model also outperforms mainstream Transformer-based architectures on MPII dataset, yielding 89.9 PCK@0.5 score on val set without extra data.

Classical clustering algorithms use all features to partition a dataset, making it difficult for users to understand the clustering results. Some scholars have proposed interpretable clustering algorithms that use a few understandable features to explain clustering results. However, the existing algorithms can only generate one interpretation and fail to satisfy the diverse needs of different users. To address this challenge, the Clustering Customized Explanation Tree (CCET), a visual analytics system, was constructed in this paper. The system helps users modify existing explanations to obtain customized explanations. Firstly, a variety of views are designed to visualize the explanations and help users judge whether the existing explanations meet the requirements. Then, an explanations modification strategy based on cluster centroids splitting is proposed making it easy for users to revise explanations according to the requirement. We demonstrate the CCET using a case study and a user study. The results show that the system can deepen users’ understanding of clustering results and make it easy for them to conduct further decision analysis.

The existing facility layout problem (FLP) only considers the layout of processing facilities. However, in the current scenario of industrial logistics, there are not only working facilities, but also a lot of transportation facilities. In order to solve the logistics facility layout problem (LFLP), this paper proposes a two-step algorithmic framework based on GPU acceleration to obtain some feasible solutions. The first step is layout of working facilities by meta-heuristic algorithm, and the second step is layout of transportation facilities by routing algorithm. Then, a GPU parallel solution is used to accelerate the optimization of material handling cost (MHC) and transportation facilities cost (TFC) during the whole process. Finally, the framework is tested on three layout instances by two different meta-heuristic algorithms. Compared to no routing step or no GPU, our method is more accurate and effective in solving the LFLP.