Extracting robust features from irregular point clouds remains a major challenge in 3D perception and structural understanding. Existing point cloud representation learning methods mainly rely on data-driven strategies to extract geometric information from scattered point clouds. However, most focus solely on point cloud spatial distribution while overlooking the relationship between local details and the whole structure, which affects the accuracy of the learned representations. Local information reflects the fine-grained variations of an object, while the whole structure is determined by the interaction and combination of these local features. For an object under external forces, slight deformations typically first arise in local regions and then propagate to the whole structure through structural connections. This indicates that applying force to an object can reveal the intrinsic dependencies between local parts and the whole structure. Inspired by this mechanism, we introduce a physics-driven auxiliary structural regularizer to capture local-whole structural relationships. Specifically, we design a dual-task encoder-decoder framework, in which the implicit feature learning branch learns the global geometric representation of the point cloud, while the physical information awareness branch uses small elastic deformation responses under controlled external forces as auxiliary supervision and regularizes the network through a data-fidelity loss and a physics-informed loss. By combining data-driven geometric modeling with physics-driven local-whole structural awareness, the proposed method achieves more accurate point cloud representations. Experimental results on object classification and segmentation benchmarks demonstrate that the proposed method achieves strong performance, verifying its effectiveness.

In many real-world applications such as autonomous vehicles and robots, LiDAR sensor is an essential equipment due to its capacity to scan surrounding environment. It produces sequential point clouds by rotating the laser emitter in a consecutive pattern. Therefore, semantic segmentation of the sequential point clouds is critical for understanding the surrounding environment for autonomous vehicles and robots. Unlike the semantic segmen tation of single scan, this task requires distinguishing the moving objects from static ones. However, existing semantic segmentation methods for single scan perform poorly on the multi-scan task due to the lack of temporal information. In this paper, we propose a novel framework, which consists of a Spatial-Aware Feature Learning module (SAFL) and a Temporal-Aware Feature Learning module (TAFL), to extract spatial and temporal informa tion in a unified pattern. Specifically, we project each point cloud into a pseudo-image by spherical projection and stack several sequential images along the temporal dimension, forming a 3D grid. First, the SAFL module extracts spatial features for each voxel using submanifold sparse convolution and reduces the resolution through a sparse convolution. Then, the TAFL module adopts a window-based Transformer along with a specially designed mask mechanism to learn the temporal information. Moreover, we design a Motion-Aware Feature Learning module (MAFL), using an optimized 2D network and residual images built from the stacked images to strengthen the moving feature learning and facilitate the prediction of moving objects. We evaluate our proposed method on the Synthia4D and SemanticKITTI multi-scan datasets and the results demonstrate that our method achieves com petitive results than most previous methods with less latency, which provides a novel idea to process 4D LiDAR tasks. The code is available at https://github.com/daojianqingchou/SMFNet.git.

Point cloud registration, a fundamental task in computer vision, aims to align point cloud pairs from different perspectives by estimating optimal transformations. This task faces challenges including large-scale outliers, low overlap scenarios, and poor generalization on same-source and cross-source registration. To address these issues, we propose DeepPAT, a deep position-aware transformer that better learns position information of local structures. We first design a deep PPFTrans encoder to extract discriminative feature descriptors with local geometry. Then, we propose a position-aware transformer to enhance the feature descriptors by incorporating relative position information. To handle density inconsistencies, density-adaptive matching is introduced to generate initial point correspondences. For outlier rejection, a FPFH-MAC estimator is proposed to construct geometrically consistent correspondence. Finally, we train our model on a mixed dataset combining synthetic and real-world data to improve generalization across cross-source data. Experiments demonstrate that our method achieves state-of-the-art performance, with 92.7% and 75.7% registration recall on 3DMatch and 3DLoMatch benchmarks, respectively. Notably, our method generalizes robustly to cross-source data, achieving 90.6% and 9.8% registration recall on Kinect-sfm and Kinect-lidar benchmarks, significantly outperforming existing methods in challenging cross-domain scenarios.

High-quality correspondences are critical to the accuracy and robustness of point cloud registration. Existing Transformer-based methods are fundamentally constrained by the quadratic computational complexity of self-attention, resulting in limited scalability. Moreover, conventional outlier removal paradigms operate by pruning initial correspondences, and thus fail catastrophically in low-overlap scenarios where initial inliers are inherently scarce. To address these challenges, we propose MaLCA, a point cloud registration method based on Mamba-enhanced features and local correspondence augmentation. We first adopt KPFCN as the backbone to extract multi-scale geometric features from raw point clouds. A Mamba selective state space model then replaces self-attention for global context modeling with linear complexity, while cross-attention is retained to facilitate inter-point-cloud feature interaction. Rather than following the conventional subtraction-based outlier removal paradigm, we introduce a prior-guided local rematching strategy combined with a fused neighbor matching mechanism that iteratively constructs dense, high-quality correspondences from sparse initial inliers, fundamentally overcoming the bottleneck of inlier scarcity in challenging scenes. Extensive experiments on the 3DMatch/3DLoMatch and 4DMatch/4DLoMatch benchmarks demonstrate that MaLCA achieves competitive registration performance across both rigid and deformable scenarios, with particular advantages in low-overlap cases.

Enhancing the service capabilities of robots relies heavily on their autonomous navigation performance and effective obstacle avoidance during motion. This study addresses key limitations of current deep reinforcement learning-based path planning methods, such as low training efficiency, suboptimal trajectory generation, and poor performance in complex trap scenarios. To address these challenges, we propose GDR-SAC (Genetic and Dynamic Reward-based Soft Actor-Critic), a novel approach that integrates a dedicated Lidar data feature extraction module for enhanced environmental perception, a customized SAC network architecture, a dynamic reward function tailored for continuous action spaces, and an innovative sampling strategy enhanced by genetic algorithms. Furthermore, we introduce IPGDR-SAC, an extension of GDR-SAC that incorporates a Point of Interest (PoI) exploration module to effectively navigate complex trap scenarios. Comprehensive training and evaluation across diverse environments, including both Gazebo simulation platforms and real-world settings demonstrate the effectiveness of the proposed approach. Experimental results show that the GDR-SAC algorithm significantly improves training efficiency and convergence speed, while generating smoother, shorter, and more efficient collision-free trajectories. Moreover, IPGDR-SAC exhibits superior autonomous navigation performance in intricate environments by leveraging Lidar data to successfully escape from entrapment situations and handle challenging navigation tasks.

Infrared-visible image fusion seeks to yield a composite result where salient thermal targets alongside the rich textural details are emphasized. However, recent advances generally rely on highly complex pipelines and excessively redundant units, making the process intricate and time-consuming. To address these limitations, we first conduct an in-depth optimization of existing methods from two key perspectives: (i) the top-level architecture and (ii) the fundamental modules. Moreover, to ensure that the simplified design maintains strong feature representation capability, three complementary mechanisms are introduced: (i) a multi-hierarchy iterative strategy, (ii) a progressive channel enhancement mechanism, and (iii) a complementary gradient-constrained loss. Based on this, we propose the progressive multi-hierarchy iterative network (PHNet). Extensive comparisons with multiple state-of-the-art methods demonstrate that PHNet achieves superior fusion quality with a remarkable reduction in parameters and FLOPs by 98.6% and 69.4%, respectively. Meanwhile, it maintains real-time performance. Considering only the model inference time, PHNet achieves inference speeds ranging from 464 FPS to 1000 FPS under different input resolutions across multiple datasets, significantly outperforming the competing methods.

Point cloud registration, a core task in 3D computer vision for aligning two point clouds via rotation and translation, underpins critical applications like robotic navigation and 3D reconstruction. Classical methods (e.g., Iterative Closest Point) easily converge to local minima under poor initial alignment. Deep learning-based approaches, while efficient, suffer from high annotation costs for large-scale data. Existing reinforcement learning (RL)-based methods rely on simple PointNet feature extractors, which are insensitive to local geometric details and thus yield suboptimal registration precision. To address these challenges, we propose ACT-Agent: Affinity-Cross Transformer for point cloud registration via reinforcement learning, a novel method that formulates point cloud registration as an RL Markov decision process for iterative optimisation. We leverage Pointnet and Affinity-Cross Transformer to extract and enhance expressive salient features and assign adaptive weights to channels based on their relative importance. We use RL to autonomously learn from feedback in the environment, freeing ourselves from dependence on data annotation. Experimental results on ModelNet40 (synthetic data) and ScanObjectNN (real-world data) demonstrate that our proposed ACT-Agent achieves higher accuracy, efficiency, and generalisation ability than the state-of-the-art methods of point cloud registration.

3D object detection plays a vital role in autonomous driving and intelligent measurement systems. Transformer-based frameworks have recently gained prominence by predicting sampling points and directly extracting features from the image space. This design bypasses explicit depth estimation, bird's eye view projection, and post-processing steps such as non-maximum suppression, thereby significantly improving inference speed. However, these methods often lag behind view transformation-based approaches in detection accuracy, particularly in the quantitative estimation of object size, orientation, and motion parameters. This paper attributes this performance gap to the difficulty of extracting reliable object features from the sampled locations. Since the sampling points are only indirectly supervised by detection losses, many of them often do not accurately correspond to actual object regions, resulting in the collection of background or irrelevant features. Furthermore, existing motion-based strategies for aligning sampling points across temporal frames often suffer from significant errors, which further compromise the quality of aggregated temporal features. To tackle these challenges, we introduce a novel 3D object detection and measurement framework that integrates two core components: point-driven sampling, which leverages supervision from point clouds to direct the sampling process toward object-relevant regions during training, thereby enhancing the fidelity of geometric measurements; and center-aligned temporal aggregation, which uses a learnable module to adaptively align and fuse temporal features, eliminating the need for explicit motion compensation while improving dynamic measurement accuracy. Experiments on the nuscenes benchmark further validate the effectiveness of our approach. Using ResNet-50, our method achieves 56.0 NDS, 45.9 mAP, and 38.4 ms latency, while the ResNet-101 variant reaches 59.5 NDS, 51.1 mAP, and 84.1 ms latency-achieving a favorable balance between accuracy and speed.

Automatic crack detection is challenging, owing to the complex and thin topologies, diversity, and background noises of cracks. Inspired by the wavelet theory, we present an instance normalization wavelet (INW) layer and embed the layer into the deep model for segmentation. The proposed layer employs prior knowledge in the wavelets to capture the crack features and filter the high-frequency noises simultaneously, accelerating the convergence of model training. Furthermore, instance normalization in our layer is utilized to mitigate the feature differences, boosting the generalization capability. In addition, a fusion layer is added to merge the information across the different layers. The comparison experiments and ablation studies demonstrate that the INW layer steadily enhances recognition and convergence performance on the DeepCrack dataset and CRACK500 dataset.

Aiming to solve problems of low-accuracy and slow grasp detection in unstructured environments,a grasp detection algorithm alter-attention pyramid network(APNet)is proposed.Generative residual convolutional neural network(GR-ConvNet)was selected as the backbone network,adaptive kernel convolution was used to replace standard convolution,and the SiLU activation function was replaced with the Hardswish activation function.A lightweight feature extraction network was developed,and efficient multiscale attention was introduced to increase focus on important grasping regions.Pyramid convolution was integrated into the residual network to effectively fuse multiscale features.The experimental results demonstrate that APNet achieves 99.3%and 95.8%detection accuracies on the Cornell and Jacquard datasets,with an average time required for single-object detection of 9 ms and 10 ms,respectively.Compared with existing algorithms,APNet demonstrated improved detection performance.In particular,APNet demonstrates an average success rate of 92%on a homemade multi-target dataset for a grasping experiment implemented in a CoppeliaSim simulation environment.

Remote sensing image change detection has advanced significantly with the development of convolutional neural networks (CNNs), yet challenges persist in identifying unbalanced changes in foreground–background categories, especially under limited samples and interference from seasonal variations, lighting changes, or structural renovations. The proposed IDJANet architecture provides an efficient solution for change detection (CD) in remote sensing images by enhancing early feature interactions between bi-temporal images. It utilizes a Siamese pre-trained FastSAM Adaptor for multi-level feature extraction, followed by the Iterative Deformable Joint Attention (IDJ-Attention) module, which applies mutually guided attention to suppress irrelevant noise and highlight actual changes. The coupled bi-temporal features are transformed into difference maps using subtraction and fusion operations, effectively capturing significant variations. Finally, deformable convolution replaces traditional convolution in the prediction stage, enabling the model to better adapt to complex geometries and improving its accuracy in detecting local changes. Experimental results on four CD datasets demonstrate the model’s effectiveness, particularly in low-sample scenarios, achieving superior performance while maintaining a favorable balance between accuracy and computational cost. This research highlights a reliable and efficient architecture for addressing the complexities of change detection in remote sensing, offering a robust solution for practical applications.

Currently, 3D object detection methods fail to detect small objects due to the fewer effective points of small objects. It is a significant challenge to reduce the loss of information of points in representation learning. To this end, we propose an effective 3D detection method with object matching set abstraction (OMSA). We observe that key points are lost during feature learning with multiple set abstraction layers, especially for downsampling and queries. Therefore, we present a novel sampling module named focus-based sampling, which raises the sampling probability of small objects. In addition, we design a multi-scale cube query to match the small objects with a close geometric alignment. Our comprehensive experimental evaluations on the KITTI 3D benchmark demonstrate significant performance improvements in 3D object detection. Notably, the proposed framework exhibits competitive detection accuracy for small objects (pedestrians and cyclists). Through an ablation study, we verify that each module contributes to the performance enhancement and demonstrate the robustness of the method against the balance factor.

Although current 3D semantic segmentation methods have achieved significant success, they suffer from catastrophic forgetting when confronted with dynamic, open environments. To address this issue, class incremental learning is introduced to update models while maintaining a balance between plasticity and stability. In this work, we propose CosPrompt, a rehearsal-free approach for class incremental semantic segmentation. Specifically, we freeze the prompts for existing classes and incrementally expand and fine-tune the prompts for new classes, thereby generating discriminative and customized features. We employ clamping operations to regulate backward propagation, ensuring smooth training. Furthermore, we utilize the learning without forgetting loss and pseudo-label generation to further mitigate catastrophic forgetting. We conduct comparative and ablation experiments on the S3DIS dataset and ScanNet v2 dataset, demonstrating the effectiveness and feasibility of our method.

To address the challenges of low detection accuracy and efficiency in YOLOv7 for indoor object detection, this paper proposes a novel algorithm, SBP-YOLOv7, tailored for complex indoor environments. The improvements include the introduction of the parameter-free SimAM attention mechanism into the backbone network to enhance the focus on target objects, the design of a Short-BiFPN (lightweight weighted bidirectional feature pyramid network) for efficient multi-scale feature fusion in the neck network, the integration of PConv (partial convolution) in the ELAN module of the backbone for lightweight design, and the adoption of GSConv (Group Shuffle Convolution) and VoVGSCSP (Volumetric Grid Spatial Cross Stage Partial) modules in the neck to further reduce computational complexity. Experiments were conducted on the indoor object subsets extracted from the PASCAL VOC and COCO datasets, as well as a custom indoor object dataset. The results show that, compared to YOLOv7, SBP-YOLOv7 reduces the number of parameters by 32.00, 28.23, and 32.00

The unsupervised domain adaptation(UDA)method aims to utilize a labeled data domain(source domain)to enhance the model's generalization ability in another unlabeled data domain(target domain).In the three-dimensional(3D)real-world context,significant differences in the geometry and distribution of data exist between the source and target domains.However,current 3D UDA methods have not paid sufficient attention to such domain gap issues,resulting in decreased predictive performance in the target domain.Therefore,a 3D UDA method based on balanced geometric perception is proposed.To achieve consistent underlying geometric information across domains,a self-supervised pretraining task based on implicit fields is designed,which involves training a point cloud implicit field using balanced local distances.Through this method,the model can fully leverage underlying geometric information and effectively learn implicit representations at varying densities,thereby mitigating the impact of outlier point cloud data.In addition,a point cloud hybrid enhancement strategy is adopted to interpolate the point cloud data and labels.This provides more intermediate state information for the point cloud data,increases the diversity of the training data,and further improves the model's generalization ability.Experimental results show that the proposed method achieves a mean intersection-over-union of 65.2%on the segmentation dataset PointSegDA and an accuracy of 71.8%on the classification dataset PointDA-10,demonstrating the effectiveness of the proposed method.

Point Cloud can be considered as non-Euclidean structure data since it is disordered and irregular. When training on point cloud, it is difficult to apply spatial discrete convolution directly. In this paper, we propose a novel three-dimensional spatial convolution operator called frame points attention convolution (FPAC). FPAC pre-defines a set of frame points in space and quantifies the correlation between the input local points and the frame points through an attention mechanism. FPAC then combines the quantified correlations with the weights of the frame points to generate spatially continuous filters. The convolution weights for different local areas in the filters are calculated dynamically, without relying on generative models or probabilistic assumptions. Furthermore, FPAC is reformulated to reduce the internal dimensions during training, which reduces memory consumption and significantly improves training speed. Several optimization measures are also implemented to further enhance the performance of FPAC. We built three common point cloud task networks using FPAC and conducted experiments to train these networks on widely used datasets. Experimental results show that the method proposed in this work is competitive with state-of-the-art methods for point cloud tasks.

Current methods for 3D shape recognition and retrieval utilize deep learning techniques, achieving commendable performance through a singular representation while neglecting the multi-modal information inherent to the same 3D object. Furthermore, certain approaches treat recognition and retrieval as distinct tasks; however, these processes should be synergistic rather than antagonistic. In this paper, we propose a multi-modal semantic embedding network designed to deliver a more comprehensive representation of 3D shapes, thereby enhancing recognition accuracy and retrieval efficacy. Initially, we employ two independent feature extractors to derive multi-view and point cloud features. Subsequently, we introduce a multi-modal feature fusion method that emphasizes uncovering correlations between diverse modal features while mitigating information degradation. Finally, we implement a joint learning strategy for the fused features that resolves modal heterogeneity and facilitates joint mapping of visual attributes with semantic labels. Extensive experiments on multiple datasets validate the superiority of our approach.

To address the issues of holes and regional deficiencies in weakly textured areas during multi-view stereo(MVS)three-dimensional(3D)reconstruction,this paper proposes a complementary 3D reconstruction method that fuses data from MVS and lidar.First,the GeoMVSNet deep learning network is used to process multi-view images captured by a smartphone camera,resulting in MVS depth maps.Next,the smartphone camera and lidar are calibrated to compute the internal and external parameters and the coordinate system transformation matrix.Through transformations of temporal and spatial consistency,viewing angles,and scale,the sparse point cloud collected by lidar is converted to the image perspective.In addition,a depth map enhancement algorithm is proposed and applied to generate dense point cloud data from the lidar.Finally,the dense depth maps generated by lidar and the MVS depth maps are fused.Experimental results show that the reconstruction quality is significantly improved in weakly textured areas on a self-built dataset when the proposed method is used,enhancing the accuracy and completeness of 3D reconstruction.Thus,this study provides an effective solution for 3D reconstruction in weakly textured regions.

Infrared and visible image fusion (IVF) endeavors to engineer composite outputs by blending optimal virtues of divergent modalities. This paper reveals, unprecedentedly, the intrinsic “attention properties” of infrared images, which directly arise from their physical characteristics (i.e., heat distribution) and can be linked to attention mechanisms naturally, as observed in the gradient-weighted class activation mapping (Grad-CAM) visualization analysis of image classification models. To incorporate this property into IVF for better fusion, we propose the source infrared cross attention (I-SCA) and further extend it to the visible modality, subsequently introducing the source visible cross attention (V-SCA). The joint use of I-SCA and V-SCA greatly alleviate longstanding issues, such as insufficient and incomplete multimodal feature interaction and fusion, in IVF. Moreover, an auxiliary component for I-SCA and VSCA, termed CBSM, is employed to boost the channel, map space, and suppress redundancy and misleading information of the source images. Specifically, we directly treat the CBSM-processed raw image as the query, while the intermediate features of another modality are treated as keys and values in I-SCA and V-SCA. Unlike attention mechanisms that divide images into patches or limit computations to local windows, our cross attention modules achieve smoother and more robust IVF through true global modeling across the entire image space with linear complexity. Comparison with current SOTA methods on three popular public datasets confirms its superiority.

Three-dimensional semantic segmentation plays a crucial role in accurately identifying terrain features and objects by effectively extracting 3D spatial information from the environment. However, the inherent sparsity of point clouds and unclear terrain boundaries in outdoor mining environments significantly complicate the recognition process. To address these challenges, we propose a novel 3D semantic segmentation network that incorporates spatiotemporal feature aggregation. Specifically, we introduced the Gated Spatiotemporal Clue Encoder, which extracts spatiotemporal context from historical multi-frame point cloud data and combines it with the current scan frame to enhance feature representation. Additionally, the Spatiotemporal Feature State Space Module is proposed to efficiently model long-term spatiotemporal features while minimizing computational and memory overhead. Experimental results show that the proposed method outperforms the baseline model, achieving a 2.1% improvement in mIoU on the self-constructed TZMD_NUC outdoor mining dataset and a 1.9% avg improvement on the public SemanticKITTI dataset. Moreover, the method simultaneously improves computational efficiency, making it more suitable for real-time applications in complex, real-world mining environments. These results validate the effectiveness of the proposed method, offering a promising solution for 3D semantic segmentation in complex, real-world mining environments, where computational efficiency and accuracy are both critical.