In transportation systems, accurately estimating the remaining useful life (RUL) of critical components, such as aircraft engines, Battery Management Systems (BMSs), is crucial for the safe and reliable operation and manufacturing of transportation vehicles. However, most existing research overlooks the underlying physical information, which is vital for more precise RUL prediction. To fill this gap, this paper proposes a physics-informed method for predicting the RUL of key components of transportation vehicles. By integrating the Mamba network with a multi-head attention mechanism, we capture and emphasize key features and trends in the equipment’s operational state, improving prediction accuracy. Additionally, we introduce a Physics-Informed Neural Network (PINN) framework to model the underlying physical relationships between RUL and sensor data, incorporating these relationships as a regularization term in the loss function to enhance predictive capability and interpretability. We conducted experimental validation using the C-MAPSS aircraft engine dataset (operation) and the transportation vehicle chip manufacturing dataset (manufacture). The results show that the proposed method significantly improves the accuracy of RUL prediction, providing strong support for the intelligent maintenance and reliability management of key components in transportation vehicles.

In industrial bin-picking, robotic systems must estimate the poses of multiple object instances, where accurate pose estimation is essential for reliable downstream manipulation and grasping. Most existing multi-instance registration methods primarily establish point correspondences based on local features to alleviate the challenges posed by occlusion and clutter. However, local features are easily disturbed by neighboring instances and lack global context, leading to unreliable correspondences and degraded registration accuracy. In addition, the absence of rotational invariance further reduces correspondence accuracy in scenes with stacked instances and highly varying object orientations. To address these challenges, we present a one-stage multi-instance point cloud registration framework for stacked-object scenes. Our framework incorporates a rotation-invariant operator to enhance the robustness of feature representations under arbitrary orientations. Then, we propose a Center-Aware Res-Masked Transformer module, which incorporates an object center embedding to enrich global instance-level context and a center-aware residual mask prediction module to balance weight distribution across objects of varying sizes during training. Extensive experiments on the challenging ROBI dataset demonstrate that our method outperforms the competitive baseline MIRETR by more than 10% in mean precision, highlighting its effectiveness in complex bin-picking scenes. Furthermore, evaluations on the unstacked Scan2CAD dataset confirm the generalizability of the proposed framework across different application scenarios.

Open-vocabulary 3D scene graph generation aims to predict 3D objects and their predicates beyond the annotated label space. Compared to closed-set 3D scene graph generation methods, the open-vocabulary approach is more general, practical, and less dependent on labor-intensive ground truth annotations. Existing open-vocabulary 3D scene graph generation methods rely on learning individual object and predicate features in the representation space while ignoring higher-level 3D scene representations, leading to overfitting and suboptimal performance. In this work, we propose a hyperbolic learning-based approach to address this problem by leveraging hyperbolic geometry to learn hierarchical 3D scene representations in the form of scene-region-instance, where the scene represents the complete 3D environment, a region contains related instances, and an instance corresponds to an individual object or predicate. Specifically, our method decomposes a 3D scene into a discrete hierarchy consisting of scene, region, and instance nodes, and embeds this hierarchy into a learned hyperbolic representation space. The learned hyperbolic embeddings are optimized in a bottom-up manner, where higher-level nodes are derived from their corresponding child nodes. The learned hierarchical 3D scene representations are incorporated as structural and semantic guidance for open-vocabulary 3D scene graph generation. We further observe that outliers in the form of erroneous hyperbolic embeddings can negatively impact hierarchical reasoning. To mitigate their negative impact, we present an enhancement strategy that learns an adaptive distance metric robust to the outliers over the learned hyperbolic representation space and subsequently improves overall performance. Extensive experiments on 3DSSG and ScanNet datasets demonstrate the effectiveness of our method in 3D scene graph generation under closed-set, open-vocabulary, and zero-shot settings.

Estimating the pose of unseen objects is a fundamental task in robotics and industrial automation. Some methods rely on prior knowledge of individual objects for this task and require the model to be trained on specific object instances or categories. Other methods can estimate the pose of unseen objects but are often limited in handling occlusions. To address these challenges, we propose an unseen object pose estimation method for objects not encountered during training. We propose a Hashing Attention Network that integrates features from multiple scales to effectively capture global information while maintaining high sensitivity to local positional details, thereby significantly enhancing the model’s predictive accuracy for occluded objects. We incorporate Locality Sensitive Hashing into the self-attention mechanism of the vision transformer. This approach significantly reduces computational complexity by computing attention for tokens at each scale based on hash similarity, both within the same scale and across multiple scales. The network was trained on selected BOP datasets, and the Linemod, T-LESS, and Wild6D datasets were used to evaluate pose estimation on unseen objects. Our results show that the proposed model outperforms existing methods across various quantitative metrics, making it well suited for industrial applications.

Stereo matching is an important part of robot vision intelligent environment perception. Currently, reflective regions and large textureless regions, repetitive texture regions, and depth discontinuity regions are still difficult and error-prone for stereo matching tasks. We propose a novel stereo matching network that can effectively solve the above problems. First, to address the difficulty in reflective regions and large textureless regions, we propose the illumination invariance feature enhancing module which can promote the feature extraction network to extract more sufficient features in reflective regions and large textureless regions. The illumination invariance feature enhancing module is a differentiable transformation of the Census-based matching cost computation and can be combined with multiple stereo networks to improve their performance in difficult regions. Then to address the difficulty in repetitive texture regions and depth discontinuity regions, we propose the differentiable trilateral feature aggregation module to make full use of the prior geometric knowledge of color similarity, spatial distance, and edge strength in the scene to enhance the aggregation ability of cost agregation network in depth discontinuity regions and repetitive texture regions. The proposed network is end-to-end and is verified by extensive experiments. The results show that the proposed method can effectively solve the problems of stereo matching in the above difficult regions and has achieved the competitive performance on Scene Flow datasets, KITTI 2012 datasets, and KITTI 2015 datasets.

Cross-view image geo-localization aims to estimate a camera's 3-DoF pose (location and orientation) by comparing the captured ground-level image to a geo-referenced overhead satellite image that covers the camera's local surroundings. However, this task remains challenging due to the significant appearance and viewpoint discrepancies between ground and aerial images in dense urban environments, which often lead to substantial pose uncertainty. To overcome this challenge, we propose a novel label-adaptive smoothing method with a multiscale iterative strategy that integrates uncertainty-aware label distribution modeling into the training process at multiple spatial resolutions, enabling more robust alignment of cross-view features and improving the accuracy of the pose estimation. Specifically, we design a fine-grained feature alignment module to predict both the mean and variance of the continuous pose distribution, which are then used to compute an unimodal-concentrated loss. However, due to varying levels of prediction uncertainty across different samples, a fixed weighting between the unimodal loss and the concentrated loss is insufficient to capture such dynamics, which may lead to unstable or low-confidence predictions. Therefore, we introduce a negative feedback-based label distribution refinement module, which adjusts the relative strength of the unimodal-concentrated loss based on the validation-set mean absolute pose error to adaptively balance the contributions of the unimodal loss and the concentrated loss, thereby enabling confident and precise pose predictions in complex urban environments. Extensive experiments on VIGOR and KITTI demonstrate that our method reduces median localization error by 41% over strong baselines, and achieves 53.05% and 63% reductions in distributional localization and orientation errors.

Cross-modal homography estimation is a fundamental task in computer vision. However, the significant modality gap between multimodal images poses a considerable challenge for this task. In this paper, we propose a novel framework, named Disentangled Representation Learning for Cross-modal Homography Estimation (DRLHomo), to address this challenge. Specifically, we design a feature disentanglement network to explicitly extract modality-shared and modality-unique features of image pairs using shared and unique feature encoders, respectively. Similarity loss and difference loss are employed to supervise this process. Furthermore, to prevent pathological disentanglement, where shared features are dominated by noise and unique features capture all modality information, we incorporate a decoder with reconstruction loss to enhance feature disentanglement. Ultimately, the homography is estimated based on the obtained modality-shared features. Extensive experiments on the GoogleMap and GoogleEarth datasets validate the effectiveness of DRLHomo in mitigating the modality gap and improving cross-modal homography estimation.

This paper proposes a robust image-based control framework for an aerial manipulator (AM) performing infrastructure contact inspection under strong coupling, uncertainties, and external disturbances. A decoupled second-order image dynamics model is derived from virtual-camera image moments. Building on this model, a disturbance observer-based nonlinear model predictive controller (NMPC) is developed to generate position commands, where disturbances are estimated online, and both physical limits and visibility constraints are explicitly enforced. To achieve reliable force interaction, an image-based first-order variable-stiffness impedance controller is designed for contact-direction force-tracking, and a geometric attitude controller based on a commanded filter is adopted to avoid Euler angle singularities. Simulation and experimental results demonstrate improved tracking accuracy and robustness. The contact force RMSE is reduced by 31.6% compared with a representative robust NMPC baseline in simulation, and by 58.3% compared with a hybrid visual compliant baseline in experiments, across nine repeated trials; the average force RMSE reduction reaches 52.9%. These results verify the force-tracking accuracy and reproducibility of the proposed framework in aerial physical interaction (APhI) tasks.

For the safety of autonomous driving systems, recognizing the gestures of cyclists and controllers on traffic roads is crucial. We propose a novel two-stream graph convolutional network (2S-GCN) to address these challenges. First, we introduced a motion-guided module connecting the motion and joint stream. By leveraging the motion stream's spatial expression capability, we guided the joint stream's learning in spatial dimensions to enhance the network's performance to short action durations. Second, we utilized a simplified skeleton topology that exclusively captures the upper body during data preprocessing, thereby preserving essential information while eliminating redundancy. This approach enhances the network's adaptability to atypical action postures. Third, we use a GCN embedded with a multichannel attention module as the backbone, which is particularly suitable for small datasets, and conduct extensive experiments on two datasets. The experimental results demonstrate that our network can accurately recognize traffic gestures and has significant advantages over state-of-the-art methods.

Estimating the 6-D pose of transparent objects remains a challenging task due to weak visual cues and incomplete depth information caused by refractions and additional reflections from the backsides of transparent objects. These challenges often lead to inaccurate perception and localization. Existing approaches struggle to jointly enhance RGB and depth features or fully exploit depth information. In this article, we propose a depth-directed method that integrates stable diffusion and Mamba-based RGB-D augmentation to address these limitations. Specifically, we use a stable diffusion model to enhance the visibility of transparent objects in RGB images by recovering texture details and reducing background interference. Then, a Mamba-based network completes the sparse and noisy raw depth maps using guidance from the refined RGB features. To effectively fuse the two modalities, we introduce a multiscale RGB-D fusion strategy, where the completed depth not only provides geometric information but also guides RGB feature extraction. This joint representation leads to more accurate and robust 6-D pose estimation. Experimental results on challenging datasets demonstrate that our method significantly enhances object visibility and improves pose estimation accuracy in complex real-world scenes.

Image stitching plays a pivotal role in the quality inspection of printed circuit boards (PCBs). However, this process encounters significant challenges owing to the limited overlapping regions between the stitched images. Existing methods aim to alleviate the influence of features from nonoverlapping regions on network performance by regularizing the loss function for these regions. Nevertheless, this approach incompletely eradicates interference from nonoverlapping regions. Moreover, PCB images often comprise numerous homogeneous regions, where indistinguishable semantic information can result in erroneous alignments between feature points. In this article, we propose a novel framework from the perspective of learning disentangled representations. Through an image reconstruction task, we disentangle the matching features of overlapping regions from the interfering features of nonoverlapping regions, thereby attaining purified matching features and eliminating noncontributory features. Specifically, we partition the encoded features of the image into shared matching features from overlapping regions and interfering features from nonoverlapping regions. By exchanging the shared features between image pairs to reconstruct the feature representations, we utilize these reassembled features to reconstruct the input image pairs. Furthermore, to address the ambiguity of homogeneous features in overlapping regions, we devise a frequency contextual correlation module (FCCM). The frequency convolution kernels in this module compute correlations between pixel blocks rather than individual pixels. Consequently, FCCM adeptly captures contextual semantic information between feature maps, resolving ambiguities induced by feature points in homogeneous regions. Extensive experimentation demonstrates that our approach effectively segregates matching alignment features from the overall image features while mitigating the interference of homogeneous region features. This contribution enhances alignment accuracy and achieves a 100% stitching success rate for PCBs, satisfying industrial requirements.

Gesture recognition constitutes a core modality for human-robot interaction (HRI). Current studies primarily address short-distance scenarios, whereas long-distance outdoor scenarios remain significantly underexplored. The principal technical challenges arise from three key challenges: First, the predominant separated spatio-temporal processing approach in current network architectures and attention mechanisms. 1) introduces redundant structural complexity and computational demands, severely constraining unmanned aerial vehicle (UAV) platform implementation. 2) Inherently limits the model's ability to learn weakly-correlated spatio-temporal relationships. 3) Additionally, existing HRI systems demonstrate inherent deficiencies in accurate hand localization and skeleton estimation under long-distance conditions. To address these issues, we propose a lightweight spatio-temporal skeleton (STS) attention transformer for long-distance gesture recognition in UAV control. First, we designed and implemented a lightweight STS Transformer to unify spatial-temporal feature processing while maintaining structural simplicity. Second, we developed an STS attention employing coupled-decoupled operators on skeleton sequences to extract weakly-correlated features. Subsequently, we developed a two-stage RGB+D framework for hand detection and skeleton estimation, achieving capability at extended ranges up to 5 m under UAV onboard camera view and 10 m in third-party camera view. Finally, extensive experiments on two datasets and real-world experiments were conducted to demonstrate the effectiveness of our network in skeleton-based gesture recognition, showing a clear advantage over the state-of-the-art (SOTA) methods.

Stereo matching is of great importance in robot operation, autonomous driving and virtual reality. Large textureless regions and depth discontinuity regions are still the error-prone regions of stereo matching tasks. Traditional correlation-based volumes only measure the feature similarity within the same channel of the feature maps, resulting in insufficient feature similarity learning between different channels, which leads to poor performance of stereo networks in large textureless regions with high feature similarity requirements. To address the problems in large textureless regions, we propose the cross-correlation based cost volume construction which adequately learn the feature similarity in different channels of the feature maps. To address the problems in depth discontinuity regions and other gradient sensitive regions, we propose the differentiable least square aggregation module which can sufficiently utilize the gradient information and enhance the aggregation ability of the cost aggregation network for gradient features. Extensive experiments show that the proposed method solves the problems effectively in the above difficult regions and achieves competitive performance on Scene Flow dataset, KITTI 2012 dataset and KITTI 2015 dataset.

In the automated inspection of photovoltaic (PV) power plants, image acquisition is typically carried out using drones equipped with infrared cameras. However, each image usually covers only a small portion of the plant, and issues such as low resolution, poor contrast, and thermal noise make precise localization of PV modules challenging. To address these limitations, this article proposes a precise PV module localization method. First, the open-source tool OpenDroneMap (ODM) is employed to stitch and georeference the acquired images and to extract PV strings based on YOLOv8. Then, we introduce the segment anything model (SAM)-unidirectional histogram (UDH) method, which integrates the SAM with a UDH. After background removal using SAM, a perspective transformation is applied for distortion correction, followed by UDH analysis to extract module boundaries. Finally, inverse mapping is performed back to the stitched orthophoto to achieve accurate localization in complex environments. The proposed approach was validated on real infrared datasets covering 441 PV strings and 12342 PV modules. Experimental results show that our method achieves an F1-score of 96.4% and an IoU of 93.0%, while maintaining both the standard deviation (SD) and confidence interval (CI) below 0.03, demonstrating superior accuracy and robustness.

Homography estimation is a fundamental task in computer vision and robotics, with broad applications across various domains. However, existing methods often struggle under significant geometric deformations between image pairs, resulting in reduced estimation accuracy. In this paper, we propose a novel homography estimation framework, termed Deformable Adaptation and Dual-view Interaction Network (DADINet), to address this challenge. Specifically, we design a Deformable Adaptation Module (DAM) that leverages the learnable receptive fields of deformable convolutions, enabling the model to flexibly adapt to geometric deformation and better capture non-rigid variations. In addition, we introduce a Dual-view Interaction Module (DIM) to enhance spatial correspondence and facilitate cross-view contextual information interaction, thereby improving both matching performance and homography estimation accuracy. Extensive experiments conducted on multiple datasets demonstrate that the proposed DADINet outperforms existing state-of-the-art methods.

Homography estimation is fundamental to various vision tasks. Iteration-based methods have recently achieved significant success in this field. However, errors introduced during iterations can lead to increased image deformation. Existing methods often focus on capturing local correspondences in the later stages of iteration while downplaying global ones, which may cause errors to persist and propagate into subsequent iterations, ultimately leading to error accumulation. To alleviate this issue, we propose Multi-range Adaptive Perception Transformer for Iterative Homography Estimation (MAPTHomo), which integrates Multi-range Attention (MRA) and Adaptive Perception Module (APM). Specifically, MRA captures both global and local correspondences, enabling the model to adapt to varying levels of deformation. The APM dynamically adjusts attention focus based on the current context. The combination of MRA and APM enhances the error-correction capability of the iterative process, effectively mitigating error accumulation. Extensive experiments demonstrate that MAPTHomo outperforms previous methods and exhibits strong generalization ability.

Visual understanding of 3-D objects is essential for robotic manipulation, autonomous navigation, and augmented reality. However, existing methods struggle to perform this task efficiently and accurately in an end-to-end manner. We propose a single-shot method based on the state space model (SSM) to predict the full 3-D information (pose, size, shape) of multiple 3-D objects from a single RGB-D image in an end-to-end manner. Our method first encodes long-range semantic information from RGB and depth images separately and then combines them into an integrated latent representation that is processed by a modified SSM to infer the full 3-D information in two separate task heads within a unified model. A heatmap/detection head predicts object centers, and a 3-D information head predicts a matrix detailing the pose, size and latent code of shape for each detected object. We also propose a shape autoencoder based on the SSM, which learns canonical shape codes derived from a large database of 3-D point cloud shapes. The end-to-end framework, modified SSM block and SSM-based shape autoencoder form major contributions of this work. Our design includes different scan strategies tailored to different input data representations, such as RGB-D images and point clouds. Extensive evaluations on the REAL275, CAMERA25, and Wild6D datasets show that our method achieves state-of-the-art performance. On the large-scale Wild6D dataset, our model significantly outperforms the nearest competitor, achieving 2.6% and 5.1% improvements on the IOU-50 and 5∘10 cm metrics, respectively.

Precise prediction of grasp configurations is essential for robotic manipulation, including tasks like grasping and assembly, especially in systems that rely on visual input. Although CNN and Transformer based approaches have shown strong performance in controlled or structured environments, their effectiveness often deteriorates in complex scenes due to limited robustness and substantial computational overhead, which hinders deployment on real-time and mobile robotic platforms. To overcome these challenges, we introduce EfficientVMamba-Pose, a compact visual state-space network designed for efficient planar grasp pose estimation. Our method leverages a novel interval-sampling selective scanning mechanism to build expressive local feature representation blocks and applies a lightweight fusion strategy to integrate multi-scale features. Additionally, we design an enhanced visual state-space module by integrating state-space dynamics with convolutional attention, improving the network’s capacity to represent both long Range dependencies and local geometric structures.Extensive experiments demonstrate that EfficientVMamba-pose achieves superior accuracy while significantly reducing computational complexity. On the Cornell and Jorge grasping datasets, our approach achieves grasp success rates of 98.35% and 95.5%, respectively. These results highlight the model’s effectiveness and strong potential for practical applications in robotic grasp pose estimation under real-world constraints.

Registration of multiview point clouds obtained from 3D scanners is a common method for 3D reconstruction. However, most existing registration methods are designed to handle point clouds with known overlap relationships that are ensured by external equipment (e.g., manipulators, turntables) or acquisition sequences, which limits the application range and increases the acquisition cost. To overcome these limitations, an unknown overlap registration (UOR) method for multiview point clouds is proposed, which can estimate overlap confidence, construct a connected graph, and remove outlier point clouds automatically. First, the overlap confidence between two point clouds is estimated by calculating the average nearest neighbor feature distance within the predicted overlap region. We then construct a minimal spanning tree based on the confidence levels and search for the central node to serve as the world coordinate. Finally, the Lie algebra-based SE(3)-sensitive perturbation scheme is introduced to solve the fine transformations, in which a robust weighting function is designed to weight point correspondences. Our method can find reliable connections among point clouds, and the proposed graph can be combined with different pairwise registration methods. The experimental results on both indoor and industrial datasets demonstrate the accuracy and effectiveness of our method.