Surface defects are a major cause of railway failures and pose serious safety risks, making accurate defect detection essential. However, existing methods often exhibit limited performance due to the scarcity of defect samples and the complexity of operating environments. To address these challenges, this paper proposes a sample self-generation-based framework for rail surface defect detection in extremely data-scarce scenarios. A multi-mode defect generation model is introduced, enabling effective data augmentation and style transfer using only one or two real samples. By decoupling the generation process into a Feature Learning Network with a sub-discriminator architecture and a Sample Generation Network, the proposed method achieves high sample diversity and quality with low computational cost. The generated samples are used to train YOLO-based detectors. Experiments show that the proposed approach improves mAP@0.5 by over 8 percentage points on YOLOv9, outperforming models trained with twice the amount of real data.

The tribovoltaic effect, which directly converts mechanical energy into electricity at dynamic semiconductor interfaces, presents a novel pathway for intrinsic mechanical sensing. Here, we materialize the "surface-assensor" paradigm by harnessing this effect to create a self-sensing bearing. This is achieved through a spontaneously formed molybdenum disulfide (MoSS) layer on the bearing races, which establishes dynamic Schottky contacts with the steel balls, effectively turning the bearing surface itself into the sensing element. Mechanical friction at these interfaces directly generates a continuous stream of real-time electrical signals via the tribovoltaic effect. We analyze the underlying mechanism of this effect within the dynamic bearing system and demonstrate that the magnitude and characteristics of the resulting tribovoltaic outputs are intrinsically modulated by operational conditions, serving as highly sensitive and direct fingerprints of rotational speed and load. By employing explainable machine learning and deep learning to decode these signals, we achieve accurate monitoring of operational states, such as rotation speed, with the identification accuracy exceeding 96%, and further demonstrate its capability in early fault diagnosis. This work successfully establishes the tribovoltaic effect as a practical and powerful sensing mechanism, validating the "surface-as-sensor" concept as a pathway to a new generation of embedded, self-sensing intelligent machinery.

Pipe alignment is a necessary step for automatic pipe assembly. Image-based visual servoing (IBVS) offers an effective solution for achieving automatic pipe alignment. Due to the complex pose of the pipe, the conventional teach-by-show (TBS) mode in IBVS fails to obtain the accurate desired image. Additionally, dynamic noise and the texture-less surface of the pipe make feature extraction difficult. In this work, we replace the entire pipe with a local two-segment pipe and introduce an IBVS system for pipe assembly. We use a virtual camera to calculate the desired image to avoid manual teaching. To simplify the desired image by excluding arc information, we design a feature composed of four lines and a point. For the pipe geometry, only the pipe diameter and bending angle are required as prior knowledge, while the virtual-camera distances are preset to define the desired image. Then, a feature extraction method is developed to provide stable feedback through semantic segmentation, line fitting and classification, and point feature enhancement. The alignment experiments conducted on pipes of different shapes demonstrate the effectiveness of the proposed method.

Deep-learning-based defect detection models require large and diverse datasets, yet obtaining sufficient defect samples in industrial environments is challenging. Existing sample-generation-based methods alleviate data scarcity, but still depend on a certain amount of training data and struggle to enforce the strict structural specifications of industrial products, limiting their applicability. To address these issues, we propose a sample-generation-based defect detection model. The core of our approach is a defect generation network that employs multiscale progressive learning to extract multiresolution features and enables augmentation from a single sample. To satisfy structural constraints without suppressing diversity, we introduce a structural attention mechanism that guides the generative process rather than imposing direct mask-based restrictions. Additionally, we design a simple yet effective foreground—background reconstruction loss that better preserves structural details compared with conventional reconstruction losses. Our method requires only a single sample to initiate data augmentation. As a result, collecting a small number of representative defect samples is sufficient to significantly enhance detection performance, and the low data requirement allows broad applicability across diverse industrial defect detection tasks. Experimental results demonstrate that our method outperforms existing models, with improved sample quality and detection performance, achieving up to a 28% and 20% increase in mAP@0.5 on the DeepPCB and NEU-DET datasets, respectively. Our method proves even more effective when the sample size is limited.

Surface defect detection plays a crucial role in industrial manufacturing. Due to the scarcity of defect samples, traditional deep learning models face significant limitations in detection performance and generalization ability. To address this issue, this paper proposes a single-image orthogonal weights generative adversarial network (SIOWGAN), a multi-generator adversarial generative model designed to effectively augment defect samples under limited sample conditions. This method requires only a single sample for data generation. The multi-generator structure, together with weight-orthogonality regularization, mitigates mode collapse and improves sample diversity under low-data conditions. Compared with existing multi-generator architectures, we propose a novel weight orthogonality training strategy, which guides different generators to learn distinct data distribution modalities without introducing any additional models. This design avoids the computational overhead and training instability that typically arise from incorporating extra models. Experimental results on the NEU-DET metal surface defect dataset demonstrate that the proposed method can generate diverse defect samples that align with the real data distribution using only 4% of the original data. The Fr & eacute;chet inception distance and IS scores of the proposed method outperform those of existing multi-generator and single-image generation baseline models. The augmented samples are used to train YOLOv10, YOLOv11, YOLOv12, and YOLO26 detectors. Detection performance improves substantially, and in several settings mAP@0.5-0.95 approaches or exceeds the model trained on the full training split. Additional ablation studies validate the effectiveness of the proposed two loss functions in promoting sample diversity and maintaining realism. In conclusion, SIOWGAN provides a scalable data augmentation approach for industrial defect detection tasks under extremely limited sample conditions.

Precise and fast pipe instance segmentation is a critical component in industrial automatic assembly, facilitating accurate object detection and pose estimation, optimizing and supervising the assembly process. However, this problem is challenging due to topological errors on fine-scale structures caused by the pipes being complex and slender. To address these challenges, we propose a topology-constrained pipe instance segmentation network (TopoPIS) for complex stacking scene to achieve accurate segmentation with topological correctness. To better extract the features of complex and variable morphological pipes, adaptive curvature convolution is introduced to dynamically adapt to the slender pipe structure and capture critical features. To handle topological errors like broken connections, we propose a novel topological constraint loss function based on persistent homology, which greatly improves the topological continuity of the segmentation. Experimental results on real-world and unseen datasets demonstrate that our TopoPIS outperforms other methods regrading segmentation accuracy and topological continuity.

Reconstructing three-dimensional (3D) shapes from a single image remains a significant challenge in computer vision due to the inherent ambiguity caused by missing or occluded shape information. Previous studies have predominantly focused on mesh models supervised by multi-view silhouettes. However, such methods are limited in reconstructing fine details. In this study, a 3D mesh model is predicted from a single image, leveraging depth consistency and without requiring viewpoint pose annotations. The model effectively learns strong shape priors that preserve finer structures and accurately predicts view poses from "correlation-supervised" viewpoints. Additionally, standard deviation and Laplacian losses were employed to regulate mesh edge distribution, resulting in more precise reconstructions. Differentiable renderer functions were derived from the 3D mesh to generate depth maps. Compared to conventional approaches, the proposed method provided superior representation of subtle structures. When applied to both synthetic and real-world datasets, the model outperformed existing methods in view-based 3D reconstruction tasks.

Pipe pose estimation provides crucial positional information for robots, enhancing assembly efficiency and precision, while its accuracy critically impacts the final product's reliability and quality. To handle unseen pipes, we propose a category-level pipe pose and size estimation network via Normalized Object Coordinate Space (NOCS) representation. Given an RGB image and its corresponding depth map, our network predicts class labels, bounding boxes and instance masks for detection, as well as NOCS maps for pose estimation. Then these predictions are aligned with the depth map to estimate pipe's pose and size. To better extract complex and variable pipe morphology, geometry-aware adaptive curvature convolution is introduced to dynamically adapt to the slender structure and improve segmentation performance. Facing the lack of pipe pose datasets with enough instances, pose, clutter, occlusion, and illumination variation, we propose a novel domain randomization mixed reality approach to efficiently generate synthetic data, which addresses the limitations of training datasets, making data generation more time- and effort-efficient. Experimental results demonstrate that our GeometryAware Adaptive Convolutional Network (GACNet) outperforms other methods and robustly estimates the pose and size of unseen pipes in real-world environments.

The polarization reflection model is a fundamental framework describing the changes in both intensity and polarization of light following reflection. Materials with distinct properties exhibit unique polarization reflection characteristics. However, traditional models assume that the scattered light from diffuse reflection is completely depolarized after undergoing internal scattering, regardless of the scattering properties of the object. In reality, multiple scattering events do not completely depolarize the scattered light. To address this discrepancy, we first established a reference coordinate system for the scattering process. Subsequently, we employed polar decomposition methods to analyze the subsurface scattering Mueller matrix. Building upon this analysis, we introduced polarization Monte Carlo simulations to empirically construct a novel polarimetric bidirectional subsurface scattering reflectance distribution function (pBSSRDF) model. We also explored the relationship between the depolarization and scattering coefficient and developed a comprehensive optimization process for the object's pBSSRDF parameters, thereby avoiding the limitations associated with coaxial and coplanar assumptions. Our experiments, conducted on both synthetic and real data for opaque and translucent objects, demonstrate high accuracy in representing the polarization states of scattered light.

In this article, we focus on constructing a multicategory railway defect detection method, which is important in both railway operation and railway maintenance. We designed a deep learning-based railway defect detection system that includes railway classification, switch spacing measurement, and railway defect sample expansion based on generative adversarial networks (GANs) and railway defect detection networks. Deep learning can build fast and accurate defect detection networks; however, its application in railway scenarios is limited due to the scarcity of defect samples and usually focuses on single-type defect detection. We build a railway defect detection system by balancing positive and negative samples, contactless switch spacing measurements, generating railway defect samples, and transfer learning. We conduct experiments to show that such an approach can be well applied to real railway detection, which greatly solves the problem of low sample size and low generalization of deep learning in railway scenarios. Furthermore, due to our method does not have any requirements for the types and scenarios of railways, most deep learning-based railway defect detection methods can be improved based on our method, which can reduce the difficulty of applying deep learning to railway defect detection. We hope this can promote the research of railway defect detection.

Fringe projection profilometry (FPP) has been widely applied to non-contact three-dimensional measurement in industries owing to its high accuracy and speed. The point cloud, which is a measurement result of the FPP system, typically contains a large number of invalid points caused by the background, ambient light, shadows, and object edge regions. Research on noisy point detection and elimination has been conducted over the past two decades. However, existing invalid point removal methods are based on image intensity analysis and are only applicable to simple measurement backgrounds that are purely dark. In this paper, we propose a novel invalid point removal framework that consists of two aspects: (1) A convolutional neural network (CNN) is designed to segment the foreground from the background of different intensity conditions in FPP measurement circumstances to remove background points and the most discrete points in background regions. (2) A two-step method based on the fringe image intensity threshold and a bilateral filter is proposed to eliminate the small number of discrete points remaining after background segmentation caused by shadows and edge areas on objects. Experimental results verify that the proposed framework (1) can remove background points intelligently and accurately in different types of complex circumstances, and (2) performs excellently in discrete point detection from object regions.

Many deep learning based railway defect detection methods have been proposed in recent years. They have greatly improved the efficiency and accuracy of defects detection. However, detection of railway defects remains challenging because of the limited number and types of defect samples, thus, general deep learning methods cannot be applied. In this paper, we designed a “Perceptually Constrained Single Image Generative Adversarial Network” (PCSGAN) to expand the number of railway defect image samples. PCSGAN uses a pyramidal structure to learn the internal features of a single image. In addition, we proposed a masking and a perceptual reconstruction loss mechanism to impose specific positional and structural constraints on the images. We tested the method using railway defects images and compared it to other single image generation models. The experiment results show that the images generated by PCSGAN take into account railway prior knowledge, generate railway structure which satisfied the constraints imposed by railway infrastructure designs, and also provide new information. High image realism and lowest Single Image FID were obtained, and the effectiveness of PCSGAN in the defect detection task were also validated.

Camera calibration is very important when planning machine vision tasks. Calibration may involve 3D reconstruction, size measurement, or careful target positioning. Calibration accuracy directly affects the accuracy of machine vision. The parameters in many image distortion models are usually applied to all image pixels. However, this may be associated with rather high pixel reprojection errors at image edges, compromising camera calibration accuracy. In this paper, we present a new camera calibration optimization algorithm that features a step function that splits images into center and edge regions. First, based on the increasing pixel reprojection errors according to the pixel distance away from the image center, we gave a flexible method to divide an image into two regions, center and boundary. Then, the algorithm automatically determines the step position, and the calibration model is rebuilt. The new model can calibrate the distortions at the center and boundary regions separately. Optimized by the method, the number of distortion parameters in the old model is doubled, and different parameters represent different distortions within two regions. In this way, our method can optimize traditional calibration models, which define a global model to describe the distortion of the whole image and get a higher calibration accuracy. Experimentally, the method significantly improved pixel reprojection accuracy, particularly at image edges. Simulations revealed that our method was more flexible than traditional methods.

To address the problem such as complex operation and poor usability in the traditional fringe projection measurement method, a flexible fringe projection measurement model based on phase height mapping was proposed. The mapping relationship from absolute phase value to the spatial coordinates was built by the measurement model, without considering the geometric constraints between the camera and the projector. The proposed calibration method needed neither the geometric constraint relationship between the camera and the projector, nor high-precision auxiliary tools such as gauge block or high-precision displacement stage. It only needed a checkerboard calibration board to complete the system calibration. The measurement system had the advantages of simple structure, high efficiency, high accuracy and good usability. Simulation and experiments had been performed to validate the effectiveness of the measurement model.

Accurate pipe pose estimation plays a pivotal role in the automatic assembly of pipelines. Recently, data-driven deep neural networks have been proven capable of estimating pose. Nonetheless, a large number of labeled datasets are required during the training process. One effective solution is to estimate pose using self-supervised learning. However, existing algorithms are difficult to deal with textureless objects (like pipes), and they avoid the occlusion problem. To this end, in this article, we propose a latent representation self-supervised pose network (LSPN) for accurate monocular pipe pose estimation. We train our network with synthetic RGB (Red, Green, Blue) data, where only a few labeled samples are used to establish the latent pose space, whereas a large number of structured unlabeled samples are used to learn latent pose representation in self-supervised learning. Experiments demonstrate that LSPN achieves excellent performance on real data and is robust to different environments, such as illumination changes and self-occlusion.

针对目前铁路道岔人工检测方法效率低、精度差的问题,提出一种基于深度学习的快速识别道岔场景及检测道岔间距的方法.采用线阵工业相机扫描获取铁路点云信息,设计残差连接的铁路道岔场景识别网络,采用树结构Parzen估计算法搜索最优超参数,采用Focal Loss损失函数解决样本数量不均衡问题,实现铁路道岔场景的准确快速识别.利用识别出的铁路道岔场景图像,开发了一种道岔基本轨与尖轨的边缘提取算法,准确测量道岔基本轨与尖轨的内侧间距.实验结果证明,该方法的识别准确率达到97.5％,识别时间在0.02 s内,道岔间距计算误差小于0.2 mm,相比人工检测方法,检测效率与检测精度均大幅提升,满足道岔检测的要求.

针对航空航天中大量应用的管路高精度快速测量难题,提出了基于蛇模型的管路三维重建方法.该方法充分利用管路灰度图像边缘特征,借助阈值分割和高斯滤波消除图像噪声,先结合蛇模型获取精准的管路图像边缘,贪心算法进行迭代以保证收敛效率,利用极线约束原理得到二维管路中心线点云,然后根据视觉投影原理,重建管路三维模型,最后使用多目视觉系统拍摄的照片进行结果优化以保证重建精度.结果表明:该方法对管路的测量精度为±0.15 mm,简便高效,无需过多的人工操作.同时,实现了管路圆弧段弯曲半径的测量,满足工业应用需求.

In recent years, addressing ill-posed problems by leveraging prior knowledge contained in databases on learning techniques has gained much attention. In this paper, we focus on complete three-dimensional (3D) point cloud reconstruction based on a single red-green-blue (RGB) image, a task that cannot be approached using classical reconstruction techniques. For this purpose, we used an encoder-decoder framework to encode the RGB information in latent space, and to predict the 3D structure of the considered object from different viewpoints. The individual predictions are combined to yield a common representation that is used in a module combining camera pose estimation and rendering, thereby achieving differentiability with respect to imaging process and the camera pose, and optimization of the two-dimensional prediction error of novel viewpoints. Thus, our method allows end-to-end training and does not require supervision based on additional ground-truth (GT) mask annotations or ground-truth camera pose annotations. Our evaluation of synthetic and real-world data demonstrates the robustness of our approach to appearance changes and self-occlusions, through outperformance of current state-of-the-art methods in terms of accuracy, density, and model completeness.