Objective Image transmission through multimode fiber (MMF) is now widely used in medical imaging, biological tissue detection, communication technology, and other fields. In multimode fiber imaging, the light pulse carrying the spatial information of the object enters the multimode fiber, and thousands of transmission modes excited in the fiber form encoded spatial information. Due to the complex mechanisms of interference, coupling, self-phase modulation, and group velocity dispersion among the fiber modes, the exit end of the fiber eventually forms a speckle image. With the development of optical modulators and computational optics, the advantages of deep learning methods in image reconstruction have become increasingly prominent. The high operational efficiency and strong resistance to fiber disturbances have pushed MMF image transmission into practical applications. Most existing studies use the MNIST handwritten digit set (28x28 resolution) for both training and testing, which is insufficient to train the generalization ability of network models. This reliance on limited data reduces the practical performance of the models. To enhance the practical application of multimode fiber imaging, we propose a hybrid model--TMnn (Transmission Matrix and Neural Network), based on complex value operations and a neural network that incorporates the physical processes of multimode fiber light field modulation. The model is applied to train and verify different natural scene image datasets, and the results show that the model training speed is significantly improved while maintaining the quality of image reconstruction. At the same time, the generalization ability of the neural network is also enhanced in the image restoration task. Methods Combining the physical mechanisms of optical fiber and neural networks, we propose a multimode optical fiber speckle reconstruction algorithm, TMnn, based on complex value operations, which is trained on a natural scene image dataset. According to the response relationship between the input and output optical fields of multimode fiber, the inverse transmission matrix of the fiber is fitted using an iterative algorithm. The reconstruction optimization is then performed through a convolutional neural network to complete the speckle image reconstruction. The model is mainly divided into two modules. The first is the reconstruction module, which constructs the complex value deep neural network to fit the transmission matrix and initially reconstructs the images. The network consists of an input layer, complex convolution layers, complex batch normalization layers, and complex dense connection layers. The second part is the optimization module, which optimizes the initially reconstructed image by constructing a 3x3 convolutional neural network. The initial reconstructed image is taken as input, and the image features are extracted deeply. Image details are then reconstructed through the convolution layer, pooling layer, and fully connected layer in sequence. Results and Discussions By comparing with traditional neural networks (SCNN, DCNN), CANN (Complex Artificial Neural Network), and USINET, we confirm the advantages of the model in terms of reconstruction effect and training speed. In terms of model training, we make a comparison with CANN on the ImageNet dataset. Compared with CANN, the SSIM index shows a significant improvement, and the number of iterations is reduced by 200. However, the addition of two complex convolution layers increases the number of parameters, which has a certain effect on the training time cost. We also compare the model with USINET, and the training information is shown in Table 2. The results show that the average SSIM index of this algorithm improved by about 0.5%, and the training time is reduced by 6.46 hours. The TMnn model outperforms USINET in the first 4 hours of training and tends to converge after about 6 hours of training, with the SSIM value stabilizing at around 0.8. This indicates that the model constructed in this paper does not compromise training speed or model performance despite the complex operations. Conclusions We integrate the physical mechanism of optical fiber transmission with deep learning technology to construct a deep neural network based on complex-valued operations, which achieves high-quality reconstruction with an SSIM index above 0.7. Through the reconstruction of various datasets, the validity and generalization of the model are demonstrated. By comparing it with traditional neural networks, fully connected complex networks, and USINET, the advantages of the model are confirmed in terms of reconstruction quality and training speed. However, the network still has limitations in reconstructing more complex, detailed images. The network structure and model parameters for feature extraction need optimization to better capture detailed features and further enhance the quality of natural scene image reconstruction.
Due to the problems of low detection accuracy and high missed detection rate caused by small and dense objects in remote sensing images, we propose an improved lightweight YOLOv10 model aimed at improving detection performance. Specifically, the standard convolution is replaced with RFAConv in the backbone network to better capture fine-grained object details. Additionally, we incorporate a coordinate attention module to refine the spatial localization and object recognition capabilities of the model. Furthermore, the CARAFE up-sampling operator is adopted in place of standard nearest-neighbor interpolation, expanding the model's receptive field. Experimental results demonstrate that the improved model significantly outperforms the original YOLOv10 on the UCAS-AOD dataset, achieving a mAP50 of 97.2% and a mAP95 of 63.2%. This approach offers an efficient and lightweight solution for object detection in remote sensing images.
Removing stripe noise is a fundamental task in remote sensing image processing, which is of great significance for improving image quality and subsequent applications. In this paper, an adaptive strip noise removal model is proposed with the spatial characteristics. Firstly, an adaptive weight function is constructed using local absolute differences to adaptively control the constraint intensity of the penalty term at different pixel points in the adaptive strip noise removal model. Secondly, L1 norm is used to constrain the local smoothness along the direction of the strip, maintaining the obvious smoothness characteristics of the strip noise in its extension direction, while L2 norm is used to restrict the image grayscale. Finally, the extended split Bregman iteration method and alternating minimization method is used to optimize the proposed image destriping model. Extensive experiments on both the synthetic and real remote sensing images validate that the proposed model can effectively remove the stripe noise and preserve more fine scale details.
Skyline detection plays a crucial role in fields such as reconnaissance and photoelectric guidance. This paper presents a deep learning-based skyline detection method utilizing heatmaps as the output of the skyline neural network. To address the downsampling issue prevalent in heatmap-based methods, we propose the use of offset compensation heatmaps to restore the downsampled points. Additionally, we enhance the YOLO network to improve the resolution of the output heatmaps. Experimental results demonstrate that our method is highly adaptable to complex scenes and performs at inference speeds suitable for edge deployment.
With the rapid development of studio informatization, the traditional era of studio construction is gradually entering digitalization. In recent years, the rapid development of cloud computing technology in the field of education has greatly changed the way teachers teach and students learn. The main goal of the digital studio cloud platform is to improve the teaching efficiency and quality of teachers, and provide students with a more convenient platform. At present, education cloud platforms are still in the early stages of development, and many designs focus on functionality while ignoring the core needs of users, leading to difficulties in user operation and hindering the comprehensive promotion of education cloud platforms. This article proposed a digital studio construction strategy based on cloud computing, using systematic and intelligent methods to improve the traditional studio teaching mode, strengthen objective evaluation and data guidance functions, and explore the standardized and systematic teaching mode of studios. This article constructed the ZStack private cloud platform to achieve flexible resource allocation and efficient utilization, and evaluated performance under different workloads through experiments, providing important reference for the optimization of future education cloud platforms.
The violent swing of the pantograph deteriorates the contact status of the pantograph and catenary system (PCS) and seriously threatens the operation security of high-speed trains. Currently, video surveillance is commonly used for pantograph pose monitoring. Due to complex weather conditions of illumination change, occlusion, and background interference on the roof, achieving effective real-time high-precision attitude measurement remains challenging. Therefore, this study proposes a deep-learning-based 3-D measurement method. First, an accurate end-to-end pantograph keypoint location network (PKLNet) is proposed to position image coordinates of the keypoints. In PKLNet, local and global contexts are integrated by an improved Transformer encoder to solve the common interference and occlusion. Differentiable numerical regression (DNR) is performed to realize an accurate coordinate regression of the keypoints. Second, the keypoint positioning is used to overcome the interference of the complex background to extract the edge, and an edge shape algorithm is employed to optimize the pose. If the edge extraction fails, the perspective- ${n}$ -point (PnP) of the keypoints is carried out for pose calculation. Evaluation of practical image sequences indicates that the average precision at PCKp@0.03 is 96%. An attitude accuracy experiment further confirms that the optimized attitude root mean squared error (RMSE) is less than 0.35° and the average speed is 36.2 frames/s, fulfilling the strict requirement of pantograph attitude measurement.
Good contact conditions between the pantograph and catenary ensure the safety of electrified railways. However, the relevant state-of-the-art status inspection methods cannot achieve high speed and robust effects under complex conditions, such as multisuspended line interference, unknown complex backgrounds, and light and dark changes. Therefore, a two-module contact status diagnosis methodology, consisted of long-term tracking and 3-D reconstruction, is proposed in this study. This long-term tracking method is a combination of tracking, detection, and discrimination, focusing on locating the contact area of high-speed movement in complex environments. First, a brand-new tracking paradigm, namely, GMSTrack, based on feature matching is developed to adapt to complex background changes. Second, we modify the loss function via GIoU-plus loss for box regression and adaptive wing loss for landmark regression on CenterNet and realize a dramatic improvement in detection accuracy at 125 FPS. Third, sequential information is collected for the transformer-based discriminator to evaluate the correctness of the current tracking result. Experimental results show that the optimal strategy that combines tracking, detection, and discrimination can locate the contact area in complex environments with an accuracy higher than 93% and with a speed higher than 84 FPS. Finally, dynamic stagger value and height of conductor that can reflect contact status are analyzed through the 3-D reconstruction of the binocular contact points. The overall status inspection methodology can be applied to the on-site high-speed trains at speeds exceeding 80 FPS.
The metrological integrated support capability is an important part of the equipment support capability of the launch site. To realize the quantitative description and scientific evaluation of the metrological integrated support capability, the FUZZY-AHP method is introduced. Based on the study of the factors affecting the capability, the evaluation index system is constructed from four aspects of personnel capability, equipment capability, technical support capability, and comprehensive management capability. A multilevel evaluation model of the metrological support capability is established by using the analytic hierarchy process. Then, with the fuzzy mathematics method, the rules of the fuzzy evaluation for the quantitative and qualitative indicators are formulated. Finally, combined with a launch mission, the FUZZY-AHP method is used to evaluate the metrological integrated support capability. The method is scientific and feasible, which provides an effective reference for the subsequent evaluation of the metrological integrated support capability.
As an important part of the online monitoring in electrified railway, pantograph operational pose inspection plays a vital role in pantograph and catenary system (PCS) security. Due to the impact of the complex conditions on the train roof, it is difficult to replace the existing video surveillance with fully intelligent operational pose inspection. Therefore, this article proposes a novel accurate monocular 3-D pantograph horn edge-based pantograph pose inspection method in the power of superior ability of deep learning. It mainly includes two steps: edge detection and pose optimization. First, we highlight a brand-new high-precision subpixel edge prediction network. The downsampling heatmap and offset prediction map in our edge detector are joined to predict full-size edge points and thus improve the accuracy to the subpixel level. In addition, our newly proposed numerical gradient smoothing loss function overcomes the “jaggy” phenomenon of the edge curves. Second, we propose a nonpoint-to-point corresponding 3-D pose measurement method of the monocular camera. In the iterative optimized process, we innovatively apply the chamfer distance as an optimization function to dynamically adjust the correspondence between the reprojection points and the predicted edge points to optimal convergence. Ablation and comparative experiments demonstrate that our method achieves excellent performance (F-score 93.0% in FEP 2.0 (fixed edge probability) with 82 frames per second (FPS), overcoming the complex conditions of the practical online monitoring sequences. The actual pose measurement results further confirm our efficiency, thereby achieving a robust real-time and high-precision monocular 3-D pantograph operating pose inspection.
The automatic inspection of real-time pantograph-and-catenary (PAC) operational status is important for ensuring the safety of the railway power supply system and realizing its intelligent operation and maintenance. However, state-of-the-art contact-loss monitoring methods cannot achieve high precision, high speed, and robust effects under complex conditions, such as frequent line changes, multi-suspended line interference, unknown complex backgrounds, and brightness variations. Therefore, a new two-stage online detection method for PAC operational status is proposed. First, we propose a three-module-assisted high-precision key point positioning detector for all possible intersection points of PAC. It can realize end-to-end PAC contact corner points detection for double branch contact mode in the process of line changes, instead of the original way of tracking and then detecting. Each auxiliary module adopts a minimalist ${3\times 3} $ convolutional layer without pre-training to enhance the feature expression ability of the corresponding layer in the backbone, thus improving the sub-pixel detection accuracy. Second, we propose a topological accuracy compensation algorithm to compensate for the mismatch error between the corner points in the image and the projection of the actual spatial contact point owing to the self-occlusion of PAC, thus improving the positioning precision of the real 3-D contact point. Physical simulation, accuracy verification, and field experiments prove that our PAC status inspection algorithm can achieve accurate conductor height and stagger value measurements under complicated environmental interference, thus ensuring the safety of train operation.
Pantograph slider is a critical component of the electrical equipment to obtain power sources for the electric locomotive, whereas the interaction between the slider and the overhead contact line can cause persistent surface wear of the collector strip. The condition monitoring for the level of abrasion of the pantograph slider is significant to avoid major damage. In this article, an innovative approach based on multiview analysis is proposed for monitoring the surface wear of the slider in a running train, accurately measuring the residual thickness of the slider, and locating the wear defects appearing. Especially, the 3-D reconstruction technology is used for detecting the wear quantity between the upper and lower edges of the slider. First, the target edges of the slider are determined using the proposed subpixel edge detection algorithm. Second, mutual matching of the edge points between the multiview images is finished using the epipolar geometric constraint. Moreover, the 3-D reconstructions of the matching points are computed with the principle of triangulation. Finally, the wear volume of the slider can be determined. Experimental results show that the proposed method can successfully measure the abrasion of the pantograph slider under complex conditions, which demonstrate the applicability, effectiveness, and good performance of the proposed approach.
The pantograph is an essential component in modern electrified trains. Due to the characters of contactless and high precision, the 3-D stereo vision, which can effectively monitor the abrasion, is of great significance for ensuring train safety. However, illumination conditions, such as overbrightness and insufficient light, have a great impact on semiglobal matching (SGM) stereo matching, inevitably affecting the integrity and accuracy of the abrasion measurement of the pantograph slipper. In this regard, a robust SGM stereo matching is proposed in this article for the 3-D abrasion measurement. First, the combination of the neighborhood information-aware census cost and multiscale fusion for cost aggregation is put forward. While insensitive to illumination condition changes, the improved census cost can quantify the texture information of the neighborhood. Moreover, multiscale cost aggregation further fuses the neighborhood information of different scales. As a result, the performance of SGM in weakly textured areas is greatly improved, especially in complex illumination conditions. Eventually, with the help of the subpixel disparity refinement and local points fitting, an accurate 3-D abrasion calculation is achieved. Practical experiments show the robustness of stereo matching. The precision verification and computation time analysis confirm the measurement accuracy and applicability.
Pantograph is a power collector to get current from the overhead catenary for electric train engine. When the train moves forward, the high frequency friction between pantograph and catenary gives rise to the wear of the carbon strips. In this paper, we propose an efficient stereo-based method to non-contact monitor the pantograph wearing. Firstly, a modified SGM-based stereo matching algorithm is raised to overcome the complicated illumination in the actual environment. Secondly, the point cloud segmentation using Radius Filter and Density-based Spatial Clustering of Applications with Noise (RF-DBSCAN) is carried out to eliminate noisy points caused by mismatching and extract the carbon strips from background. Finally, for the purpose of wearing detection, the processed point cloud is aligned with stand CAD model by the proposed coarse-to-fine iterative closest point (CF-ICP) method. The experimental data which is obtained from a real train maintenance depot reveals that root mean square (RMS) of the wearing inspection is less than 4 mm, which outperform than other classic methods
Many multi-vision measurement systems like outdoor industrial engineering utilize optical filter to filter unexpected light which makes it impossible to use traditional planar pattern for calibration. In this paper, we propose a stereo vision global calibration method for non-overlapping views that comprises two cameras, an optical line laser whose wave band within the optical filter's and 1D square serrated target. Vertex-points coordinates in the camera coordinate frame (CCF) are obtained based on the cross ratio invariance theory and vanishing points. Initial rotation vector and translation vector are confirmed by flexibly moving the target to at least three different positions. After a nonlinear refinement, external parameters are precisely determined. Experimental results show that the measurement accuracy reaches to 0.06 mm within the cameras' FOV of 450mm*370mm.