Real-time semantic segmentation is a prerequisite for autonomous vehicles. Despite significant advancements, attaining an optimal balance between segmentation accuracy and efficiency remains a challenge for existing algorithms. To this end, we propose an orthogonal dual-path network, terms ODPNet, for real-time semantic segmentation of urban road scenes. First, the network is decoupled into two orthogonal paths, namely the Semantic Path and the Refinement Path, to encode high-level semantics and low-level details, respectively. Subsequently, we introduce an Atrous Decoupled Pyramid Pooling Module (ADPPM) to overcome the receptive field rigidity of traditional square pooling. By synergizing asymmetric pooling with atrous convolutions, the ADPPM effectively aggregates multi-scale contextual information and simultaneously enhances the modeling of long-range dependencies along orthogonal spatial dimensions. Furthermore, we design a Multi-View Aggregation Module (MVAM) to bridge the semantic-spatial gap through a trans-dimensional attention mechanism, thereby enhancing feature discriminability. Additionally, we leverage boundary supervision via the Canny operator to explicitly guide the preservation of fine-grained structural details to better perform orthogonalization of the dual-path. Extensive experiments on the Cityscapes and CamVid datasets demonstrate the effectiveness of ODPNet, showcasing a superior trade-off between inference speed and segmentation accuracy. Specifically, on the Cityscapes dataset, ODPNet-Lite achieves 78.9% mIoU at 162.1 FPS, while ODPNet-Base and ODPNet-Deep achieve 79.9% mIoU at 64.2 FPS and 80.4% mIoU at 48.2 FPS, respectively.

Semantic segmentation is a key technology for autonomous vehicles to understand the surrounding scenes. Multi-branch network architectures have demonstrated their efficiency and effectiveness in real-time semantic segmentation tasks. Although PIDNet achieves a balance between performance and efficiency, it is inadequate in fine-grained segmentation and multi-scale feature integration. In this paper, we propose a real-time semantic segmentation network, called ARFNet, which employs a three-branch structure. To enhance the perception and fusion of multi-scale features, we introduce a Hierarchical Dense Atrous Pyramid Module (HDAPM). Additionally, we propose a novel Trans-Dimensional Interaction Module (TDIM) to systematically enhance feature representation through cross-dimensional attention mechanisms. The effectiveness of our method is demonstrated by extensive experiments on the Cityscapes and CamVid datasets, showing that it achieves a promising trade-off between inference speed and segmentation accuracy. Specifically, ARFNet achieves 79.8% mIoU at 94.8 FPS on the Cityscapes dataset and 80.9% mIoU at 154.1 FPS on the CamVid dataset.

Real-time semantic segmentation has broad prospects in computer vision. Existing state-of-the-art approaches generally employ bilateral networks to encode spatial and contextual information. Nevertheless, the real-time methods exhibit unsatisfactory performance. To address this problem, in this work we propose a novel shared trunk and dual-branch network named STDBNet for real-time semantic segmentation. In particular, we devise a Shared Trunk Module, which efficiently diminishes superfluous channels and parameters. Subsequently, we present a Split Dual-Branch Module, consisting of Detail Branch and Semantic Branch, with the former capturing detailed information and the latter capturing contextual information correspondingly. Furthermore, at the Semantic Branch, an Efficient Pyramid Pooling Module is designed towards the end to further expand the receptive fields and harvest multi-scale contextual information. Finally, to efficiently merge the features from the two branches and then get the final results, we introduce an Attention-optimized Feature Fusion Module, which utilizes redesigned spatial attention mechanism for feature augmentation. Extensive experiments conducted on the Cityscapes dataset indicate that the devised STDBNet obtains 77.6% mIoU with 91.5 FPS on one GeForce 2080Ti GPU, which outperforms BiSeNet with a favorable trade-off between accuracy and efficiency.

Semantic segmentation is an important research topic in the environment perception of intelligent vehicles. Many semantic segmentation networks based on bilateral architecture have been proven effective. However, semantic segmentation networks based on this architecture has the risk of pixel classification errors and small objects being overwhelmed. In this paper, we solve the problem by proposing a novel three-branch architecture network called LCFNets. Compared to existing bilateral architecture, LCFNets introduce compensation branch for the first time to preserve the features of original images. Through two efficient modules, Lightweight Detail Guidance Fusion Module (L-DGF) and Lightweight Semantic Guidance Fusion Module (L-SGF), detail and semantic branches are allowed to selectively extract features from this branch. To balance the three-branch features and guide them to fuse effectively, a novel aggregation layer is designed. Depth-wise Convolution Pyramid Pooling module (DCPP) and Total Guidance Fusion Module (TGF) enable the aggregation layer to extract the global receptive field and realize multi-branch aggregation with fewer calculation complexity. Extensive experiments on Cityscapes and CamVid datasets have shown that our family of LCFNets provide a better trade-off between speed and accuracy. With the full resolution input and no ImageNet pre-training, LCFNet-slim achieves 76.86% mIoU at 114.36 FPS and LCFNet achieves 77.96% mIoU at 92.37 FPS on Cityscapes. On the other hand, LCFNet-super achieves 79.10% mIoU at 47.46 FPS. Source code can be found from the: https://github.com/yiwen-bai/LCFNets .

In order to improve the performance of lane detection algorithms under complex scenes like obstacles, we proposed a multi-lane detection method based on dual attention mechanism. Firstly, we designed a lane segmentation network based on a spatial and channel attention mechanism. With this, we obtained a binary image which shows lane pixels and the background region. Then, we introduced HNet which can output a perspective transformation matrix and transform the image to a bird's eye view. Next, we did curve fitting and transformed the result back to the original image. Finally, we defined the region between the two-lane lines near the middle of the image as the ego lane. Our algorithm achieves a 96.63% accuracy with real-time performance of 134 FPS on the Tusimple dataset. In addition, it obtains 77.32% of precision on the CULane dataset. The experiments show that our proposed lane detection algorithm can detect multi-lane lines under different scenarios including obstacles. Our proposed algorithm shows more excellent performance compared with the other traditional lane line detection algorithms.

Semantic segmentation is used by intelligent transportation systems to understand and sense the traffic environment. However, achieving semantic segmentation in real-time is a challenge due to the necessity of both high accuracy and fast processing. This is especially valuable for applications such as autonomous driving and industrial robotics. In this paper, we propose a real-time semantic segmentation network, called LCFNet, which makes use of three-branch structure. The LCFNet consists of Lightweight Detail Guidance Fusion (L-DGF) and Lightweight Semantic Guidance Fusion (L-SGF) modules. Both modules aggregate information from various network layers. In the termination of network, a Total Guidance Fusion (TGF) module is proposed for processing information from all three branches. Depth-wise Convolution Pyramid Pooling (DCPP) module is also included to optimize accuracy and simplify computation. The effectiveness of LCFNet is demonstrated on two typical semantic segmentation datasets, Cityscapes and CamVid. On a single NVIDIA GeForce GTX 2080 Ti GPU, LCFNet reaches 77.02% mIoU at 95.97 FPS and 81.17% mIoU at 204.82 FPS, respectively.

To improve the performance of image semantic segmentation on accuracy and efficiency for practical applications, in this study, we propose a real-time semantic segmentation algorithm based on improved BiSeNet. First, the redundancy of certain channels and parameters of BiSeNet is eliminated by sharing the heads of dual branches, and the affluent shallow features are effectively extracted at the same time. Subsequently, the shared layers are divided into dual branches, namely, the detail branch and the semantic branch, which are used to extract detailed spatial information and contextual semantic information, respectively. Furthermore, both the channel attention mechanism and spatial attention mechanism are introduced into the tail of the semantic branch to enhance the feature representation; thus the BiSeNet is optimized by using dual attention mechanisms to extract contextual semantic features more effectively. Finally, the features of the detail branch and semantic branch are fused and up-sampled to the resolution of the input image to obtain semantic segmentation. Our proposed algorithm achieves 77. 2% mIoU on accuracy with real-time performance of 95. 3 FPS on Cityscapes dataset and 73. 8% mIoU on accuracy with realtime performance of 179. 1 FPS on CamVid dataset. The experiments demonstrate that our proposed semantic segmentation algorithm achieves a good trade-off between accuracy and efficiency. Furthermore, the performance of semantic segmentation is significantly improved compared with BiSeNet and other existing algorithms.

Recently, the task of lane detection has been greatly improved with the rapid development of deep learning and autonomous driving. However, there exist limitations like the challenging complex scenarios and real-time efficiency. In this paper, we present a novel Attention Based Dual Path Network (ADPNet) to handle the task of lane detection. The ADPNet treat the process of lane detection as a task of binary semantic segmentation, where the Detail Path is designed to capture detailed low-level information and the Semantic Path with dual attention module is designed to capture contextual high-level information. We use the Feature Aggregation Module to fuse the information of the two paths, followed by the process of lane fitting to get a parametric description of lanes. The proposed ADPNet achieves good trade-off between the accuracy and real-time efficiency on TuSimple and CULane, which are two popular lane detection benchmark datasets. The results demonstrate that our architecture outperforms the current state-of-the-art methods.

In the dairy farming industry, we can obtain the temperature, color, and location information of dairy cows by patrol inspection robot so as to monitor the health status and abnormal behaviors of dairy cows. We build and calibrate a heterogeneous binocular stereo vision (HBSV) system comprising a high-definition color camera and infrared thermal camera and mount it on a patrol inspection robot. First, based on the traditional chessboard, an easy-to-make calibration board for the HBSV system is designed. Second, an accurate locating and sorting algorithm for the calibration points of the calibration board is designed. Then, the cameras are calibrated and the HBSV system is stereo-calibrated. Finally, target locating is achieved based on the above calibration results and Yolo target detection technology. In this paper, several experiments are carried out from many aspects. The target locating average error of HBSV system is 3.11%, which satisfies the needs of the dairy farming environment. The video's FPS captured by using HBSV is 7.3, which is 78% higher than that by using binocular stereo vision system and infrared thermal camera. The results show that the HBSV system has application value to a certain degree.

Road detection is a crucial research topic in computer vision, especially in the framework of autonomous driving and driver assistance. Moreover, it is an invaluable step for other tasks such as collision warning, vehicle detection, and pedestrian detection. Nevertheless, road detection remains challenging due to the presence of continuously changing backgrounds, varying illumination (shadows and highlights), variability of road appearance (size, shape, and color), and differently shaped objects (lane markings, vehicles, and pedestrians). In this paper, we propose an algorithm fusing appearance and prior cues for road detection. Firstly, input images are preprocessed by simple linear iterative clustering (SLIC), morphological processing, and illuminant invariant transformation to get superpixels and remove lane markings, shadows, and highlights. Then, we design a novel seed superpixels selection method and model appearance cues using the Gaussian mixture model with the selected seed superpixels. Next, we propose to construct a road geometric prior model offline, which can provide statistical descriptions and relevant information to infer the location of the road surface. Finally, a Bayesian framework is used to fuse appearance and prior cues. Experiments are carried out on the Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) road benchmark where the proposed algorithm shows compelling performance and achieves state-of-the-art results among the model-based methods.

针对基于深度学习的DeepLabV3+语义分割算法在编码特征提取阶段大量细节信息被丢失,导致其在物体边缘部分分割效果不佳的问题,本文提出了基于DeepLabV3+与超像素优化的语义分割算法。首先,使用DeepLabV3+模型提取图像语义特征并得到粗糙的语义分割结果;然后,使用SLIC超像素分割算法将输入图像分割成超像素图像;最后,融合高层抽象的语义特征和超像素的细节信息,得到边缘优化的语义分割结果。在PASCAL VOC 2O12数据集上的实验表明,相比较DeepLabV3+语义分割算法,本文算法在物体边缘等细节部分有着更好的语义分割性能,其mIoU值达到83.8%,性能得到显著提高并达到了目前领先的水平。