A new 3DpredicNet is proposed to segment potholes and predict their 3-D volume using monocular computer vision (red, green, and blue (RGB) images) simultaneously. The network produces pixelwise segmentation masks and depth maps from a single RGB image. These outputs are combined with camera parameters such as height, viewing angle, focal length, and sensor dimensions using geometric formulas to calculate pothole volume. This approach allows quantitative damage assessment without specialized depth sensors. To develop a state-of-the-art deep learning network, various advanced modules were extensively modified and integrated, including convolution blocks, residual intensive convolution modules, depthwise separable convolution blocks, dual attention modules, downsampling and upsampling operations, and residual criss-cross attention modules. To train and test the 3DpredicNet, a dataset containing 440 images named Pothole440 was developed by collecting RGB images ($6000 imes 4000 imes 3$), and its corresponding 3-D scan data collected were used as the ground truth of pothole 3-D depth. The 3DpredicNet outperformed other recent networks, such as UNet, Deeplabv3+, AttentionUNet, MobileNetV3, YOLOv11, U-Mamba, ScSegamba, and SegResNet, for the pothole segmentation and its depth prediction on the Pothole440 dataset. 3DpredicNet achieved an average error of 0.036 across absolute relative error, squared relative error, root-mean-squared error, and root-mean-squared logarithmic error metrics, outperforming all the other networks by 37.93%. 3DPredicNet demonstrated strong segmentation performance, achieving the highest average across all evaluation metrics-including precision, recall, accuracy, F1-score, and mean intersection over union-although its accuracy was slightly lower than that of a competing method. Furthermore, its processing speed of 44.50 frames per second (FPS) comfortably meets the requirements for real-time video processing.

In structural health monitoring using computer vision, deep learning-based damage identification and three-dimensional (3D) reconstruction of the structure are current hot topics. Traditional photogrammetry techniques are cost-inefficient and time-consuming for 3D reconstruction, and there is no such solid 3D pixelwise damage mapping technique. To overcome these limitations, a new deep neural network (DNN)-based 3D reconstruction method, including damage mapping, is proposed in this article. As the DNN-based 3D reconstruction method, Nerfacto—an advanced version of Neural Radiance Fields models—was selected for achieving high-fidelity 3D reconstruction. This Nerfacto model was modified to create a high-definition 3D reconstruction model of the structure of interest (i.e., a 3-span bridge system). To map damages within the reconstructed 3D model using the modified Nerfacto, the state-of-the-art semantic transformer representation network (STRNet) with test time augmentation (TTA) was also developed for precise pixel-wise crack segmentation. Through extensive case studies, including parametric studies, we found that the modified Nerfacto can learn various appearance features of the structure and generate a very high-definition 3D model. Moreover, the segmented damage (i.e., cracks) from the STRNet with TTA could be mapped onto the reconstructed 3D model. This study demonstrates the potential of combining deep learning with 3D reconstruction for proactive and preventative maintenance strategies, ensuring the safety and longevity of vital structural assets.

This research presents an innovative approach to generating three-dimensional (3D) digital twin models for damage assessment in large-scale civil structures. By refining the Nerfacto model, which builds upon Neural Radiance Fields (NeRF), the study achieves highly accurate 3D reconstruction with detailed, pixel-level damage visualization. To enhance precision, STRNet with Test Time Augmentation (TTA) was utilized for pixel-wise crack segmentation, ensuring seamless incorporation of damage data into the digital twin model. Extensive case studies validated the effectiveness of this methodology for large-scale infrastructure, highlighting its potential for proactive structural health monitoring. Future investigations will focus on real-world outdoor applications, integrating UAV-based imaging while addressing challenges such as scale variations and lighting inconsistencies. This framework provides a reliable foundation for automating infrastructure inspection and maintenance.

Since 2017, extensive damage detection using advanced deep learning models and computer vision techniques has been actively explored. However, efficiently mapping detected damage in 3D digital twin model remains a challenge, as few studies have successfully integrated deep learning and computer vision for precise damage representation. To address this gap, this study investigates an enhanced NeRF-based model, ABM-Nerfacto [1], designed for high-definition and efficient damage mapping in 3D digital twin. This advancement facilitates more effective structural health monitoring, infrastructure maintenance, and a comprehensive pixel-level overview of damage distribution. The Nerfacto model was extensively modified and integrated with an advanced attention mechanism to improve its ability to learn structural features and damage patterns. When applied to a bridge system, the developed model demonstrated exceptional accuracy in pixel-wise damage mapping, successfully generating a highfidelity 3D digital twin.

While many structural damage detection methods have been developed in recent decades, few data-driven methods in unsupervised learning mode have been developed to solve the practical difficulties in data acquisition for civil infrastructures in different scenarios. To address such a challenge, this article proposes a number of improved unsupervised novelty detection methods and conducts extensive comparative studies on a laboratory scale steel bridge to examine their performances of damage detection. The key concept behind unsupervised novelty detection in this article is that only normal data from undamaged/baseline structural scenarios are required to train statistical models with these methods. Then, these trained models are used to identify abnormal testing data from damaged scenarios. To detect structural damage in the form of loosening bolts in the steel bridge, four machine-learning methods (i.e., K-nearest neighbors method, Gaussian mixture models, one-class support vector machines, density peaks-based fast clustering method) and one deep learning method using a deep auto-encoder are selected. Meanwhile, some modifications and improvements are made to enable these methods to detect structural damage in unsupervised novelty detection mode. In their comparative studies, the advantages and disadvantages of these methods are analyzed based on their results of structural damage detection.

Accurately predicting groundwater level (GWL) changes induced by seismic events remains a key challenge in hydrogeological research, with significant implications for earthquake preparedness and water resource management. This study proposes a novel machine learning framework that combines automated feature engineering with multi-model classification to forecast persistent GWL changes following earthquakes. Using a well-curated dataset of 2,563 GWL responses from 495 wells in New Zealand, we employ the AutoFeat library to construct a set of engineered features that capture complex interactions between seismic parameters and aquifer characteristics. A range of classifiers, including Gradient Boosting, Random Forest, Logistic Regression, and others, were benchmarked using the PyCaret library, with model performance evaluated through accuracy, precision, recall, F1-score, and area under the ROC curve (AUC). Gradient Boosting emerged as the top-performing model, especially after hyperparameter optimization, achieving an AUC of 0.89 and F1-score of 0.62. Feature importance analysis further demonstrated the superior predictive value of engineered variables over traditional seismic metrics. The results validate the effectiveness of combining feature construction and ensemble learning for improved earthquake-induced GWL prediction and offer a scalable, interpretable approach for future seismic-hydrological modeling applications.

Recent advancements in structural health monitoring have highlighted the necessity for accurate threedimensional (3D) damage mapping on digital twins, moving beyond traditional methods such as photogrammetry, which frequently struggle to capture intricate planar surfaces. To address this limitation, this paper proposes a new advanced 3D reconstruction method and its integration with 3D damage mapping techniques. As the 3D reconstruction method, an Attention-based Modified Nerfacto (ABM-Nerfacto) model is developed, and is integrated with an advanced damage segmentation method. Using a three-span continuous bridge with concrete piers as an example structure, and concrete cracks as the example damage, the state-of-the-art STRNet is utilized for crack segmentation. Through extensive parametric studies and comparative evaluations, the proposed ABMNerfacto model was demonstrated to produce high-quality 3D reconstructions and corresponding damage mappings for this bridge system. This integrated approach provides a promising solution for comprehensive 3D digital twin-based structural health monitoring.

This paper proposes a framework for obstacle-avoiding autonomous unmanned aerial vehicle (UAV) systems with a new obstacle avoidance method (OAM) and localization method for autonomous UAVs for structural health monitoring (SHM) in GPS-denied areas. There are high possibilities of obstacles in the planned trajectory of autonomous UAVs used for monitoring purposes. A traditional UAV localization method with an ultrasonic beacon is limited to the scope of the monitoring and vulnerable to both depleted battery and environmental electromagnetic fields. To overcome these critical problems, a deep learning-based OAM with the integration of You Only Look Once version 3 (YOLOv3) and a fiducial marker-based UAV localization method are proposed. These new obstacle avoidance and localization methods are integrated with a real-time damage segmentation method as an autonomous UAV system for SHM. In indoor testing and outdoor tests in a large parking structure, the proposed methods showed superior performances in obstacle avoidance and UAV localization compared to traditional approaches.

This article provides a comprehensive review of deep learning-based structural health monitoring (DL-based SHM). It encompasses a broad spectrum of DL theories and applications including nondestructive approaches; computer vision-based methods, digital twins, unmanned aerial vehicles (UAVs), and their integration with DL; vibration-based strategies including sensor fault and data recovery methods; and physics-informed DL approaches. Connections between traditional machine learning and DL-based methods as well as relations of local to global approaches including their extensive integrations are established. The state-of-the-art methods, including their advantages and limitations are presented. The review draws on current literature on the topic, also providing a synergistic analysis leading to the understanding of the evolution of DL as a basis for presenting the future research and development needs. Our overall finding is that despite the rapid progression of digital technology along with the progression of DL, the DL-based SHM appears to be in its infant stages with enormous potential for future developments to bring the SHM technology to a common practical use with wide scope applications, performance reliability, cost, and degree of automation. It is anticipated that this review paper will serve as a basic resource for readers seeking comprehensive and holistic understanding of the subject matter.

Download This Paper Open PDF in Browser Add Paper to My Library Share: Permalink Using these links will ensure access to this page indefinitely Copy URL Copy DOI

The study introduces a novel approach for time-varying reliability analysis of structures called "hybrid UKF-PDEM" by integrating the unscented Kalman filter (UKF) and the probability density evolution method (PDEM). The UKF estimates the displacement, velocity, stiffness, and damping parameters of a structure at each time step subjected to dynamic loading for structural damage quantification. The estimated parameters at each time step are then input into the PDEM to calculate the time-varying probability density function (PDF) of the estimated states. The estimated PDF is used to update the uncertainty matrix of the estimated states in each iteration and to determine the time-varying reliability curves of the structure. To demonstrate the effectiveness of the proposed method, we applied it to a numerical model of a three-degree-of-freedom system and a full-scale seven-story building with different damage scenarios. The method is used to estimate the level of damage and calculate the corresponding reliability curve of the system over time for each damage scenario, utilizing the estimated structural responses and stiffness values. The extracted reliability values for each damage scenario follow the level of damage over time. This study shows that the newly developed method is computationally efficient for building a digital twin and enables real-time damage identification and reliability analysis in various structural systems. The method's applicability to different types of structures highlights its versatility and potential for widespread use in assessing the integrity of buildings and infrastructure.

Potholes pose significant safety risks to drivers and cause damage to vehicles. This paper modified a novel approach called the monocular depth estimation and segmentation (modified 3DPredicNet) network [1] to accurately estimate depth maps and segment potholes. To facilitate model training and evaluation, a comprehensive dataset consists of RGB images captured using a DSLR camera and corresponding 3D scan data for generating depth maps. The depth maps derived from the 3D scans are utilized for pothole depth estimation, while masks are used for pothole segmentation. The evaluation results reveal the model's ability to accurately predict and segment potholes in RGB images, achieving a minimum absolute relative error (ARel) of 0.062, square relative error (SRel) of 0.011, and root mean square error (RMSE) of 0.118 when tested on the newly developed dataset. Moreover, when tested on the newly developed dataset, the model demonstrates good pothole segmentation performance, attaining a high mean intersection over union (mIoU) of 81.05. Furthermore, when utilizing the publicly available dataset, the modified 3DPredicNet achieved accurate depth estimation with ARel of 0.093 and SRel of 0.033.

The use of active noise control/cancelation (ANC) has increased because of the availability of efficient circuits and computational power. However, most ANC systems are based on traditional linear filters with limited efficiency due to the highly nonlinear and nonstationary nature of various noises. This paper proposes an advanced deep learning–based feedback ANC named DNoiseNet that overcomes the limitations of traditional ANCs and addresses primary and secondary path effects, including acoustic delay. Mathematical operators (i.e., atrous convolution, pointwise convolution, nonlinear activation filters, and recurrent neural networks) learn multilevel temporal features under different noises in various environments, such as construction sites, vehicle interiors, and airplane cockpits. Due to the nature of feedback control using a single error sensor, an estimation of the reference noise signal must be regenerated. In this paper, a multilayer perceptron (MLP) neural network–based secondary path estimator is also proposed to improve the performance of DNoiseNet. In extensive parametric and comparative studies, the DNoiseNet with the MLP secondary path estimator exhibited the best performance in root mean square error and noise attenuation metrics.

Deep learning can produce great results when adequate data are available for training. Surface damage such as cracks and corrosion are visible and available on the surface; however, subsurface damages are not visible; therefore, it is difficult, expensive, and time-consuming to prepare the dataset for subsurface damage. Attention-based generative adversarial network (AGAN) (Ali and Cha, Autom Constr 141:104412, 2022) generates artificial data using the real data collected with thermal cameras from the developed concrete samples. This study further investigates the performance of AGAN for generating synthetic thermal image data. Both damage and intact data were used to train AGAN. In addition, the learned perceptual image patch similarity (LPIPS) and Fréchet inception distance (FID) were used to compare AGAN performance with the two types of GAN. AGAN's LPIPS and FID values show better performance compared to modified DCGAN and D2GAN.

Workers in construction sites are at risk of irregular heartbeats, speech problems, insomnia, and permanent hearing loss due to being exposed to high noise levels, usually above 85 dB. Mitigating construction noise is challenging because noise emitted by construction activities and machinery contains high frequency, low frequency, and impact transient noises. The common way of canceling such noises is using active noise barriers (ANBs), which are a combination of active noise control (ANC) systems and passive noise cancellation (PNC) methods. Low-frequency noises can be canceled using ANC systems, and the PNC method is effective in canceling high-frequency noises. However, most traditional ANC methods utilize linear filters with minimal performance for the cancellation of impact sounds released by equipment such as jackhammers and rock drills or construction activities such as hammering or shoveling. Furthermore, there are practical limitations in installing ANBs because of their huge passive elements that might also impede airflow. In this research, a deep learning-based feedback ANC controller is developed to tackle the abovementioned issues. The developed method not only can mitigate both low-frequency and high-frequency noises but also can effectively cancel impact noises produced by stationary and moving noise sources. As a result, conventional ANBs can be replaced by an ANC device with the algorithm, and there is no need to install bucky PNC equipment. In fact, deep layers with nonlinear activation functions empower the proposed method to learn inherent nonlinear behaviors of various noises, electrical devices, and physical acoustic paths and produce anti-noise signals with the same amplitude and opposite phase of primary noise to eliminate them. It is shown the designed network outperformed the traditional well-known ANC algorithm, filtered-x least mean square (FxLMS), with a considerable gap.

Colorectal cancer (CRC) is a common form of cancer that affects the large intestine. CRC is one of the most severe and aggressive forms of cancer, and thus, early treatment and detection are essential. Early detection of CRC is primarily available through the detection of polyps through endoscopic imaging procedures. This method is labor-intensive and subject to human error. To circumvent these issues associated with human error and improve upon limitations associated with human detection, deep learning-based procedures have been developed and convolutional neural networks (CNNs) have been introduced for the automated detection and segmentation of polyps. Current problems associated with polyp segmentation with CNNs are overfitting, boundary pixel definitions, an inability to account for the different range of textures, sizes, and shapes with polyps, among other issues. With the ultimate goal of addressing these issues, we developed a multiscale segmentation network (MSSNet) designed specifically for polyps (Lewis and Cha, Sci Rep, 2023). In this paper, we conducted some additional case studies to investigate the performance of MSSNet. This dual model network surpasses state-of-the-art results (SOTA) and is evaluated using the CVC-ClinicDB dataset. The mean intersection-over-union (mIoU) and dice (mDice) score were 0.889 and 0.935, respectively.

Detection of colorectal polyps through colonoscopy is an essential practice in prevention of colorectal cancers. However, the method itself is labor intensive and is subject to human error. With the advent of deep learning-based methodologies, and specifically convolutional neural networks, an opportunity to improve upon the prognosis of potential patients suffering with colorectal cancer has appeared with automated detection and segmentation of polyps. Polyp segmentation is subject to a number of problems such as model overfitting and generalization, poor definition of boundary pixels, as well as the model's ability to capture the practical range in textures, sizes, and colors. In an effort to address these challenges, we propose a dual encoder-decoder solution named Polyp Segmentation Network (PSNet). Both the dual encoder and decoder were developed by the comprehensive combination of a variety of deep learning modules, including the PS encoder, transformer encoder, PS decoder, enhanced dilated transformer decoder, partial decoder, and merge module. PSNet outperforms state-of-the-art results through an extensive comparative study against 5 existing polyp datasets with respect to both mDice and mIoU at 0.863 and 0.797, respectively. With our new modified polyp dataset we obtain an mDice and mIoU of 0.941 and 0.897 respectively.

This chapter presents an autonomous navigation system based on computer vision for unmanned aerial vehicles (UAVs). The emerging area of autonomous UAV has shown rapid development in the field of structural health monitoring (SHM) for the collection of important structural data from civil infrastructure. Much of previous research has been focused on the use of magnetic compass sensor-based methods for control and localization of UAV in GPS-denied environments. However, these methods are vulnerable to magnetic interference of the surrounding environment and result in poor control of UAV with dangerous levels of path deviations. Therefore, this chapter suggests the use of unique small fiducial markers that are unsusceptible to magnetic interference and can be permanently attached to walls or ceilings of structures to enable robust autonomous flight of UAVs. A pseudo markers map is created where the location of each marker is stored in a marker's library. The markers are detected by the UAV during the flight and the prebuilt library information is used to localize UAV's position in the whole structure. It has been shown through various experiments that the use of computer vision-based technique can significantly improve the control of UAV in challenging indoor and outdoor GPS-denied environments. This marker-based localization method is a low-cost, flexible, and practical solution to realize true autonomous UAVs in civil structures such as bridges, parkade, and buildings.

This chapter presents an obstacle avoidance method (OAM) to realize an autonomous collision-avoiding unmanned aerial vehicle (UAV) for the collection of structural data from civil infrastructure. OAM is of high importance to avoid serious accidents during autonomous flights of UAVs for monitoring purposes. A collision must be avoided to avert the loss of human and financial loss. Therefore, this chapter provides a new unique real-time OAM that consists of four steps: obstacle detection, obstacle clustering, distance estimation, and generation of new waypoints. For obstacle detection, deep learning algorithm YOLOv3 is implemented, which can detect obstacles at 10 frames per second during the flight. Obstacles are detected in the form of bounding boxes in the image stream of the UAV. Next, if more than one obstacle is detected, the k-means clustering algorithm is used to group the obstacles based on their relative position and the nearest obstacle group is selected to be avoided first. Then the distance from the nearest obstacle to the UAV is estimated using monocular depth estimation and a pinhole camera model. If the obstacle is dangerously close to the UAV, it must be avoided. For this purpose, a new obstacle avoidance waypoint is generated based on the obstacle position and flight path of the UAV. Experimentally, it has been shown that our obstacle avoidance method has real-time and robust performance compared to existing state-of-the-art OAM methods.