The US Federal Highway Administration (FHWA) vehicle classification scheme is designed to serve multiple transportation needs, such as road infrastructure design, pavement maintenance scheduling, and traffic-induced emission estimation. Although a plethora of studies have advanced computer vision-based techniques for vehicle classification, no vision-based method has yet achieved the desired level of accuracy for all 13 FHWA vehicle category classifications primarily due to the interclass similarity issue, particularly among trucks. To fill this gap, this study developed a two-stage vision-based method that leverages both semantic and geometric features extracted from surveillance videos. In the first stage, a cascaded Mask R-CNN model is employed to classify vehicles into six broad categories based on semantic features (i.e., vehicle appearance). In the second stage, the geometric features (i.e., axle configuration) are extracted and exploited to further classify trucks into nine specific FHWA categories. Additionally, a verification scheme is introduced to validate the classification results with the aim of filtering out the misclassified vehicles and improving the overall classification accuracy. The proposed method was evaluated through field experiments under different traffic scenarios. It achieved an overall classification accuracy of 98.6% across all 13 FHWA vehicle categories, with an additional 1.1% improvement thanks to the verification scheme. This study contributes to the body of knowledge by introducing a more accurate method for FHWA vehicle classification, particularly for trucks, expanding the potential of image-based techniques in a variety of intelligent transportation applications.

Tack coat is a thin asphalt applied between the existing surface and asphalt overlay during road rehabilitation. Its coverage uniformity is vital in providing adhesive bonding between the two layers in the structure. This paper proposes a novel method for automatic uniformity inspection of tack coat coverage using images collected by drones. In this method, color thresholding is first performed to extract coarse region of the tack coat. Then, the boundaries of the tack coat are detected to refine the tack coat region by masking outliers. Next, the region of interest is processed to extract texture features using gray-level cooccurrence. Finally, the texture features are leveraged to grade the tack coat uniformity level using LightGBM. Experiments were conducted to examine the performance of the method. The resulting accuracy signifies the potential of this method in developing high-efficiency and cost-effective solutions to enhance the current tack coat inspection practice.

Unmanned aerial vehicles (UAVs) have recently become popular in collecting positional data of road users. In comparison to other tools at ground level, UAVs have the advantages of low cost, wider view coverage, and significantly less occlusion. However, the depth relief of road users and the perspective distortion of the onboard camera induce nonnegligible error while applying UAVs for localization of road users. This study proposed a method for accurate road user localization in aerial images. First, the localization error induced by the depth relief and perspective distortion was examined and modeled. Then, a deep-learning-based method was applied for automatic road user detection and localization in the aerial images by leveraging oriented bounding boxes to achieve higher localization accuracy compared to applying horizontal bounding boxes. Finally, an error compensation strategy was proposed to eliminate the perspective- and depth-relief-induced localization error by rectifying the oriented bounding boxes obtained from the previous step. Field experiments were conducted to evaluate the method's performance. The results demonstrated its promising accuracy for road user location estimation and its potential to improve the reliability of UAVs in traffic applications.

Unmanned aerial vehicles (UAVs) have become increasingly popular for traffic data collection. However, the depth relief of road users and the perspective distortion of the onboard camera induce nonnegligible measurement errors while exploiting UAVs to localize road users. To address this issue, this paper presents a method for accurate road user location estimation in aerial images. First, a deep-learning-based method was employed to detect road users in aerial images using oriented bounding boxes. Then, the localization error induced by the depth relief and perspective distortion was examined and modeled, based on which an error compensation scheme was developed to offset the localization error for each road user so that higher localization accuracy is attainable. Field experiments were conducted to evaluate the proposed method's performance. The results demonstrated a promising accuracy in estimating the location of road users, signifying the method's potential to improve the credibility of UAVs in traffic applications.

Field-of-view calibration is essential for establishing the relationship between 2D image coordinates inside the camera and 3D real-world coordinates of the traffic scene. For many surveillance-based traffic applications, the field-of-view calibration involves precisely extracting the three orthogonal vanishing points. However, many traffic scenes lack parallel lines along all three dominant directions, making it difficult to successfully calibrate using the present methods. This study proposed a novel method for estimating the three mutually orthogonal vanishing points in traffic scenes. To determine the dominant directions of the real-world coordinate frame, this method exploits the visual features of both the road environment and moving vehicles and avoids the need for parameter tuning through trial and error in different scenarios. To evaluate the performance of the proposed method, laboratory tests were conducted. The outcomes demonstrated the potential of the method in traffic scenes with scarce parallel line features.

Digitalization of real-world traffic scenes is a fundamental task in development of digital twins of road transportation. However, the existing digitalization approaches are either expensive in equipment costs or inapplicable to collect granular level data of traffic scenes. This study proposed a vision-based method for real-time digitalization of traffic scenes through modeling and merging the road infrastructure (static components) and road users (dynamic components) progressively. Specifically, the former is reconstructed by leveraging unmanned aerial vehicles (UAVs) and structure from motion; and the latter is digitized via using roadside surveillance videos and a new reconstruction process through applying deep learning and view geometry. Last, the digital model of the traffic scene is built by merging the digital models of static and dynamic components. A field experiment was performed to evaluate the performance of the proposed method. The results showed that the traffic scene can be successfully digitalized by the proposed method with promising accuracy, thus signifying the method's potential for the development of the digital twins of road transportation in support of intelligent transportation applications.

Overheight vehicle collisions continuously pose a serious threat to transportation infrastructure and public safety. This study proposed a vision‐based method for automatic vehicle height measurement using deep learning and view geometry. In this method, vehicle instances are first segmented from traffic surveillance video frames by exploiting mask region‐based convolutional neural network (Mask R‐CNN). Then, 3D bounding box on each vehicle instance is constructed using the obtained vehicle silhouette and three orthogonal vanishing points in the surveilled traffic scene. By doing so, the vertical edges of the constructed 3D bounding box are directly associated with the vehicle image height. Last, the vehicle's physical height is computed by referencing an object with a known height in the traffic scene using single view metrology. A field experiment was performed to evaluate the performance of the proposed method, leading to the mean and maximum errors of 3.6 and 6.6, 5.8 and 12.9, 4.4 and 8.1, and 9.2 and 18.5 cm for cars, buses, vans, and trucks, respectively. The experiment also demonstrated the ability of the method to overcome vehicle occlusion, shadow, and irregular appearance interferences in height estimation suffered by existing image‐based methods. The results signified the potential of the proposed method for overheight vehicle detection and collision warning in real traffic settings.

The emergence of autonomous vehicles (AVs) provides a sustainable solution to reshape the current transportation system and help mitigate the negative environmental impacts from transportation activities. However, the AVs may undergo unreliable and insufficient perception by the onboard sensors due to occlusion issues and complex traffic conditions, especially in crowded urban intersections. This study proposed a vision-based method for automated detection and localization of road users in traffic scenarios by leveraging surveillance videos. In this method, the traffic scenario is surveilled in a large visual range and the locations of all surveilled road users are determined. The field experiment was conducted to evaluate the performance of the proposed method. The experiment results demonstrated the promising accuracy of the proposed method for road user location estimation and its potential to provide the AVs with a full-participant perception in complex traffic scenarios and assist them to make the right driving decisions.

It is well accepted that data normalization is an essential and imperative step in using the direct linear transformation (DLT) method for homography estimation. Nevertheless, the existing data normalization methods are either point or line-based, thus making them inapplicable in scenarios where both point and line correspondences are available. To fill this gap and further widen the applicability of the DLT method, this study proposed a unified data normalization method for homography estimation using combined point and line correspondences. In this method, the existing point-based normalization method was exploited and then extended to normalize the line coefficients by conducting extra data processing created by the authors. Finally, a unified normalization method was developed by fusing the normalization principles implemented on the point coordinates and line coefficients alone, which allows for choices of using different combinations of point and line correspondences for homography estimation. The field results showed that the proposed method can significantly improve the accuracy and offer higher reliability for homography estimation.

Computer vision-based over-height vehicle detection has received increasing attention for preventing vehicle-to-structure collisions. However, the prior studies usually rely on blob detection of vehicles for vehicle height measurement, which is oftentimes error-prone because the traditional blob detection methods are sensitive to shadow interference and illumination change. Also, it is practically difficult to extract the vehicle’s pixel height from the detected blobs automatically and accurately due to the lack of depth information on 2D images, especially for vehicles with irregular top shapes. Furthermore, the applications of the existing methods are still plagued by the occlusion issue (e.g., part of a vehicle is blocked by another vehicle) that commonly occurred in surveilled views. Therefore, this research proposed a new method for vehicle height estimation. In the proposed method, the vehicle instances are first segmented from images using Mask R-CNN, which can significantly eliminate the adverse effect of shadow interference and illumination change. Then, 3D bounding box is established for every surveilled vehicle with or without being partially blocked by leveraging the identified vehicle instance and the three orthogonal vanishing points in the surveilled scene. By doing so, the vertical edges of established 3D bounding box are directly associated with the vehicle’s height, thus overcoming the difficulty of identifying the vehicle’s pixel height in 2D images. Finally, the vehicle’s physical height is estimated using a known object length in the same surveilled scene according to view geometry. The evaluation results signify the potential of the proposed method for use in over-height collision prevention.

The thermal effect that leads to spurious image drift is one major concern in the videogrammetric measurement. On the ground that the image drift originates from the thermal-induced displacement of the image sensor board, this study proposes to establish the thermal-induced displacement function of the image plane for the prediction of image drift and the elimination thereof. The displacement function of the image plane is calibrated with stationary targets easily available in laboratory. In measurements other than the thermal effect calibration, the thermal-induced displacement of the image plane is predicted from the precalibrated displacement function. It is then subtracted from the measured pixel coordinates of measurand, resulting in normalized pixel coordinates in which the thermal-induced image drift has been removed. To exemplify this approach, two thermal effect calibration tests with different camera orientations are fulfilled. The displacement function of the image plane is simulated with a first-order polynomial model. Verification tests are performed in both temperature-controlled and outdoor environments. The approach achieves satisfactory performance in both environments.

The stochastic subspace identification (SSI) method has been recognized as the most dominant and popular system identification technique in the time domain. Nevertheless, it cannot cope with the non-synchronicity of dynamic response measurements that happens sometimes in structural health monitoring practice. To overcome this, this study proposes a strategy for the SSI algorithm to realize system identification from non-synchronous dynamic response measurements. The modal identification is carried out in a pairwise manner by pairing a time-shifted signal with a common reference signal. The state space model is employed to fit the pair of reference and time-shifted signals and the SSI algorithm is exploited to extract the modal parameters. The core of the strategy is the tactical usage of the mean phase deviation (MPD) of the mode for seeking out the actual time lag as well as the actual mode shape components simultaneously. By so doing, the strategy fulfills integrated time lag estimation and modal extraction for non-synchronous dynamic response measurements. Furthermore, the strategy also overcomes the difficulty to determine the model order with the use of the stabilization diagram and reduces the computational cost substantially with the help of the periodicity of the MPD. To examine the performance of the strategy, intensive validations are conducted by making use of the non-synchronous acceleration measurements of the Jiangyin Bridge subjected to a ship collision and the artificially misaligned acceleration measurements of the Canton Tower struck by an earthquake. Both the time lags and the mode shapes identified by the strategy can be well validated, indicating that the proposed strategy is competent for modal identification of non-synchronous dynamic response measurements.

Compaction is an important task in asphalt pavement construction. It ensures high quality of the pavement to provide required long‐term performance. Intelligent compaction (IC) integrated with global positioning system (GPS) is an innovative technology to help control the compaction quality of asphalt pavement by continuously providing the operator with the roller's current position and the number of rolling passes. However, the high equipment cost and signal disturbance are two identified issues when adopting the GPS for position estimation. This paper proposed a thermal‐based method for real‐time roller path tracking and mapping in pavement compaction operations. In this method, first, the incremental roller motion between each two consecutive frames was jointly estimated via calculation of the roller's heading direction by exploiting the pavement boundary information and the roller's translation using the optical flow technique. Then, the accuracy of the estimated incremental motion was optimized by incorporation of the roller's lateral position estimated by a position optimization model proposed by the authors. Finally, the roller's global location was estimated by chaining frame‐by‐frame roller motions recursively. Both laboratory and field validations were conducted to examine the performance of the proposed method, leading to the maximum mean absolute errors along the lateral direction to be 5.2 mm and 3.3 cm, respectively, and the maximum cumulative error rates along the longitudinal direction to be 0.85% and 0.73%, respectively. The validation results signified the method's potential as a low‐cost and applicable alternative to the GPS in the current IC technology for roller's position tracking and mapping.

There has been a sustained interest towards the non-contact structural displacement measurement by means of videogrammetric technique. On the way forward, one of the major concerns is the spurious image drift induced by temperature variation. This study therefore carries out an investigation into the temperature effect of videogrammetric technique, focusing on the exploration of the mechanism behind the temperature effect and the elimination of the temperature-caused measurement error. 2D videogrammetric measurement tests under monotonic or cyclic temperature variation are first performed. Features of measurement error and the casual relationship between temperature variation and measurement error are then studied. The variation of the temperature of digital camera is identified as the main cause of measurement error. An excellent linear relationship between them is revealed. After that, camera parameters are extracted from the mapping between world coordinates and pixels coordinates of the calibration targets. The coordinates of principle point and focal lengths show variations well correlated with temperature variation. The measurement error is thought to be an outcome mainly attributed to the variation of the coordinates of principle point. An approach for eliminating temperature-caused measurement error is finally proposed. Correlation models between camera parameters and temperature are formulated. Thereby, camera parameters under different temperature conditions can be predicted and the camera projective matrix can be updated accordingly. By reconstructing the world coordinates with the updated camera projective matrix, the temperature-caused measurement error is eliminated. A satisfactory performance has been achieved by the proposed approach in eliminating the temperature-caused measurement error.

Compaction is one of the most important phases in construction of asphalt concrete (AC) pavements, as it directly affects the density and thereby the performance of pavements. This paper proposed a thermal-based compaction technology for real-time roller path tracking and mapping in pavement compaction operations, based on which roller operators can better control their compaction quality. In the proposed method, the incremental change of a roller position in a short interval was decomposed into two motion components (i.e., the change in heading direction and the linear translation). The global position of the roller was then recovered by chaining the frame-by-frame motion in terms of their changes. Two sets of experimental data from different pavement construction sites were used to test the performance of the proposed technology. The results showed that the developed technology is a promising alternative to the current GPS-based intelligent compaction (IC) in pavement compaction operations.

Videogrammetry has demonstrated great potential in structural health monitoring (SHM), and there has been sustained interest towards applying this non-contact technique in SHM. This paper focuses on the effect of temperature variation on the measurement accuracy of videogrammetry to examine its feasibility in deformation/displacement monitoring. A long-term indoor videogrammetric measurement test was conducted, and the performance of the videogrammetric displacement monitoring technique under ambient temperature was examined. The result showed that temperature variation does cause non-negligible errors which contain not only daily fluctuation pattern but also overall trend. In terms of daily fluctuation pattern, the horizontal measurement error and indoor air temperature are in satisfactory consistency, while the vertical measurement error are not. In terms of overall trend, the vertical measurement error is highly correlated with indoor air temperature, having a positive linear relationship between them. However, the horizontal one has a complicated pattern when temperature varies.

There has been an increasing number of attempts to apply videogrammetric technique to displacement measurement of civil engineering structures. Its potentials in structural health monitoring have also gained more attention. This study carried out an investigation on the effect of temperature variation on the measurement accuracy of videogrammetric technique in an effort to examine its feasibility for structural health monitoring. Long-term indoor videogrammetric measurement tests have been conducted, and the performance of the videogrammetric displacement monitoring technique under ambient temperature conditions has been examined. The results show that temperature variations cause non-negligible errors in measured displacements. In line with the temperature variation, the displacement measurement error also contains not only daily fluctuation pattern but also overall trend. In terms of daily fluctuation pattern, the horizontal measurement error and temperatures of vision measurement system are in satisfactory consistency, while the vertical measurement error does not coincide well with temperatures of vision measurement system. In terms of overall trend, the vertical measurement error is highly correlated with temperatures of vision measurement system, while the horizontal one is almost uncorrelated with temperatures of vision measurement system. As an outcome of the dominance of overall trend in the temperature variation over a long time period, the vertical measurement error and temperatures of vision measurement system conform to a favorable linear relationship, while the horizontal measurement error tends to be constrained in a small range when the temperatures of vision measurement system exceed a certain value.