Labor shortage due to the declining birth rate has become a serious problem in the construction industry, and automation of construction work is attracting attention as a solution to this problem. This paper proposes a method to realize state estimation of dump truck position, orientation and articulation angle using multiple GNSS for automatic operation of dump trucks. RTKGNSS is commonly used for automation of construction equipment, but in mountainous areas, mobile networks often unstable, and RTK-GNSS using GNSS reference stations cannot be used. Therefore, this paper develops a state estimation method for dump trucks that does not require a GNSS reference station by using the Centimeter Level Augmentation Service (CLAS) of the Japanese Quasi-Zenith Satellite System (QZSS). Although CLAS is capable of centimeter-level position estimation, its positioning accuracy and ambiguity fix rate are lower than those of RTK-GNSS. To solve this problem, we construct a state estimation method by factor graph optimization that combines CLAS positioning and moving-base RTK-GNSS between multiple GNSS antennas. Evaluation tests under real-world environments have shown that the proposed method can estimate the state of dump trucks with the same accuracy as conventional RTK-GNSS, but does not require a GNSS reference station.

One of the most challenging tasks for dump trucks to automate in earthmoving operations is the unloading of earth and sand in an area. The human operator performs the unloading task as much earth and sand as possible within the area empirically. Automation of this process requires the proposal of a new method to realize the work that is done empirically by humans. Therefore, I propose a new method to predict the shape of the sediment pile after unloading based on the characteristics of the pile, and to determine the unloading position based on the predicted shape of the pile.

This paper describes the effect of applying spiral model to the development process of robot system for a new entrant company. The robot system was developed to remotely control a conventional hydraulic excavator in order to improve the safety of operators in disaster emergency restoration. The issues of development are the definition of requirements and integration for a practical system in a real environment by a new entrant company. The constraints to the new entry of smaller companies are the following three points. (1) Lack of industry knowledge and data to define requirements (2) Lack of on-site environment and machinery for investigation and testing (3) Lack of experience in robot development To solve the problems under these constraints, the spiral model divides the development based on the prototype into 4-steps, and repeats this series of processes. This method was applied to clarify the necessary functions and performance of the robot step by step, and to construct a system with robustness in a real environment. As a result, this robot system has been successfully utilized in emergency disaster recovery tasks due to landslides, and removing debris in the Fukushima Daiichi Nuclear Power Plant, reducing the mental and physical burden on the operators.

This paper describes autonomous driving of dump truck using a retrofit control device and its field application at a quarry site. The retrofit control system is installed to an unconventional dump truck so that the authors considered the installation position of the equipment on the chassis. In addition, the authors newly developed a measurement device of steering angle and bed angle. Experimental results show that the dump truck can autonomously drive in the actual quarry site including winded narrow path and slopes.

Against the backdrop of a declining and aging workers and efforts to improve productivity at construction sites, there is a strong demand for automation of construction vehicles. We have developed autonomous dump trucks using retrofit technologies. This paper describes automatic earth and sand transportation of large-size 6 wheel dump trucks in cooperation with a human-operated backhoe. The automation system uses a drone to measure the 3D terrain, a retrofit driving robot and AI to control the dump truck, and sensor boxes to measure the backhoe operation. This achievement contributes to the automation of construction vehicles owned by construction company and rental companies.

A large-scale dump truck that automatically transports earth and sand in cooperation with a human-operated backhoe is of interest to the construction industry. A human-operated dump truck generally drives slightly past the desired loading position and then backs up to it for loading the sediment. The turning and loading positions are subjectively decided according to the working posture of the backhoe and the surrounding environment, and the safety margin of cooperative works. Backhoe operators want to perform the same maneuvers for human-operated/automated dump trucks. The movements of the autonomous vehicle should be similar to those of a human-operated one. However, it is difficult to derive a human-like path that does more than minimize costs. This study proposes a path-planning method that generates a path including a turning back, according to the changing backhoe position and surrounding conditions. We modeled the positional relationship during loading between a backhoe and dump truck, determining the loading and turning positions and related parameters from operational data collected in trials with human-operated construction vehicles. The proposed method allowed the autonomous dump truck path to resemble a human-like one. The authors have retrofitted an existing large-scale six-wheeled dump truck for automatic operation. Automatic loading in cooperation with a human-operated backhoe was realized all 17 times using the retrofitted dump. The average stopping accuracy was 0.57 m and 9.7°.

There is an urgent need to automate earthmoving activities for large-scale six-wheeled dump trucks using retrofitted sensors and driving robots. A dump truck stops at a loading position with its bed facing the backhoe to fill the bed with sediment quickly. It also stops with its bed facing the leaving position. Therefore, it is necessary to plan ahead to safely and accurately enter and exit those working positions. However, existing path planners did not generate a path with a turning point that is suitable for large-scale six-wheeled dump truck loading sediment. In this paper, we propose an online pathplanning method with the following main features. First, a path is planned to move the dump truck forward and/or backward to the working position to load and/or leave sediment. Second, when a new working position is designated, a path from the turning position to the new goal position is automatically established. We verified the proposed method using a simulator of the dump truck (3.5 m × 11 m), showing that it is possible to properly account for the turning radius of the dump truck. The result reveals that it is possible to replan with a working position shift of 4.8 m or 33° from a 25 m distance and that the time required for replanning can be shortened to ≤ 7.56 s. We also confirmed that the large-scale autonomous dump truck can automatically stop at the loading position in cooperation with a humanoperated backhoe 10 out of 10 times with a stopping accuracy of 0.57 m and 5.29°.

We are trying to automate the work of unloading soil by installing retrofit equipment on large dump trucks. To automate the work, it is necessary to establish a standard for the unloading position. However, no method has been established for determining the optimal unloading position because the unloading of soil is done by the experience and feeling of the operator. To determine the best position, we considered how to unload more soil at the designated unloading location. Using simulator that can represent a variety of soil types, we investigated the relationship between different soil types, unloading position, and the amount and shape of soil piled in the virtual space.

This paper describes an improvement of pneumatic motor response for autonomous steering of conventional six-wheel dump trucks. Pneumatic motors are robust against overload, so it is considered suitable for the steering actuator of six-wheel dump trucks from the hardware point. However, a disadvantage of the pneumatic motor is the low performance of path tracking of the vehicle because of phase delay and dead time of the motor. Therefore, to improve the problem, in this research, the following four points were implemented: (1) modeling of the plant, (2) identification of model parameters, (3) designing feedforward controller with the inverse model of the plant, (4) evaluation of responsiveness by simulation. According to the above implementations, the delay time was reduced by about 70% for the application range of the feedforward compensation. Furthermore, by clarifying the pneumatic motor's applicable condition limits, the design guideline of the path planning for dump truck according to the vehicle speed is introduced.

There is an urgent need to automate earthmoving works for large-scale six-wheeled dump trucks. At actual sites, a dump truck stops at a loading position, informed by the backhoe's backet, with its bed facing the backhoe. Therefore, it is necessary to plan a path including a turning back and rapidly replan according to the goal position change. However, there is no suitable path-planning method for earthmoving works of a large-scale six-wheeled dump truck. Here we propose a path-planning method that moves the dump truck forward and then backward to the working position. According to the goal position change, it also rapidly generates a path from the turning position to the new goal position. We confirmed the proposed path resembled the trajectory of human-operated dump trucks, and the time required for replanning was shortened. We also confirmed the autonomous dump truck could do earthmoving works in cooperation with a human-operated backhoe.

The autonomous dump truck must judge whether the backhoe is ready for loading the sediment for smooth and safe cooperation with the human-operated backhoes. Analyzing time-series data of the human-operated backhoe loading motion is an effective method to build a prediction model. The transition of several primitive motions enables us to predict the timing when the backhoe starts loading. However, in transition modeling, manually selecting the appropriate primitive motions in the pre-loading motions requires considerable effort. In addition, a robust loading prediction is required for sensor layout changes in the installations of them. Here, we propose a BP-HMM-based prediction method of the pre-loading motion. The algorithm automatically finds the transition of several primitive motions from time-series data and its annotation labels. The selection of three angular velocities as features of the BP-HMM increases the robustness of the prediction method. The proposed method built a suitable prediction model for three different combinations of operators and backhoes. The prediction method was robust for sensor layout changes, and showed an accuracy of 100%. The proposed prediction method contributes to the automation of earthmoving work by enabling smooth cooperation between autonomous dump trucks and human-operated backhoe.

In Japan, expectations for the automation of construction machines are increasing to solve the labor shortage in the construction industry. In this research, a robotization method by retrofitting a robot to conventional construction machines is introduced to lower the introduction barrier for regional construction companies. The target machine is a six-wheeled dump truck. With a retrofitted internal sensor unit and derived kinematics of six-wheeled articulated dump truck, a conventional Global Navigation Satellite System (GNSS)-based path tracking method was implemented on it. In addition, to ensure safety during operation, an emergency stop function was installed on the dump truck with three-dimensional Light Detection and Ranging (3D LiDAR). Initial experiments of forward and backward path tracking with an actual dump truck confirmed the validity of the method, and the maximum tracking error was 1 m. Further, in an emergency stop experiment, the dump truck detected the obstacle and stopped immediately after obstacle detection within the emergency-stop region, i.e., 25 m x 3 m in front of the dump truck. Based on the initial experiments, the authors concluded that even the retrofitted conventional dump truck could perform basic functions for autonomous driving, such as path tracking and emergency stop.

We develop a large-size dump truck that automatically transports sediments loaded by human-operated backhoes. The dump truck is equipped with a driver robot named SAM and sensors. The driver robot can drive the dump truck remotely and autonomously. However, the difficulty of loading sediments is to decide the suitable dump truck parking location that changes according to the working position of the backhoe. Here, this paper proposes a method to decide the dump truck loading location. We proposed two ideas for obtaining the loading location. These two ideas were evaluated based on the human-operated dump truck loading data. We confirmed that the actual dump truck automatically parked the loading location based on the backhoe cabin.

Due to the declining birthrate and aging population, the shortage of labor in the construction industry has become a serious problem, and increasing attention has been paid to automation of construction equipment. We focus on the automatic operation of articulated six-wheel dump trucks at construction sites. For the automatic operation of the dump trucks, it is important to estimate the position and the articulated angle of the dump trucks with high accuracy. In this study, we propose a method for estimating the state of a dump truck by using four global navigation satellite systems (GNSSs) installed on an articulated dump truck and a graph optimization method that utilizes the redundancy of multiple GNSSs. By adding real-time kinematic (RTK)-GNSS constraints and geometric constraints between the four antennas, the proposed method can robustly estimate the position and articulation angle even in environments where GNSS satellites are partially blocked. As a result of evaluating the accuracy of the proposed method through field tests, it was confirmed that the articulated angle could be estimated with an accuracy of 0.1∘ in an open-sky environment and 0.7∘ in a mountainous area simulating an elevation angle of 45∘ where GNSS satellites are blocked.

Semantic maps are an important tool to provide robots with high-level knowledge about the environment, enabling them to better react to and interact with their surroundings. However, as a single measurement of the environment is solely a snapshot of a specific time, it does not necessarily reflect the underlying semantics. In this work, we propose a method to create a semantic map of a construction site by fusing multiple daily data. The construction site is measured by an unmanned aerial vehicle (UAV) equipped with a LiDAR. We extract clusters above ground level from the measurements and classify them using either a random forest or a deep learning based classifier. Furthermore, we combine the classification results of several measurements to generalize the classification of the single measurements and create a general semantic map of the working site. We measured two construction fields for our evaluation. The classification models can achieve an average intersection over union (IoU) score of 69.2% during classification on the Sanbongi field, which is used for training, validation and testing and an IoU score of 49.16% on a hold-out testing field. In a final step, we show how the semantic map can be employed to suggest a parking spot for a dump truck, and in addition, show that the semantic map can be utilized to improve path planning inside the construction site.

Backhoe loads sediment onto the bed of dump trucks during earthmoving work. The prediction of backhoe loading time is essential for ensuring safe cooperation between the backhoe and dump trucks. However, it is difficult to predict the instant at which the backhoe is ready to load sediment, because of the similarity in motions observed during gathering sediment. Moreover, since operators have different skill levels, the prediction requires a unique model for each operator. In this study, we attempt to predict the instant at which the backhoe is ready to load sediment into the dump truck. For this purpose, the beta-process hidden Markov model (BP-HMM) is employed to build a backhoe motion model for a specific operator. Time series data of backhoe loading motions for crushed rocks and wood chips, which were measured using 6-axis inertial measurement unit (IMU) sensors equipped at the cab, boom, and arm of the backhoe, were used for modeling with the BP-HMM. Several primitive motions of the backhoe, which occur at the completion of preparation before the loading process begins, were discovered as a result of the motion modeling based on the BP-HMM. We developed the prediction of the instant using three primitive motions. At best, the proposed method could predict the instant with a probability of 67% and 100%, at 6.0 s and 0.7 s before the loading motions began, respectively. This phased prediction can be used to reduce the idle time and risk for dump trucks during earthmoving work with the backhoe.

In the construction industry of Japan, the number of labors, particularly experienced labors, is decreasing by decreasing birthrate and aging population. In addition, the number of death in the construction industry is one-third of the number in all industries. To resolve the above problems, autonomous construction system has been researched by various companies and research institutes. In this research, we aim at the realization of autonomous surface compression work by vibration roller with environment-installed sensors. Our approach is to install plural LiDARs (Light Detection and Ranging) in the work field and estimate the position of vibration roller. In this paper, we propose a position estimation method based on the environment-installed LiDARs, path planning for a vibration roller, and path tracking control of it. Furthermore, we conducted indoor experiments to confirm the proposed system using the original 1/10 vibration roller model and outdoor experiments to confirm the accuracy of position estimation method in real-time using an actual vibration roller.

Development of autonomous construction vehicles is needed because of the shortage and aging of skilled workers. Our purpose is to realize motions of loading and unloading sand and gravel by an autonomous dump truck. This paper proposes a path with two waypoints in front of the goal to realize a switchback. As a result, an autonomous dump truck can go to the turning position first, and then go back to the goal while keeping its wheels straight. We achieved to reproduce the motion of a dump truck including switchback by the proposed method.