This paper proposes a tightly coupled visual-IMU odometry. Before fusing visual and inertial measurements, a real time and robust image matching algorithm which is based on inertial information is proposed. Delaunay triangulation and three consecutive image matching method are employed to remove sporadic outliers. Also, non-maximum-suppression and inertial information are used to decrease feature searching time. Consequently, a tightly coupled UKF fusion method is used to estimate ego-motion. The trifocal tensor geometry relationship between three consecutive images is used as measurement information which is wrapped into a RANSAC scheme to be robust against outliers. Therefore, the proposed method can handle well in dynamic environment. The experiments show the effectiveness of the proposed method in KITTI data which is a publicly available real world dataset.

In order to improve the picking speed and accuracy of robot, the objects detection and localization algorithm based on Mask RCNN and stereo vision is designed to complete the autonomous detection and 3D spatial location of the target to be detected. Aiming at the problem that the detection accuracy of the neural network may be low and the object contour centroid estimation is not accurate, the ORB descriptor is used to confirm the target contour matching centroid. The experimental results show that the proposed algorithm can accurately accomplish the object detection and localization, and it is of great significance for the research of fully automatic picking robots.

This paper proposes a tightly coupled visual-IMU odometry. Before fusing visual and inertial measurements, a real time and robust image matching algorithm which is based on inertial information is proposed. Delaunay triangulation and three consecutive image matching method are employed to remove sporadic outliers. Also, non-maximum-suppression and inertial information are used to decrease feature searching time. Consequently, a tightly coupled UKF fusion method is used to estimate ego-motion. The trifocal tensor geometry relationship between three consecutive images is used as measurement information which is wrapped into a RANSAC scheme to be robust against outliers. Therefore, the proposed method handles well in dynamic environment. In the experiments phase the KITTI data is used which is a publicly available real world dataset and has rich dynamic information. Visual odometry experiments indicate that the proposed method gives a better result between pure IMU reckoning and image motion estimation. Although the algorithm shows the effectiveness in KITTI data, it has outliers in feature matching which will give a false motion evaluation result. Future work will focus on improving the confidence level of feature matching.

Image matching is the core of the computer vision. The pyramidal image matching algorithm based on inertial in-formation is proposed to solve the problems of long computation time and sparse matching points,which exist in the current algo-rithms. Two-level image pyramid is used and the non-maximum suppression method is employed to control the number of feature points in this algorithm. When matching the four images,the algorithm makes use of the epipolar constraint and inertial informa-tion to constraint the searching space,and compares the sum of absolute differences(SAD). The simulation experimental results show that the algorithm can control the obtained matching points,and improve the real-time performance and the degree of confi-dence of the matching points.