In this paper, we present a Head‐Up Display (HUD) system that displays perspective correct surrounding information for driver safety. In our implementation, we display contrast enhanced ego lane information on the HUD during driving conditions with poor visibility. The use of HUD for Augmented Reality (AR) needs an accurate model of the picture generation process for proper visualization of the 3D content that is perspective accurate from the user view point. The system uses multiple computer vision techniques for the accurate visualization.First, we use a camera calibration algorithm that uses view independent spatial geometry and view dependent perspective transformation that compensates for the distortion of the optics as well as the combiner. The calibration is done by capturing the test patterns from the front facing camera located close to the rear view mirror and a camera from the driver's point of view. Further registration of the HUD image is done through a head tracker that uses the camera mounted for the driver monitoring on the steering wheel. The second module detects the lane markings on the road and returns the location of the polygon in the camera co‐ordinate system. The image data bounded by this polygon is further enhanced to reduce the effect of haze, rain and insufficient illumination and blended with the camera input from the front facing camera. The HUD then projects the synthesized image that is perspective correct rendered in the vehicle co‐ordinate system.

We propose a fast and robust 2D-affine global motion estimation algorithm based on phase-correlation in the Fourier-Mellin domain and robust least square model fitting of sparse motion vector field and its application for digital image stabilization. Rotation-scale-translation (RST) approximation of affine parameters is obtained at the coarsest level of the image pyramid, thus ensuring convergence for a much larger range of motions. Despite working at the coarsest resolution level, using subpixel-accurate phase correlation provides sufficiently accurate coarse estimates for the subsequent refinement stage of the algorithm. The refinement stage consists of RANSAC based robust least-square model fitting for sparse motion vector field, estimated using block-based subpixel-accurate phase correlation at randomly selected high activity regions in finest level of image pyramid. Resulting algorithm is very robust to outliers such as foreground objects and flat regions. We investigate the robustness of the proposed method for digital image stabilization application. Experimental results show that the proposed algorithm is capable of estimating larger range of motions as compared to another phase correlation method and optical flow algorithm.

With the advent of high frame rate driving in Liquid Crystal Displays (LCD), the need for high quality frame rate up-conversion has become very important. Various motion-compensated frame rate up-conversion (MCFRC) algorithms [1],[2] and [3] have been developed to interpolate moving objects along their motion trajectories. In addition to robust motion estimation, accurate occlusion handling is also very essential for high quality interpolated video frames. Incorrect handling of occlusion can lead to artifacts that manifest itself as halo around moving objects and can be extremely objectionable. Many occlusion handling algorithms have been proposed in the literature over the years. Use of cascaded median filters was proposed in [4] to handle occlusion. Intensity mismatch [2] and the number of projected vectors in neighboring blocks [5] were also used to detect occlusion. In this paper, we present a motion compensated video upconversion algorithm that produces occlusion free video. The algorithm is categorized in two steps. In the first stage occlusion area is detected using motion vector clustering and segmentation and in the next stage occlusion region is handled using motion vector steered video in-painting. This algorithm does not introduce one additional frame latency that is required for occlusion handling by other methods such as [6].

The final frontier that has eluded the Liquid Crystal Displays to equal the image quality of the impulsive type displays is the inability to render a sharp image when objects are in motion. This motion-blur issue is a significant problem for large Full-HD LCD where the large size and high resolution makes the impairment of the image very easy to see. Driving the LCDs at double frame-rate, 100 or 120 Hz, has the potential to remove this issue but it requires very complex processing. In this paper we will discuss the advanced video processing required for high frame rate driving of LCD TVs.