The ability to passively reconstruct a scene in 3D provides significant benefit to Situational Awareness systems employed in security and surveillance applications. Traditionally, passive 3D scene modelling techniques, such as Shape from Silhouette, require images from multiple sensor viewpoints, acquired either through the motion of a single sensor or from multiple sensors. As a result, the application of these techniques often attracts high costs, and presents numerous practical challenges. This paper presents a 3D scene reconstruction approach based on exploiting scene shadows, which only requires information from a single static sensor. This paper demonstrates that a large amount of 3D information about a scene can be interpreted from shadows; shadows reveal the shape of objects as viewed from a solar perspective and additional perspectives are gained as the sun arcs across the sky. The approach has been tested on synthetic and real data and is shown to be capable of reconstructing 3D scene objects where traditional 3D imaging methods fail. Providing the shadows within a scene are discernible, the proposed technique is able to reconstruct 3D objects that are camouflaged, obscured or even outside of the sensor's Field of View. The proposed approach can be applied in a range of applications, for example urban surveillance, checkpoint and border control, critical infrastructure protection and for identifying concealed or suspicious objects or persons which would normally be hidden from the sensor viewpoint.
Multi-frame super-resolution of images containing complex motion fields remains an elusive target requiring precise estimation of such motion between the frames. Although accuracy of optical flow algorithms has been increasing steadily it is not yet sufficient to provide confident subpixel resolution enhancement during super-resolution reconstruction. One way to approach the problem is to augment optical flow estimation input into super-resolution procedure by reliable confidence measure. Numerous optical flow confidence or conversely uncertainty measures have been developed recently. However, their performance has been studied and evaluated with respect to error metric involving optical flow vectors, namely angular and end point errors averaged over the entire image. Super-resolution processing requires consideration of interpolation error i.e. the difference between the reference image and the motion compensated image. This work presents comparative evaluation of optical flow confidence measures and makes conclusions of their usefulness to applications utilizing image compensation.
Recent infrared (IR) sensors are mostly based on a focal-plane array (FPA) structure. However, IR images suffer from the fixed pattern noise (FPN) due to non-uniform response of a FPA structure. Various nonuniformity correction (NUC) techniques have been developed to alleviate the FPN. They can be categorized into reference-based and scene-based approaches. In order to deal with a temporal drift, however, a scene-based approach is needed. Among scene-based algorithms, conventional algorithms compensate only for the offset non-uniformity of IR camera detectors based on the global motion information. Local motions in a video, however, can introduce inaccurate motion information for NUC. Considering global and local motions simultaneously, we propose a correction algorithm of gain and offset. Experiment results using simulated and real IR videos show that the proposed algorithm provides performance improvement on the FPN reduction.
Many applications require to register images within subpixel accuracy like computer vision especially super-resolution (SR) where the estimated subpixel shifts are very crucial in the reconstruction and restoration of SR images. In our work we have an optical sensor that is mounted on an unmanned airborne vehicle (UAV) and captures a set of images that contain sufficient overlapped area required to reconstruct a SR image. Due to the wind, The UAV may encounter rotational effects such as yaw, pitch and roll which can distort the acquired as well as processed images with shear, tilt or perspective distortions. In this paper we propose a hybrid algorithm to register these UAV images within subpixel accuracy to feed them in a SR reconstruction step. Our algorithm consists of two steps. The first step uses scale invariant feature transform (SIFT) to correct the distorted images. Because the resultant images are not registered to a subpixel precision, the second step registers the images using a fast Fourier transform (FFT) based method that is both efficient and robust to moderate noise and lens optical blur. Our FFT based method reduces the dimensionality of the Fourier matrix of the cross correlation and uses a forward and backward search in order to obtain an accurate estimation of the subpixel shifts. We discuss the relation between the dimensionality reduction factors and the image shifts as well as propose criteria that can be used to optimally select these factors. Finally, we compare the results of our approach to other subpixel techniques in terms of their efficiency and computational speed.
The problem of horizontal imaging through the atmospheric boundary layer is common in defense, surveillance and remote sensing applications. Like all earth-bound imaging systems the resolving capability of an imaging system is limited by atmospheric turbulence. Using speckle imaging techniques it is often possible to overcome these effects and recover images with resolution approaching the diffraction-limit. We examine the performance of a bispectrum-based speckle imaging technique when applied to imaging scenarios near the ground. Computer simulations were used to generate three sets of 70, turbulence degraded images with varied turbulence strength. Early results indicate the bispectrum to be a robust estimator for images corrupted by the anisoplanatic turbulence encountered when imaging horizontally. Bispectrum reconstructed image frames show an improvement of nearly 60% in Mean Squared Error (MSE) on average over the examined turbulence strengths. The improvement in MSE was found to increase as additional input frames used for image reconstruction though using a few as 10 input frames provided a 50% improvement in MSE on average over turbulence strengths.
When using the conventional fixed smoothing factor to display the stabilized video, we have the issue of large undefined black border regions (BBR) when camera is fast panning and zooming. To minimize the size of BBR and also provide smooth visualization to the display, this paper discusses several novel methods that have demonstrated on a real-time platform. These methods include an IIR filter, a single Kalman filter and an interactive multi-model filter. The fundamentals of these methods are to adapt the smoothing factor to the motion change from time to time to ensure small BBR and least jitters. To further remove the residual BBR, the pixels inside the BBR are composited from the previous frames. To do that, we first store the previous images and their corresponding frame-to-frame (F2F) motions in a FIFO queue, and then start filling the black pixels from valid pixels in the nearest neighbor frame based on the F2F motion. If a matching is found, then the search is stopped and continues to the next pixel. If the search is exhausted, the pixel remains black. These algorithms have been implemented and tested in a TI DM6437 processor.
We present a spatially adaptive defogging algorithm for enhancement of color and visibility of unmanned aerial vehicle (UAV) images. It is hard to identify an object of interest from airborne image acquired by a satellite, an airplane, and a UAV because of various atmospheric distortions. For overcoming this problem, the proposed algorithm decomposes the input foggy image into the original fog-free component and the atmospherically distorted component, and then estimates the original image based on the image degradation model. We first generate a normalized image using the maximum value among RGB color channels of a foggy Image. We estimate the atmospheric light in the labeled image. We also generate a modified transmission map using the labeled image and a guided filter. A major contribution of the proposed work is the enhancement of details using a guided filter as well as defogging. We can significantly enhance the visibility of a foggy image by using the estimated atmospheric light and the transmission map. The proposed algorithm can remove foggy components better than existing defogging techniques because the specular component and the labeled image are used.
An OpenCL implementation of the Active Contours Without Edges algorithm is presented. The proposed algorithm uses the General Purpose Computing on Graphics Processing Units (GPGPU) to accelerate the original model by parallelizing the two main steps of the segmentation process, the computation of the Signed Distance Function (SDF) and the evolution of the segmented curve. The proposed scheme for the computation of the SDF is based on the iterative construction of partial Voronoi diagrams of a reduced dimension and obtains the exact Euclidean distance in a time of order O(N/p), where N is the number of pixels and p the number of processors. With high resolution images the segmentation algorithm runs 10 times faster than its equivalent sequential implementation. This work is being done as an open source software that, being programmed in OpenCL, can be used in dierent platforms allowing a broad number of nal users and can be applied in dierent areas of computer vision, like medical imaging, tracking, robotics, etc. This work uses OpenGL to visualize the algorithm results in real time.
Super-resolution (SR) refers to reconstructing a single high resolution (HR) image from a set of subsampled, blurred and noisy low resolution (LR) images. The reconstructed image suffers from degradations such as blur, aliasing, photo-detector noise and registration and fusion error. Wiener filter can be used to remove artifacts and enhance the visual quality of the reconstructed images. In this paper, we introduce a new fast stochasticWiener filter for SR reconstruction and restoration that can be implemented efficiently in the frequency domain. Our derivation depends on the continuous-discrete-continuous (CDC) model that represents most of the degradations encountered during the image-gathering and image-display processes. We incorporate a new parameter that accounts for LR images registration and fusion errors. Also, we speeded up the performance of the filter by constraining it to work on small patches of the images. Beside this, we introduce two figures of merits: information rate and maximum realizable fidelity, which can be used to assess the visual quality of the resultant images. Simulations and experimental results demonstrate that the derived Wiener filter that can be implemented efficiently in the frequency domain can reduce aliasing, blurring, and noise and result in a sharper reconstructed image. Also, Quantitative assessment using the proposed figures coincides with the visual qualitative assessment. Finally, we evaluate our filter against other SR techniques and its results were very competitive.
Many techniques have been recently developed for classification of hyperspectral images (HSI) including support vector machines (SVMs), neural networks and graph-based methods. To achieve good performances for the classification, a good feature representation of the HSI is essential. A great deal of feature extraction algorithms have been developed such as principal component analysis (PCA) and independent component analysis (ICA). Sparse coding has recently shown state-of-the-art performances in many applications including image classification. In this paper, we present a feature extraction method for HSI data motivated by a recently developed sparse coding based image representation technique. Sparse coding consists of a dictionary learning step and an encoding step. In the learning step, we compared two different methods, L1-penalized sparse coding and random selection for the dictionary learning. In the encoding step, we utilized a soft threshold activation function to obtain feature representations for HSI. We applied the proposed algorithm to a HSI dataset collected at the Kennedy Space Center (KSC) and compared our results with those obtained by a recently proposed method, supervised locally linear embedding weighted k-nearest-neighbor (SLLE-WkNN) classifier. We have achieved better performances on this dataset in terms of the overall accuracy with a random dictionary. We conclude that this simple feature extraction framework might lead to more efficient HSI classification systems.
The recently introduced non-local means (NLM) image denoising technique broke the traditional paradigm according to which image pixels are processed by their surroundings. Non-local means technique was demonstrated to outperform state-of-the art denoising techniques when applied to images in the visible. This technique is even more powerful when applied to low contrast images, which makes it tractable for denoising infrared (IR) images. In this work we investigate the performance of NLM applied to infrared images. We also present a new technique designed to speed-up the NLM filtering process. The main drawback of the NLM is the large computational time required by the process of searching similar patches. Several techniques were developed during the last years to reduce the computational burden. Here we present a new techniques designed to reduce computational cost and sustain optimal filtering results of NLM technique. We show that the new technique, which we call Multi-Resolution Search NLM (MRS-NLM), reduces significantly the computational cost of the filtering process and we present a study of its performance on IR images.
Video cameras are widely used for monitoring public areas, such as train stations, airports and shopping centers. When crowds are dense, automatically tracking individuals becomes a challenging task. We propose a new tracker which employs a particle filter tracking framework, where the state transition model is estimated by an optical-flow algorithm. In this way, the state transition model directly uses the motion dynamics across the scene, which is better than the traditional way of a pre-defined dynamic model. Our result shows that the proposed tracker performs better on different tracking challenges compared with the state-of-the-art trackers, while also improving on the quality of the result.
We look at the design of projective measurements based upon image priors. If one assumes that image patches from natural imagery can be modeled as a low rank manifold, we develop an optimality criterion for a measurement matrix based upon separating the canonical elements of the manifold prior. Any sparse image reconstruction algorithm has improved performance using the developed measurement matrix over using random projections. Some insights into the empirical estimation of the image patch manifold are developed and several results are presented.
The road network is one of the most important types of information in the Geographic Information System (GIS). However, automatic extraction of roads is still considered a challenging problem. In this paper, we focus on robust extraction of main roads. In the proposed algorithm, we first determine the roadness of each pixel using the eigenvalues of its Hessian matrix. The roadness represents the belongingness of a pixel to a road; and its determination is performed on a multi-scale basis so that it is robust to various widths of roads. We then perform directional grouping to the determined initial road map and remove outliers in each group via directionally morphological filtering. Finally, we determine roads by combining the results from each group. Experimental results show that the proposed algorithm can automatically extract most main roads in various remote sensing images.
This work aims to quantify and improve the sensitivity of monocular passive ranging techniques in sequences that are degraded by the the quasi-periodic inter frame motion and localised blurring associated with atmospheric scintillation. Ranges from tens of meters up to 1km in uncontrolled outdoor conditions were investigated. This work has application in surveillance, border control, artillery, and the autonomous navigation of ground vehicles. A field trial with long focal length, large aperture cameras was held. The targets were placed at known distances and a scintillometer was used to measure the turbulence between the cameras and each target. A variety of depth from defocus and depth from focus algorithms were applied to a the video sequences to determine range to the targets. Increasing levels of atmospheric turbulence were observed ranging from no turbulence to mild, moderate and severe turbulence. Algorithms for the mitigation of atmospheric turbulence were applied to the recorded video sequences. The passive ranging techniques were then applied to the turbulence-mitigated video sequences and the results compared to that of the raw sequences. Finally, findings regarding the best performing combination of ranging and mitigation techniques to use under different scintillation conditions are presented.
In this paper, we present a novel high dynamic range (HDR) imaging method using a single input image. Conventional multiple image-based HDR methods are successful only on condition that there is no motion in the scene during the acquisition of multiple, differently exposed low dynamic range (LDR) images. If these constraints are not satisfied, a ghost artifact is produced in the resulting HDR image. In order to overcome these limitations, we generate multiple, differently exposed LDR images from a single input image. We call these multiple images a set of layered exposed (LE) images. In order to generate an appropriate set of LE images, the proposed method divides input image into 9 subregions and computes local mean in each subregion to estimate the minimum and maximum local mean. The estimated local means become ranges of histogram equalization (HE). HDR image is generated by fusing differently exposed LE images. More specifically, given a set of LE images, we perform weighted fusion to produce the resulting HDR image, which is inherently free from ghost artifacts since all LE images are geometrically identical. Experimental results show that the proposed method outperforms the existing algorithms in the sense of both removing ghost artifacts and enhancement of image contrast.
In high resolution remote sensing satellite imaging system, image restoration is an important step to visualize ne details and mitigate the noise. The raw image data often presents poor imaging quality due to various reasons and Point Spread Function (PSF) measures such blurriness characteristic of the image using point source. Satellite image from Korea Multi-purpose Satellite 2 (KOMPSAT-2) also requires Modular Transfer Function (MTF) compensation process to achieve more realistic image which entails removing ringing artifacts at the edges and restraining excess use of denoising eect in order to keep it more realistic. This paper focuses on the deconvolution of KOMPSAT-2 image utilizing PSF attained from Korea Aerospace Research Institute compared to deconvolution with the estimated PSF blur kernel. The deconvolution algorithm considered are Richard-Lucy, Damped Richard-Lucy, Bilateral Richard-Lucy and Sparse Prior deconvolution algorithms.
In general edges are considered to be abrupt changes or discontinuities in two dimensional image signal intensity distributions. The accuracy of front-end edge detection methods in image processing impacts the eventual success of higher level pattern analysis downstream. To generalize edge detectors designed from a simple ideal step function model to real distortions in natural images, research on one dimensional edge pattern analysis to improve the accuracy of edge detection and localization proposes an edge detection algorithm, which is composed by three basic edge patterns, such as ramp, impulse, and step. After mathematical analysis, general rules for edge representation based upon the classification of edge types into three categories-ramp, impulse, and step (RIS) are developed to reduce detection and localization errors, especially reducing "double edge" effect that is one important drawback to the derivative method.But, when applying one dimensional edge pattern in two dimensional image processing, a new issue is naturally raised that the edge detector should correct marking inflections or junctions of edges. Research on human visual perception of objects and information theory pointed out that a pattern lexicon of "inflection micro-patterns" has larger information than a straight line. Also, research on scene perception gave an idea that contours have larger information are more important factor to determine the success of scene categorization. Therefore, inflections or junctions are extremely useful features, whose accurate description and reconstruction are significant in solving correspondence problems in computer vision. Therefore, aside from adoption of edge pattern analysis, inflection or junction characterization is also utilized to extend traditional derivative edge detection algorithm. Experiments were conducted to test my propositions about edge detection and localization accuracy improvements. The results support the idea that these edge detection method improvements are effective in enhancing the accuracy of edge detection and localization.
Blur estimation is an important technique for super resolution, image restoration, turbulence mitigation, deblurring and autofocus. Low-cost methods have been proposed for blur estimation. However, they can have large stochastic errors when computed close to the edge location and biased estimates at other locations. In this paper, we define an efficient, accurate and precise estimate that can be computed at the edge location based on the first-order derivative. Our method is compared and benchmarked against previous state-of-the-art. The results show that the proposed method is fast, unbiased and with low stochastic error.
Previously we introduced the concept of continuous quantification of uniqueness, as a general purpose technique designed to be applicable to any situation in which there is a need to decide which of several equally effective objects to choose for a task, that requires recognition of the chosen object, in a variety of contexts, by comparing attributes which contain a non trivial amount of context dependent variability. We defined that uniqueness assessment as an algorithm that computes a fuzzy set membership function that measures some but not all aspects of the probability that the sought after object will not be confused with other objects in the space being searched. We evaluated the usefulness of that concept by experimentally assessing the extent to which the uniqueness of the SAD global minimum of locally computed image subset dissimilarity was both a predictor of bidirectional match compliance with the Epipolar Constraint, and a predictor of bidirectional match disparity correctness, for the classical stereoscopic correspondence problem of computer vision, and in that context found the uniqueness of the aforementioned global minimum to be a useful but imperfect predictor of success. In this paper we compare the usefulness of the uniqueness of the aforementioned global minimum to that of, the magnitude of that same global minimum, the magnitude of variability across contributors to that global minimum, uniqueness of that variability, and co-occurrence of the global minimum of local image subset dissimilarity and global minimum of variability across contributors to local image subset dissimilarity.
