Zero -shot learning (ZSL) is a framework to classify images that belong to unseen visual classes using their semantic descriptions about the unseen classes. We develop a new ZSL algorithm based on coupled dictionary learning. The core idea is to enforce the visual features and the semantic attributes of an image to share the same sparse representation in an intermediate embedding space, modeled as the shared input space of two sparsifying dictionaries. In the ZSL training stage, we use images from a number of seen classes for which we have access to both the visual and the semantic attributes to train two coupled dictionaries that can represent both the visual and the semantic feature vectors of an image using a single sparse vector. In the ZSL testing stage and in the absence of labeled data, images from unseen classes are mapped into the attribute space by finding the joint -sparse representations using solely the visual dictionary via solving a LASSO problem. The image is then classified in the attribute space given semantic descriptions of unseen classes. We also provide attribute -aware and transductive formulations to tackle the "domain-shift"and the "hubness"challenges for ZSL, respectively. Experiments on four primary datasets using VGG19 and GoogleNet visual features, are provided. Our performances using VGG19 features are 91.0%, 48.4%, and 89.3% on the SUN, the CUB, and the AwA1 datasets, respectively. Our performances on the SUN, the CUB, and the AwA2 datasets are 57.0%,49.7%, and 71.7%, respectively, when GoogleNet features are used. Comparison with existing methods demonstrates that our method is effective and compares favorably against the state-of-the-art. In particular, our algorithm leads to decent performance on the all four datasets.2

A classic approach toward zero-shot learning (ZSL) is to map the input domain to a set of semantically meaningful attributes that could be used later on to classify unseen classes of data (e.g. visual data). In this paper, we propose to learn a visual feature dictionary that has semantically meaningful atoms. Such a dictionary is learned via joint dictionary learning for the visual domain and the attribute domain, while enforcing the same sparse coding for both dictionaries. Our novel attribute aware formulation provides an algorithmic solution to the domain shift/hubness problem in ZSL. Upon learning the joint dictionaries, images from unseen classes can be mapped into the attribute space by finding the attribute aware joint sparse representation using solely the visual data. We demonstrate that our approach provides superior or comparable performance to that of the state of the art on benchmark datasets.

Multi-waveband infrared (IR) sensors capture more spectral information of atmospheric particles and may provide better penetration thru dust under dynamically changing conditions. Therefore, enhancing the visibility of multi-waveband infrared images obtained in degraded visual environment (DVE) is an important way to improve the perception of the environment under DVE conditions. In this paper, we present a system to enhance visibility in DVE conditions by modifying the wavelet coefficients of multi-waveband IR images. In the proposed system, input multi-waveband IR images are transferred into the wavelet domain using an integer lifting wavelet transformation. The low-frequency wavelet coefficients in each waveband are independently modified by an adaptive histogram equalization technique for improving the contrast of the images. To process high-frequency wavelet coefficients, a joint edge-mapping filter is applied to the multi-waveband high-frequency wavelet coefficients to find an edge map for each subband of wavelet coefficients; then a nonlinear filter is used to remove noise and enhance edge coefficients. Finally, the inverse lifting wavelet transformation is applied to the modified multi-waveband wavelet coefficients to obtain enhanced multi-waveband IR images. We tested the proposed system with degraded multi-waveband IR images obtained from a helicopter landing in brownout conditions. Our experimental results show that the proposed system is effective for enhancing the visibility of multi-waveband IR images under DVE conditions.

We describe a foveated compressive sensing approach for image analysis applications that utilizes knowledge of the task to be performed to reduce the number of required sensor measurements and sensor size, weight, and power (SWAP) compared to conventional Nyquist sampling and compressive sensing-based approaches. Our Compressive Optical Foveated Architecture (COFA) adapts the dictionary and compressive measurements to structure and sparsity in the signal, task, and scene by reducing measurement and dictionary mutual coherence and increasing sparsity using principles of actionable information and foveated compressive sensing. Actionable information is used to extract task-relevant regions of interest (ROIs) from a low-resolution scene analysis by eliminating the effects of nuisances for occlusion and anomalous motion detection. From the extracted ROIs, preferential measurements are taken using foveation as part of the compressive sensing adaptation process. The task-specific measurement matrix is optimized by using a novel saliency weighted coherence minimization with respect to the learned signal dictionary. This incorporates the relative usage of the atoms in the dictionary. We utilize a patch-based method to learn the signal priors. A tree-structured dictionary of image patches using K-SVD is learned which can sparsely represent any given image patch with the tree structure. We have implemented COFA in an end-to-end simulation of a vehicle fingerprinting task for aerial surveillance using foveated compressive measurements adapted to hierarchical ROIs consisting of background, roads, and vehicles. Our results show 113x reduction in measurements over conventional sensing and 28x reduction over compressive sensing using random measurements.

We address zero-shot learning using a new manifold alignment framework based on a localized multi-scale transform on graphs. Our inference approach includes a smoothness criterion for a function mapping nodes on a graph (visual representation) onto a linear space (semantic representation), which we optimize using multi-scale graph wavelets. The robustness of the ensuing scheme allows us to operate with automatically generated semantic annotations, resulting in an algorithm that is entirely free of manual supervision, and yet improves the state-of-the-art as measured on benchmark datasets.

Enhancing the visibility of infrared images obtained in a degraded visibility environment is very important for many applications such as surveillance, visual navigation in bad weather, and helicopter landing in brownout conditions. In this paper, we present an IR image visibility enhancement system based on adaptively modifying the wavelet coefficients of the images. In our proposed system, input images are first filtered by a histogram-based dynamic range filter in order to remove sensor noise and convert the input images into 8-bit dynamic range for efficient processing and display. By utilizing a wavelet transformation, we modify the image intensity distribution and enhance image edges simultaneously. In the wavelet domain, low frequency wavelet coefficients contain original image intensity distribution while high frequency wavelet coefficients contain edge information for the original images. To modify the image intensity distribution, an adaptive histogram equalization technique is applied to the low frequency wavelet coefficients while to enhance image edges, an adaptive edge enhancement technique is applied to the high frequency wavelet coefficients. An inverse wavelet transformation is applied to the modified wavelet coefficients to obtain intensity images with enhanced visibility. Finally, a Gaussian filter is used to remove blocking artifacts introduced by the adaptive techniques. Since wavelet transformation uses down-sampling to obtain low frequency wavelet coefficients, histogram equalization of low-frequency coefficients is computationally more efficient than histogram equalization of the original images. We tested the proposed system with degraded IR images obtained from a helicopter landing in brownout conditions. Our experimental results show that the proposed system is effective for enhancing the visibility of degraded IR images.

LIDAR devices for on-vehicle use need a wide field of view and good fidelity. For instance, a LIDAR for avoidance of landing collisions by a helicopter needs to see a wide field of view and show reasonable details of the area. The same is true for an online LIDAR scanning device placed on an automobile. In this paper, we describe a LIDAR system with full color and enhanced resolution that has an effective vertical scanning range of 60 degrees with a central 20 degree fovea. The extended range with fovea is achieved by using two standard Velodyne 32-HDL LIDARs placed head to head and counter rotating. The HDL LIDARS each scan 40 degrees vertical and a full 360 degrees horizontal with an outdoor effective range of 100 meters. By positioning them head to head, they overlap by 20 degrees. This creates a double density fovea. The LIDAR returns from the two Velodyne sensors do not natively contain color. In order to add color, a Point Grey LadyBug panoramic camera is used to gather color data of the scene. In the first stage of our system, the two LIDAR point clouds and the LadyBug video are fused in real time at a frame rate of 10 Hz. A second stage is used to intelligently interpolate the point cloud and increase its resolution by approximately four times while maintaining accuracy with respect to the 3D scene. By using GPGPU programming, we can compute this at 10 Hz. Our backfilling interpolation methods works by first computing local linear approximations from the perspective of the LIDAR depth map. The color features from the image are used to select point cloud support points that are the best points in a local group for building the local linear approximations. This makes the colored point cloud more detailed while maintaining fidelity to the 3D scene. Our system also makes objects appearing in the PanDAR display easier to recognize for a human operator.

This paper addresses the problem of finding small and low-contrast moving targets in infrared (IR) video sequences produced by sensors with inconsistent parameters, such as intensity offset and gain as well as bad pixels. This sensor variability makes it difficult to apply methods based on frame registration using simple pixel differences. Our proposed algorithm uses regression to normalize the variations of intensity offset and gain between compared registered frames. A statistical criterion is used to calculate the threshold for the difference between normalized intensities of two frames. The algorithm for finding the differences between frames is also used to create a bad pixel mask either on-or offline. This mask is essential for the reduction of false detection rates. Our experiments show that this approach produces good results and can be used for detection of small, low-contrast targets in high dynamic range IR data. The proposed algorithm also produces good results for detecting moving targets in cases when objects are occluded by sparse vegetation. (C) 2015 Society of Photo-Optical Instrumentation Engineers (SPIE)