Edge computing is becoming very popular. Many researchers are using edge devices for computing artificial intelligence, especially in real-time applications of computer vision by the use of Convolutional Neural Networks (CNNs). Edge devices can encounter non-ideal environments that impact the effectiveness of a CNN model's predictions. We propose a tool flow methodology for implementing a real-time intelligent image enhancement technique for deploying and validating CNNs on edge devices. In this work, we implement this proposed methodology in a modular and user-friendly way. We show that this methodology is capable of real-time intelligent image enhancement on various CNN models.

The human visual system is not very sensitive to the absolute luminance of an image, but rather responds to local luminance changes, i.e., the gradient of an image. In this paper, we propose a constrained optimization approach for image gradient enhancement. The gradient strength of the enhanced image can be controlled directly using a target gradient strength parameter in the cost function. To suppress artifacts and ensure that contrast improves, a novel constraint is included in the optimization. Due to the number of variables in optimization-based image enhancement techniques being equal to the number of gray scales, we quantize the image using a $k$ -means clustering-based histogram mergence (KCHM) method before enhancement. KCHM can significantly reduce the number of image gray scales while effectively preserving the subjective quality. This is useful considering the reduction of variables is good for solving optimization and reducing computation cost. Experimental results demonstrate that the proposed method can significantly improve the subjective image quality by enhancing both the contrast and the image gradient.

This book provides ample coverage of theoretical and experimental state-of-the-art work as well as new trends and directions in the biometrics field. It offers students and software engineers a thorough understanding of how some core low-level building blocks of a multi-biometric system are implemented. While this book covers a range of biometric traits, its main emphasis is placed on multi-sensory and multi-modal face biometrics algorithms and systems.

This work centers around a state of the art gamma and neutron radiation detector, which is able to display radiation information about its surrounding at every second. Information about the count level is displayed along with the energy range of the detected radiation. Currently, similar sensors are only capable of passive radiation detection and function in stationary positions on the sides of highways, the entrances to buildings, and at major intersections. The sensor has design considerations which allow it to move about and be more aggressive in its search. Its small size and low profile design allow it to be mounted on a robotic platform and search in small spaces such as underneath a car. Ultimately it would be able to autonomously localize an unknown number of dangerous radioactive sources, allowing workers to stay at a safe distance. The sensor has been characterized using exempt Cs-137 and Co-60 gamma sources. A thorough profile was done of the measurements made by the sensor at varying distances with different combinations of radioisotopes present. Based on the findings, an algorithm was developed to scan an area similar to that of the wheelbase of a car and record the radiation levels. These measurements are used to localize from one to three radioactive sources.

An appropriate response to an emergency situation involving hazardous materials requires a good and accurate knowledge of the characteristics of the materials in question. Inspection and characterization of materials collected from hazardous environments is also of great value to first responders and security personnel. In this paper, we describe 3D modeling of material surfaces from stereo images obtained from a Large Chamber Scanning Electron Microscope (LC-SEM), a one of a kind microscope in the US. By applying an annealing based two-step energy minimization technique, the stereo images are reconstructed into 3D. A virtual 3D view obtained from the stereo pair verifies the validity of the automatic 3D model constructed. The spectral information at two energy levels, one for AlK and the other for NbL, is extracted using Energy Dispersive X-ray Spectroscopy (EDS) and overlaid on the 3D surface. Thus, in addition to performing 3D metrology on the surface, one is able to visually inspect the distribution characteristics of constituent materials and correlate them with surface structure such as creases or dents caused by fracture or other impacts in hazardous environments.

Novel image fusion approaches, including physics-based weighted fusion, illumination adjustment and rank-based decision level fusion, for spectral face images are proposed for improving face recognition performance compared to conventional images. A new multispectral imaging system is briefly presented which can acquire continuous spectral face images for our concept proof with fine spectral resolution in the visible spectrum. Several experiments are designed and validated by calculating the cumulative match characteristics of probe sets via the well-known recognition engine-FaceIt®. Experimental results demonstrate that proposed fusion methods outperform conventional images when gallery and probes are acquired under different illuminations and with different time lapses. In the case where probe images are acquired outdoors under different daylight situations, the fused images outperform conventional images by up to 78%.

Most existing camera placement algorithms focus on coverage and/or visibility analysis, which ensures that the object of interest is visible in the camera's field of view (FOV). According to recent literature, handoff safety margin is introduced to sensor planning so that sufficient overlapped FOVs among adjacent cameras are reserved for successful and smooth target transition. In this paper, we investigate the sensor planning problem when considering the dynamic interactions between moving targets and observing cameras. The probability of camera overload is explored to model the aforementioned interactions. The introduction of the probability of camera overload also considers the limitation that a given camera can simultaneously monitor or track a fixed number of targets and incorporates the target's dynamics into sensor planning. The resulting camera placement not only achieves the optimal balance between coverage and handoff success rate but also maintains the optimal balance in environments with various target densities. The proposed camera placement method is compared with a reference algorithm by Erdem and Sclaroff. Consistently improved handoff success rate is illustrated via experiments using typical office floor plans with various target densities.

Camera handoff is a crucial step to obtain a continuously tracked and consistently labeled trajectory of the object of interest in multi-camera surveillance systems. Most existing camera handoff algorithms concentrate on data association, namely consistent labeling, where images of the same object are identified across different cameras. However, there exist many unsolved questions in developing an efficient camera handoff algorithm. In this paper, we first design a trackability measure to quantitatively evaluate the effectiveness of object tracking so that camera handoff can be triggered timely and the camera to which the object of interest is transferred can be selected optimally. Three components are considered: resolution, distance to the edge of the camera’s field of view (FOV), and occlusion. In addition, most existing real-time object tracking systems see a decrease in the frame rate as the number of tracked objects increases. To address this issue, our handoff algorithm employs an adaptive resource management mechanism to dynamically allocate cameras’ resources to multiple objects with different priorities so that the required minimum frame rate is maintained. Experimental results illustrate that the proposed camera handoff algorithm can achieve a substantially improved overall tracking rate by 20% in comparison with the algorithm presented by Khan and Shah.

In order to achieve improved recognition performance in comparison with conventional broadband images, this paper addresses a new method that automatically specifies the optimal spectral range for multispectral face images according to given illuminations. The novelty of our method lies in the introduction of a distribution separation measure and the selection of the optimal spectral range by ranking these separation values. The selected spectral ranges are consistent with the physics analysis of the multispectral imaging process. The fused images from these chosen spectral ranges are verified to outperform the conventional broadband images by 3%-20%, based on a variety of experiments with indoor and outdoor illuminations using two well-recognized face-recognition engines. Our discovery can be practically used for a new customized sensor design associated with given illuminations for improved face-recognition performance over the conventional broadband images.

To achieve size preserving tracking, in addition to controlling the camera's pan and tilt motions to keep the object of interest in the camera's field of view (FOV), the camera's focal length is adjusted automatically to compensate for the changes in the target's image size caused by the relative motion between the camera and the target. The estimation accuracy of these changes determines the effectiveness of the resulting zoom control. The existing method of choice for real-time target scale estimation applies structure from motion (SFM) based on the weak perspective projection model. In this paper we propose a target scale estimation algorithm with a linear solution based on the more advanced paraperspective projection model, which improves the accuracy of scale estimation by considering center offset. Another key issue in SFM based algorithms is the separation of target and background features, especially when composite camera (pan/tilt/zoom) and target motions are involved. This paper designs a fast target feature separation/grouping algorithm, the 3D affine shape method. The resulting separation automatically adapts to the target's 3D geometry and motion and is able to accommodate a large amount of off-plane rotation, which most existing separation/grouping algorithms find difficult to achieve. Experimental results illustrate the effectiveness of the proposed scale estimation and feature separation algorithms in tracking translating and rotating objects with a PTZ camera while preserving their sizes. In comparison with the leading size preserving tracking algorithm described by Tordoff and Murray, our algorithm is able to reduce the cumulative tracking error significantly from 17.4% to 3.3%.

In a multi-camera surveillance system, both camera handoff and placement play an important role in generating an automated and persistent object tracking, typical of most surveillance requirements. Camera handoff should comprise three fundamental components, time to trigger handoff process, the execution of consistent labeling, and the selection of the next optimal camera. In this paper, we design an observation measure to quantitatively formulate the effectiveness of object tracking so that we can trigger camera handoff timely and select the next camera appropriately before the tracked object falls out of the field of view (FOV) of the currently observing camera. In the meantime, we present a novel solution to the consistent labeling problem in omnidirectional cameras. A spatial mapping procedure is proposed to consider both the noise inherent to the tracking algorithms used by the system and the lens distortion introduced by omnidirectional cameras. This does not only avoid the tedious process, but also increases the accuracy, to obtain the correspondence between omnidirectional cameras without human interventions. We also propose to use the Wilcoxon Signed-Rank Test to improve the accuracy of trajectory association between pairs of objects. In addition, since we need a certain amount of time to successfully carry out the camera handoff procedure, we introduce an additional constraint to optimally reserve sufficient cameras’ overlapped FOVs for the camera placement. Experiments show that our proposed observation measure can quantitatively formulate the effectiveness of tracking, so that camera handoff can smoothly transfer objects of interest. Meanwhile, our proposed consistent labeling approach can perform as accurately as the geometry-based approach without tedious calibration processes and outperform Calderara’s homography-based approach. Our proposed camera placement method exhibits a significant increase in the camera handoff success rate at the cost of slightly decreased coverage, as compared to Erdem and Sclaroff’s method without considering the requirement on overlapped FOVs.

Most existing camera placement algorithms focus on coverage and/or visibility analysis, which ensures that the object of interest is visible in the camera's field of view (FOV). However, visibility, which is a fundamental requirement of object tracking, is insufficient for automated persistent surveillance. In such applications, a continuous consistently labeled trajectory of the same object should be maintained across different camera views. Therefore, a sufficient uniform overlap between the cameras' FOVs should be secured so that camera handoff can successfully and automatically be executed before the object of interest becomes untraceable or unidentifiable. In this paper, we propose sensor-planning methods that improve existing algorithms by adding handoff rate analysis. Observation measures are designed for various types of cameras so that the proposed sensor-planning algorithm is general and applicable to scenarios with different types of cameras. The proposed sensor-planning algorithm preserves necessary uniform overlapped FOVs between adjacent cameras for an optimal balance between coverage and handoff success rate. In addition, special considerations such as resolution and frontal-view requirements are addressed using two approaches: 1) direct constraint and 2) adaptive weights. The resulting camera placement is compared with a reference algorithm published by Erdem and Sclaroff. Significantly improved handoff success rates and frontal-view percentages are illustrated via experiments using indoor and outdoor floor plans of various scales.

In this paper, we present the application of two linear machine learning techniques; ridge regression and kernel regression for the estimation of illumination chromaticity. A number of machine learning techniques, neural networks and support vector machines in particular, are used to estimate the illumination chromaticity. These nonlinear approaches are shown to outperform many traditional algorithms. However, neither neural networks nor support vector machines were compared to linear regression tools in the past. We evaluate the performance of linear machine learning techniques and draw comparison with nonlinear machine learning techniques. Kernel regression achieves a mean root mean square chromaticity error of 0.052 while neural network results in 0.071. An improvement of 26% is achieved. Both quantitative and qualitative results show that the performances of the linear techniques are better when compared to nonlinear techniques on the same data set. Machine learning approaches are also compared with the gray-world and the scale by max algorithms. We perform uncertainty analysis of machine learning algorithms using a bootstrapped training data set to evaluate their consistency in the estimation of illumination chromaticity. Applications like video tracking and target detection, where illumination chromaticity estimation is important will be benefited by a better performance of linear machine learning algorithms.

Dual-camera systems have been widely used in surveillance because of the ability to explore the wide field of view (FOV) of the omnidirectional camera and the wide zoom range of the PTZ camera. Most existing algorithms require a priori knowledge of the omnidirectional camera's projection model to solve the nonlinear spatial correspondences between the two cameras. To overcome this limitation, two methods are proposed: 1) geometry and 2) homography calibration, where polynomials with automated model selection are used to approximate the camera's projection model and spatial mapping, respectively. The proposed methods not only improve the mapping accuracy by reducing its dependence on the knowledge of the projection model but also feature reduced computations and improved flexibility in adjusting to varying system configurations. Although the fusion of multiple cameras has attracted increasing attention, most existing algorithms assume comparable FOV and resolution levels among multiple cameras. Different FOV and resolution levels of the omnidirectional and PTZ cameras result in another critical issue in practical tracking applications. The omnidirectional camera is capable of multiple object tracking while the PTZ camera is able to track one individual target at one time to maintain the required resolution. It becomes necessary for the PTZ camera to distribute its observation time among multiple objects and visit them in sequence. Therefore, this paper addresses a novel scheme where an optimal visiting sequence of the PTZ camera is obtained so that in a given period of time the PTZ camera automatically visits multiple detected motions in a target-hopping manner. The effectiveness of the proposed algorithms is illustrated via extensive experiments using both synthetic and real tracking data and comparisons with two reference systems.

In this paper, we provide objective measures to evaluate compression methods for machine recognition applications. Vidware Vision, a black box compression method developed by Vidware Incorporated, is used in the case and its performance is compared to existing compression methods, i.e., JPEG and JPEG 2000, based on various measures. The encoding and decoding time are used to characterize computational complexity. Fulland noreference image quality measures are exploited to describe distortions and degradations in the decompressed images. In addition, since this paper focuses on the performance of compression methods relating to machine recognition applications, we propose a non-separable rational function based Tenengrad (NSRT2) measure to evaluate the sharpness of decompressed images. Based on our experimental results, Vidware Vision TM is robust to changes in compression ratio and presents gradually degraded performance at a considerably slower speed in terms of computational complexity and image quality. Particularly, according to full-reference measures Vidware Vision outperforms JPEG and JPEG 2000 when the compression ratio is larger than 140. The effectiveness of our proposed NSRT2, as a new comparison tool, is also validated via experiments and performance comparisons with other tested measures. .

A Markov Random Field (MRF) based local stereo matching algorithm that estimates parameters automatically from statistics is proposed. For an iterative optimization, cost functions working on local support neighborhood are developed. Data model parameters are pre-estimated from one of the stereo images by applying a noise equivalence hypothesis. The smoothness model parameters are estimated with maximum likelihood (ML) applying disparity gradient constraint and 3*sigma confidence boundary. The confidence boundary also defines the parameters for handling discontinuities in data and smoothness. Additionally, homogeneous points are included into the support neighborhood to achieve high matching rate along surface borders. Finally, a pair of cost functions is modeled to match the images symmetrically for improved matching. Experiments on ground truth datasets show that among the existing algorithms with statistical estimation of the parameters, the proposed algorithm delivers the highest matching rate.

When imaging a sample, it is desirable to have the entire area of interest in focus in the acquired image. Typically, microscopes have a limited depth of field (DOF) and this makes the acquisition of such an all-in-focus image difficult. This is a major problem in many microscopic applications and applies equally in the realm of scanning electron microscopy as well. In multifocus fusion, the central idea is to acquire focal information from multiple images at different focal planes and fuse them into one all-in-focus image where all the focal planes appear to be in focus.Large chamber scanning electron microscopes (LC-SEM) are one of the latest members in the SEM family that has found extensive use for nondestructive evaluations. Large objects (~1 meter) can be scanned in micro- or nano-scale using this microscope. An LC-SEM can provide characterization of conductive and non-conductive surfaces with a magnification from 10× to 200,000×. The LC-SEM, as with other SEMs, suffers from the problem of limited DOF making it difficult to inspect a large object while keeping all areas in focus.

In this paper, we propose a series of techniques to enhance the computational performance of existing Belief Propagation (BP) based stereo matching that relies on automatic estimation of the Markov random field (MRF) parameters. First, we show how convergence in matching can be achieved faster than with the existing message comparison technique by skipping comparisons in early inferences. Second, assuming that a stereo pair is captured with identical cameras, we apply a hypothesis called noise equivalence to pre-estimate the likelihood parameters and thus, avoid costly nested inferences to reduce the computational time. The likelihood parameters and intensity information are used for accelerated message propagation in image regions lacking gradients. Third, the prior model parameters are estimated with a combination of maximum likelihood (ML) estimation and disparity gradient constraint to further reduce the computational time. Supporting experiments for the proposed algorithms show encouraging results on ground truth test images.

Although sensor planning in computer vision has been a subject of research for over two decades, a vast majority of the research seems to concentrate on two particular applications in a rather limited context of laboratory and industrial workbenches, namely 3D object reconstruction and robotic arm manipulation. Recently, increasing interest is engaged in research to come up with solutions that provide wide-area autonomous surveillance systems for object characterization and situation awareness, which involves portable, wireless, and/or Internet connected radar, digital video, and/or infrared sensors. The prominent research problems associated with multisensor integration for wide-area surveillance are modality selection, sensor planning, data fusion, and data exchange (communication) among multiple sensors. Thus, the requirements and constraints to be addressed include far-field view, wide coverage, high resolution, cooperative sensors, adaptive sensing modalities, dynamic objects, and uncontrolled environments. This article summarizes a new survey and analysis conducted in light of these challenging requirements and constraints. It involves techniques and strategies from work done in the areas of sensor fusion, sensor networks, smart sensing, Geographic Information Systems (GIS), photogrammetry, and other intelligent systems where finding optimal solutions to the placement and deployment of multimodal sensors covering a wide area is important. While techniques covered in this survey are applicable to many wide-area environments such as traffic monitoring, airport terminal surveillance, parking lot surveillance, etc., our examples will be drawn mainly from such applications as harbor security and long-range face recognition.

Most existing camera placement algorithms focus on coverage and/or visibility analysis, which ensures that the object of interest is visible in the camera's field of view (FOV). However, visibility, a fundamental requirement of object tracking, is insufficient for persistent and automated tracking. In such applications, a continuous and consistently labeled trajectory of the same object should be maintained across different cameraspsila views. Therefore, a sufficient overlap between the cameraspsila FOVs should be secured so that camera handoff can be executed successfully and automatically before the object of interest becomes untraceable or unidentifiable. The proposed sensor planning method improves existing algorithms by adding handoff rate analysis, which preserves necessary overlapped FOVs for an optimal handoff success rate. In addition, special considerations such as resolution and frontal view requirements are addressed using two approaches: direct constraint and adaptive weight. The resulting camera placement is compared with a reference algorithm by Erdem and Sclaroff. Significantly improved handoff success rate and frontal view percentage are illustrated via experiments using typical office floor plans.