The US Department of Defense has a need to successfully navigate in operational regions where GPS is degraded or denied. When GPS is denied, navigation of aerial platforms, including manned and unmanned aerial vehicles (UAVs), for Intelligence, Surveillance, and Reconnaissance (ISR) missions, targeting missions, or autonomous cargo delivery missions, becomes compromised. In the absence of GPS, navigating from pure inertial solutions leads to rapidly growing position errors due to drift in the inertial measurement unit. Vision Aided Navigation (VAN) approaches can aid the inertial solution to reduce navigation error, but require salient and distinct scene content for image alignment. In this paper, we present an approach to optimal path planning for VAN over operational ground regions that minimizes navigation position error. The approach uses automated pre-mission visual fiducial discovery to identify regions in imagery of the fly-over area that contain unique, salient, discriminative, and stable feature content. The discovered visual fiducial regions are used to form a map of probabilities of successful VAN at each point of the gridded fly-over region. An optimal path planning algorithm uses the probability map to determine the path over the fly-over region that maximizes navigability and minimizes VAN positioning error. Constraints, such as no-fly zones and path length constraints, are incorporated into the formulation to generate a constrained optimization problem. We present the mathematical formulation of the constrained path planning optimization problem and generate numerical results demonstrating performance.

Image correlation has proven useful for image filtering, matching, pattern recognition, and image registration over many decades. The two classical correlation forms, amplitude and phase correlation, display different properties. Amplitude correlation often provides a low, broad peak in the correlation domain. The broadness of the peak provides robustness to matching imagery exhibiting non-translational geometric offsets, such as rotation or scale differences. By contrast, phase correlation tends to provide a high, narrow peak. The high peak signifies high matching confidence while the narrow peak width provides accurate shift localization. However, the phase correlation peak degrades rapidly when matching against images with non-translational geometric offset. To provide tradeoffs between properties of these traditional correlation forms, in this paper we present a general, flexible form of correlation called Spectrally-Shaped Correlation (SSC). SSC provides control over the Fourier domain normalization of the correlation components. We apply SSC to the problem of image registration. We show how SSC contains the classical amplitude, phase, and phase-only correlation forms as special cases. First, we present the general theory of Fourier transforms for multi-channel imagery, modeled as hypercomplex-valued imagery. We present mathematical details of the transform techniques and develop the SSC approach. We then present numerical results demonstrating registration of real image data, acquired from a UAV operating in an urban environment, to reference imagery. We demonstrate a performance improvement of the SSC over the classical forms of correlation.

While Machine Learning (ML) Automatic Target Recognition (ATR) represents the state-of-the art in target recognition, model-based ATR plays a valuable role. Model-based ATR complements machine learning ATR approaches by filling a near-term niche. While explainable Artificial Intelligence (AI) is not yet fully realized, model-based ATR serves to validate machine learning recognition decisions, and thus instills confidence in ML target calls. Alternatively, model-based ATR can act as a stand-alone ATR component, particularly in scenarios in which a small number of targets are of interest, e.g., "target-of-the-day" engagements. Model-based ATR approaches need no training data, and thus provide an alternative to machine learning approaches in the absence of sufficient quantities of real, or sufficiently high-fidelity synthetic, training data. In this paper, we present an approach to model-based ATR, called Shape-Based ATR (SB-ATR), which captures salient target shape information for recognizing targets in wide-area satellite imagery. SB-ATR finds the right blend of coarse 3-D target shape abstraction and target realism to provide robustness against target variations and environmental operating conditions, while simultaneously providing high-performance target recognition. The approach uses newer, robust forms of image correlation for matching a predicted target shape against the image. Shape prediction searches over target pose, and uses satellite metadata and solar geometry to generate realistic target shape and shadow predictions. The correlation matchers provide tolerance to illumination variations, moderate occlusions, image distortions and noise, and geometric differences between models and real targets. We present technical details of the shape-based approach, and provide numerical target recognition results on real-world satellite imagery demonstrating performance.

Modern satellites tag their images with geolocation information using GPS and star tracking systems. Depending on the quality of the geopositioning equipment, errors may range from a few meters to tens of meters on the ground. At the current state of art, there is no established method to automatically correct these errors limiting the large-scale joint utilization of cross-platform satellite images. In this paper, an automatic geolocation correction framework that corrects images from multiple satellites simultaneously is presented. As a result of the proposed correction process, all the images are effectively registered to the same absolute geodetic coordinate frame. The usability and the quality of the correction framework are demonstrated through a 3-D surface reconstruction application. The 3-D surface models given by original satellite geopositioning metadata, and the corrected metadata, are compared. The quality difference is measured through an entropy-based metric applied to the orthographic height maps given by the 3-D surface models. Measuring the absolute accuracy of the framework is harder due to lack of publicly available high-precision ground surveys. However, the geolocation of images of exemplar satellites from different parts of the globe are corrected, and the road networks given by OpenStreetMap are projected onto the images using original and corrected metadata to demonstrate the improved quality of alignment.

High-resolution and accurate Digital Elevation Model (DEM) generation from satellite imagery is a challenging problem. In this work, a stereo 3-D reconstruction framework is outlined that is applicable to nonstereoscopic satellite image pairs that may be captured by different satellites. The orthographic height maps given by stereo reconstruction are compared to height maps given by a multiview approach based on Probabilistic Volumetric Representation (PVR). Height map qualities are measured in comparison to manually prepared ground-truth height maps in three sites from different parts of the world with urban, semi-urban and rural features. The results along with strengths and weaknesses of the two techniques are summarized.

The complexity of modern defense systems requires a holistic approach to modeling system performance. Reductionist methods fail to capture emergent behaviors and other complex behaviors characteristic of these systems. For sensor systems that include ground moving target indicator (GMTI) radar sensors, having a model for probability of detection is useful for simulating, analyzing, and evaluating system performance. To this end, this paper presents mathematical modeling of detection performance of a GMTI ground radar sensor system. Specifically, we define a detection scenario in which a single ground mover within the radar scan sector is randomly distributed on a road network according to traffic loading constraints. We perform mathematical modeling to generate a general truth expression for probability of ground mover detection over general road networks. We present special cases in which closed-form expressions for the probability of detection exist. We generate empirical detection results using Monte Carlo simulation with a GMTI radar simulation to validate the mathematical modeling.

This paper presents a geo-localization framework of street-level outdoor images using multiple sources of overhead reference imagery including LIDAR, Digital Elevation Maps and Multi-Spectral Land Cover/Use imagery. We describe five different matchers and an adaptive linear fusion process which combines individual matchers' probability maps into a single map. These matchers exploit mountain elevation profiles, rendered camera views, landmarks, landuse classes and building heights. We successfully validated our framework on 100 queries with geographic truth in two world regions (each of 10, 000km2) in the USA.

Modern sensors have a range of modalities including SAR, EO, and IR. Registration of multimodal imagery from such sensors is becoming an increasingly common pre-processing step for various image exploitation activities such as image fusion for ATR. Over the past decades, several approaches to multisensor image registration have been developed. However, performance of these image registration algorithms is highly dependent on scene content and sensor operating conditions, with no single algorithm working well across the entire operating conditions space. To address this problem, in this paper we present an approach for dynamic selection of an appropriate registration algorithm, tuned to the scene content and feature manifestation of the imagery under consideration. We consider feature-based registration using Harris corners, Canny edge detection, and CFAR features, as well as pixel-based registration using cross-correlation and mutual information. We develop an approach for selecting the optimal combination of algorithms to use in the dynamic selection algorithm. We define a performance measure which balances contributions from convergence redundancy and convergence coverage components calculated over sample imagery, and optimize the measure to define an optimal algorithm set. We present numerical results demonstrating the improvement in registration performance through use of the dynamic algorithm selection approach over results generated through use of a fixed registration algorithm approach. The results provide registration convergence probabilities for geo-registering test SAR imagery against associated EO reference imagery. We present convergence results for various match score normalizations used in the dynamic selection algorithm.

Spatial domain log-polar approaches have demonstrated success for video image registration. However, the log-polar representation is sensitive to origin location. This drawback often necessitates performing a parameter sweep over log-polar origin location, which can be time-consuming. In this paper we present an alternative approach that is appropriate for small to moderate scale, rotational, and skew misalignments but allows large translational offset. We use a form of robust phase correlation to estimate the gross translation, then perform a local search over log-polar origin to fine tune the translation. We sequentially estimate affine transform parameters by maximizing a measure of registration solution verity. We also investigate the effect of scale and rotational initial alignment errors on the robustness of the initial phase correlation to estimate gross translation. We present results using video imagery acquired from a real aerial video surveillance system.

Realtime multisensor image registration algorithms must be computationally efficient. Often, simplifying assumptions are made to reduce computational time. However, these simplifications usually trade registration convergence performance for reduced runtime. For non-realtime applications where computational resources are not severely limited, this tradeoff may be reversed to improve convergence performance at the expense of increased computational cost. To this end we introduce a smart iterative approach to minimize mis-registrations and thus optimize registration convergence probability. The approach involves performing a registration sweep over a smart sampling of parameters governing feature generation. This approach involves use of two components; a feature sensitivity measure (FSM) and a registration verification metric (VM). The FSM measures the effect of parameter values on feature set variability. This measure enables choice of a suitable parameter sampling density to use for performing iterative registration solution search. The VM provides feedback on the registration solution verity in the absence of ground truth and is used to identify a converged solution. First, we provide an overview of the registration framework used to generate convergence results. Next we introduce the FSM and present mathematical properties. We then describe the VM and present the iterative algorithm. We present numerical results illustrating FSM convergence with increasing parameter sampling density for Canny edge features in SAR imagery. We illustrate use of FSM convergence behavior to select a suitable parameter sampling density for use in the iterative algorithm. Finally, SAR-to-EO registration performance results are presented showing improved convergence probability.

Accurate geo-location of imagery produced from airborne imaging sensors is a prerequisite for precision targeting and navigation. However, the geo-location metadata often has significant errors which can degrade the performance of applications using the imagery. When reference imagery is available, image registration can be performed as part of a bundle-adjustment procedure to reduce metadata errors. Knowledge of the metadata error statistics can be used to set the registration transform hypothesis search space size. In setting the search space size, a compromise is often made between computational expediency and search space coverage. It therefore becomes necessary to detect cases in which the true registration solution falls outside of the initial search space. To this end, we develop a registration verification metric, for use in a multisensor image registration algorithm, which measures the verity of the registration solution. The verification metric value is used in a hypothesis testing problem to make a decision regarding the suitability of the search space size. Based on the hypothesis test outcome, we close the loop on the verification metric in an iterative algorithm. We expand the search space as necessary, and re-execute the registration algorithm using the expanded search space. We first provide an overview of the registration algorithm, and then describe the verification metric. We generate numerical results of the verification metric hypothesis testing problem in the form of Receiver Operating Characteristics curves illustrating the accuracy of the approach. We also discuss normalization of the metric across scene content.

Image registration is usually a required first processing step for such activities as surveillance, video tracking, change detection, and remote sensing. Often, different sensors are used for the collection of the test and reference imagery. The sensor phenomenology differences can present problems for automatic selection of registration algorithm parameters because of different cross-sensor feature manifestation. In previous work involving edge-based multisensor image registration, we applied a previously-developed automated approach to parameter selection, designed specifically for edge detection. In this work, we adapt and apply a dynamic feature selection algorithm (DFSA) that we recently developed for use in registration algorithm selection for registering images with varying scene content type. We adapt and apply the DFSA to the problem of selecting appropriate registration algorithm parameter values in an edge-based registration algorithm. The approach involves generating test-to-reference feature match scores over a sampling of the transform hypothesis space. The approach is scene-adaptive thereby requiring no a priori information on image scene content. Furthermore, in the DFSA we leverage prior match score calculation generated in a hierarchical grid search to reduce additional computational expense. We give a brief overview of the registration algorithmic framework, and present a description of the dynamic feature selection algorithm. Numerical results are presented for performing test SAR-to-reference EO image registration to show the registration convergence performance improvement resulting from use of the DFSA. Numerical results are generated over images exhibiting different scene content types. We also evaluate the effect of match score normalization on the registration convergence performance improvement.

This paper generalizes the previously developed automated edge-detection parameter selection algorithm of Yitzhaky and Peli. We generalize the approach to arbitrary multidimensional, continuous or discrete parameter spaces, and feature spaces. This generalization enables use of the parameter selection approach with more general image features, for use in feature-based multisensor image registration applications. We investigate the problem of selecting a suitable parameter space sampling density in the automated parameter selection algorithm. A real-valued sensitivity measure is developed which characterizes the effect of parameter space sampling on feature set variability. Closed-form solutions of the sensitivity measure for special feature set relationships are derived. We conduct an analysis of the convergence properties of the sensitivity measure as a function of increasing parameter space sampling density. For certain parameter space sampling sequence types, closed-form expressions for the sensitivity measure limit values are presented. We discuss an approach to parameter space sampling density selection which uses the sensitivity measure convergence behavior. We provide numerical results indicating the utility of the sensitivity measure for selecting suitable parameter values.

Accurate image registration is critical for applications such as precision targeting, geo-location, change-detection, surveillance, and remote sensing. However, the increasing volume of image data is exceeding the current capacity of human analysts to perform manual registration. This image data glut necessitates the development of automated approaches to image registration, including algorithm parameter value selection. Proper parameter value selection is crucial to the success of registration techniques. The appropriate algorithm parameters can be highly scene and sensor dependent. Therefore, robust algorithm parameter value selection approaches are a critical component of an end-to-end image registration algorithm. In previous work, we developed a general framework for multisensor image registration which includes feature-based registration approaches. In this work we examine the problem of automated parameter selection. We apply the automated parameter selection approach of Yitzhaky and Peli to select parameters for feature-based registration of multisensor image data. The approach consists of generating multiple feature-detected images by sweeping over parameter combinations and using these images to generate estimated ground truth. The feature-detected images are compared to the estimated ground truth images to generate ROC points associated with each parameter combination. We develop a strategy for selecting the optimal parameter set by choosing the parameter combination corresponding to the optimal ROC point. We present numerical results showing the effectiveness of the approach using registration of collected SAR data to reference EO data.