A deep learning algorithm for Gaussian noise removal from both grayscale and color images is developed. As opposed to most existing discriminative methods that train a specific model for each noise level, the proposed method can handle a wide range of noise levels using only two trained models, one for low noise levels and the other for high noise levels. In the proposed algorithm, the training process consists of three successive steps. In the first step, a classifier is trained to classify the noisy and clean images. In the second step, a denoiser network aims to remove the noise in the image features that are extracted by the trained classifier. Finally, a decoder is utilized to map back the denoised images features into images pixels. To evaluate the performance of the model, the Berkeley segmentation dataset of 68 images (BSDS68) and 12 widely used images are used, and the denoising performance for additive white Gaussian noise is compared with several state-of-the-art methods in terms of peak signal-to-noise ratio (PSNR) and visual quality. For grayscale image denoising of BSDS68, our method gives the highest PSNR on all noise levels (significant mean improvement of 0.99). For color image denoising of BSDS68, except for one low noise level, the proposed method gives the highest PSNR on all other noise levels (mean improvement of 0.3). (C) 2020 SPIE and IS&T

Abstract. The search for effective noise removal algorithms is still a real challenge in the field of image processing. An efficient image denoising method is proposed for images that are corrupted by salt-and-pepper noise. Salt-and-pepper noise takes either the minimum or maximum intensity, so the proposed method restores the image by processing the pixels whose values are either 0 or 255 (assuming an 8-bit/pixel image). For low levels of noise corruption (less than or equal to 50% noise density), the method employs the modified mean filter (MMF), while for heavy noise corruption, noisy pixels values are replaced by the weighted average of the MMF and the total variation of corrupted pixels, which is minimized using convex optimization. Two fuzzy systems are used to determine the weights for taking average. To evaluate the performance of the algorithm, several test images with different noise levels are restored, and the results are quantitatively measured by peak signal-to-noise ratio and mean absolute error. The results show that the proposed scheme gives considerable noise suppression up to a noise density of 90%, while almost completely maintaining edges and fine details of the original image.

Physical therapeutic exercise (PTE) is the planned process of performing bodily movements, postures, or physical activities to provide a patient with the ability to remediate or prevent impairments at a minimum. The efficacy of the PTE involves measuring accurately the range of motion (ROM) of joint functions and parameters that indicate the onset of fatigue, jerky motion, and muscle/joint resistance to the PTE. A physical therapist (PT) typically determines the efficacy of a PTE by measuring joint angles in clinical diagnosis to assess the ROM using the simple device Goniometer since motion capture systems are generally expensive, difficult to use, and currently not suited for real-time operations. The joint angle measurement using Goniometer suffers from low accuracy, low reliability and subjective. Furthermore, a patient when performing PTE by themselves at remote locations like their home or community centers cannot use a Goniometer to determine the efficacy. In this study, we present the approach of using an inexpensive, simple human motion capture system (HMCS) consisting of a single camera and a graphical processing unit (GPU) to perform the efficacy of the PTE in real-time. The approach involves the use of general purpose graphic processing unit (GPGPU) computer vision technique to track and record human motion and relate the tracked human motion to the prescribed physical therapy regimen in real-time. We have developed a tracking algorithm derived from the Klein’s algorithm known as the Modified Klein’s algorithm (MKA) capable of tracking human body parts while the original Klein’s algorithm was only capable of tracking objects with sharp edges. The MKA algorithm is further modified for parallel execution on a GPU to operate in real-time. Using the GPU, we are able to track multiple markers in a high definition (HD) frame of the HD video in 1.77 msecs achieving near real-time capability of ROM measurements. Furthermore, the error in the ROM measurements in comparison to Goniometer measurements is in the range of -2.4° to +1.4°, which is well within the joint measurement prescribed standards. The suitability of the HMCS for elbow PTE is also presented.

Scale Invariant Feature Transform (SIFT) algorithm is a widely used computer vision algorithm that detects and extracts local feature descriptors from images. SIFT is computationally intensive, making it infeasible for single threaded im-plementation to extract local feature descriptors for high-resolution images in real time. In this paper, an approach to parallelization of the SIFT algorithm is demonstrated using NVIDIA’s Graphics Processing Unit (GPU). The parallel-ization design for SIFT on GPUs is divided into two stages, a) Algorithm de-sign-generic design strategies which focuses on data and b) Implementation de-sign-architecture specific design strategies which focuses on optimally using GPU resources for maximum occupancy. Increasing memory latency hiding, eliminating branches and data blocking achieve a significant decrease in aver-age computational time. Furthermore, it is observed via Paraver tools that our approach to parallelization while optimizing for maximum occupancy allows GPU to execute memory bound SIFT algorithm at optimal levels.

A new, seven parameter class of micromanipulators is found with decoupled dynamics. The resonant frequencies and damping ratios are found to be simple functions of the parameters, making it possible to design manipulators to meet resonant frequency or damping ratio specifications. Methods for designing the dynamics so that they are amenable to control are derived, which leads to significant increase in the closed-loop performance and robustness. To illustrate the new theory, a manipulator currently used on the International Space Station (ISS) is redesigned to greatly enhance its fault tolerance and closed-loop performance. Even though both manipulators control the same payload over the same bandwidth with exactly the same struts, the H∞ controller for the new manipulator is five times less sensitive to worst case disturbances than the H∞ controller for the existing manipulator. Moreover, the decoupling facilitates the design of Nyquist stable controllers with nonlinear dynamic compensators which give the new decoupled manipulator 100 times higher performance at low frequencies. This greatly increased performance and robustness requires implementation of only the six compensators along the diagonal of the six degree-of-freedom system, versus implementation of all 36 compensators for the ISS manipulator. This performance improvement is achieved with no increase in system cost, mass, or power budget; it is exclusively the result of the new theory presented in this paper.

This paper develops a robust optimization-based method to design orbits on which the sensory perception of the desired physical quantities are maximized. It also demonstrates how to incorporate various constraints imposed by many spacecraft missions, such as collision avoidance, co-orbital configuration, altitude and frozen orbit constraints along with Sun-synchronous orbit constraints. The paper specifically investigates designing orbits for constrained visual sensor planning applications as its case study. For this purpose, the key elements to form an image in such vision systems are considered and effective factors are taken into account to define a metric for perception quality. The method employs a max-min model to ensure robustness against possible perturbations and model uncertainties. While fulfilling the mission requirements, the algorithm devises orbits on which a higher level collective observation quality for the desired sides of the targets is available. The simulation results confirm the effectiveness of the proposed method for several scenarios involving low and medium Earth orbits as well as a challenging space-based space surveillance program application.

Most of the prevailing optimization packages only accept a quadratic conic representation of a second order cone program. The algorithms that convert the Lorentz conic constraint to such a representation, dramatically increase the size of the original problem by adding new variables and constraints. This impairs the solver performance, particularly in the large-scale problems, where the memory availability is one of the main concerns. This paper proposes a novel conversion algorithm, to achieve a minimal representation as well as a reduced order scheme that substantially decreases the dimensions of the converted model while maintaining the properties of the original problem. The algorithm provides a convenient way to achieve a good compromise between the problem size and the level of approximation by a single parameter. The simulation results confirm the effectiveness of the conversion algorithm on some mixed integer second order cone program optimization problems arising in robust multi-agent multi-target sensor allocation applications. The conducted analyses indicate that while the desired robustness level is achieved, the problem size can be substantially reduced in exchange for negligible performance degradation.

A new, seven parameter class of micromanipulators is found with decoupled dynamics. The flexibility of the new class is demonstrated by designing a Stewart platform with fully equalized and decoupled dynamics (i.e., all resonant frequencies are equal). Although this has been attempted before by other researchers, it has not been previously accomplished. The ability to design the dynamics so they are amenable to control leads to significant increases in closed loop performance and robustness. The dynamically decoupled and equalized manipulator is compared to an asymmetric manipulator in closed loop. Even though both manipulators control the same payload over the same bandwidth with exactly the same struts, the H ∞ controller for the equalized manipulator is five times less sensitive to worst case disturbances than the H ∞ controller for the asymmetric manipulator. Moreover, the decoupling facilitates the design of Nyquist stable compensators which give the decoupled manipulator one hundred times higher performance at low frequencies. This greatly increased performance and robustness requires implementation of only the six compensators along the diagonal of the six degree-of-freedom system, versus implementation of all 36 compensators for the asymmetrical manipulator.

This paper develops a non-myopic planning scheme that robustly maximizes the quality of the acquired information in an uncertain multi-camera multi-target vision system. To devise a robust plan, the probabilistic uncertainties associated with the system states are propagated through a non-linear quality metric utilizing the Unscented Transform. The metric considers different contributing factors that affect the quality of the observations for Pan-Tilt-Zoom cameras, such as the resolving ability as a function of distance, occlusion and the observation quality of different sides of the targets. The robust planning algorithm is formulated as a Mixed Integer Second Order Cone Program which employs the propagated statistics of the perception qualities at different time samples. Exploiting the proposed formulation, the trade-off between robustness and performance can be controlled by the confidence value parameter. This adds the capability of reaching suitable compromises to maximize observation quality despite system uncertainties. Extensive simulations confirm the effectiveness of the proposed planning scheme for a typical multi-agent surveillance application.

This dissertation introduces new ways of harnessing the strength of multi-agent systems to enhance the ability of a special class of cooperative control systems. In such systems, some dynamically decoupled agents pursue to collaboratively accomplish a common objective in the presence of uncertainty. Concretely, the principal theoretical and analytical contributions of this study addresses devising high performance sensor management systems by developing motion planning, and sensor allocation techniques (task assignment). Motion planning methods determine agents' trajectories on which the perception quality of the desired physical quantities of the targets is optimized. To include wide range of applications, both parametric and non-parametric path planning scenarios are separately investigated in two different real-world scenarios. For parametric case, a robust optimization based method is developed to design orbits on which the sensory perception of the desired physical quantities are maximized. Various constraints imposed by many spacecraft missions such as collision avoidance, co-orbital configuration, altitude and frozen orbit constraints along with Sun-Synchronous orbit are incorporated. In non-parametric scenario, a ground-based multi-objective patrolling/inspection mission is considered. A sequential scheme which allows compromises between trajectory length and observation quality is proposed. A new way of iteratively modifying all the agent trajectories until it is possible to collectively obtain quality images of all sides of all objects is developed. For each scenario, an adequate perception model that embodies the effect of key contributing elements in the overall system performance is introduced. The next contributions concern the development of non-myopic (multi-step) planning schemes aimed at dynamic allocation of the agents to the targets to enhance overall perception performance by exploiting their synergistic operation. Since the corresponding multi-objective problem belongs to strongly NP-hard class of combinatorial optimization problems, unless P=NP, it cannot even be solved in pseudo-polynomial time (e.g., by dynamic programming). To deal with large-scale systems with an enormous number of agents and targets, ILP-Greedy, a hybrid polynomial time approximation scheme algorithm, is proposed. The ILP-Greedy approach combines convex and greedy optimization methods to achieve near-optimal solutions with an approximate running time of order O(n1.32). Furthermore, a stochastic non-myopic sensor allocation method is developed to specifically tackle uncertain multi-agent multi-target sensor management systems. The robust planning algorithm is formulated as a Mixed Integer Second Order Cone Program which employs the propagated statistics of the perception qualities at different time samples. Exploiting the proposed formulation, the trade-off between robustness and performance can be controlled by the confidence value parameter. This adds the capability of reaching suitable compromises to maximize observation quality despite system uncertainties. The simulation results confirm the effectiveness of the proposed methods for several scenarios such as space situational awareness applications as well as ground-based multi-camera multi-target sensor management systems.

This paper analytically finds a new, significantly more general class of locally decoupled 2p - UPS, parallel manipulators encompassing and expanding several earlier classes. Its closed form solution leads directly to simple analytic formulas for the system eigenvalues in terms of geometric design parameters. To illustrate the power of the theory, the new methods are used to re-design an actual manipulator currently in use on the International Space Station. Unlike the existing manipulator, the new design is dynamically decoupled and isotropic.

New methods are found for arranging force actuators around a rigid body so that the system has locally decoupled and optimal manipulation characteristics. The closed-form solution leads directly to simple analytic formulas for the singular values of the manipulator Jacobian in terms of geometric design parameters. This makes it possible to easily design the local kinematics so that they meet desired specifications. Explicit formulas for designing the wrench/twist capabilities, achieving isotropy, maximizing the volume of achievable motions, and maximizing the minimum singular values are derived. Applications include design of generalized Gough-Stewart platforms (GSPs) and other parallel machines. To illustrate the power of the theory, the new methods are used to redesign an actual manipulator currently in use on the International Space Station (ISS). Unlike the existing manipulator, the new design is kinematically decoupled, isotropic, and fault tolerant - all highly desirable properties, especially in aerospace applications.

In order to maintain space situational awareness, it is necessary to maintain surveillance of objects in Earth orbit. A system of space-based imaging sensors could make much more detailed inspections of the existing resident space objects (RSOs). However, in order to preserve bandwidth, it is desirable to send the groundstation only a subset of all images which are taken by the inspection system. This paper presents a change detection algorithm which can detect changes in the appearance of an RSO. A new inspection image is compared to a previously taken base image. In each image, the translation vector and rotation matrix between the camera and the RSO, or pose, is slightly different. Assuming that the points making up each image of the RSO are within a single plane, it is possible to generate a planar homography which is a linear mapping between the two images. The homography is used to estimate the rotation and translation between the camera coordinate systems. This knowledge can be used to warp the inspection image so that it appears as though it was taken from the same coordinate system as the base image. Finally, basic morphological image processing and image thresholding techniques are used to perform change detection. The algorithm was evaluated by applying it to raytraced inspection images exhibiting varying lighting and pose conditions. Simulation results show that the algorithm can reliably detect damage to the RSO or the rendezvous of a suspicious object.

This paper presents a planning algorithm suitable whenever n objects must be collectively characterized by m observers and their relative motions are known a priori. This situation arises in Space Situational Awareness (SSA) problems due to the fixed orbits of spacecraft, and also occurs in several other aerospace and manufacturing environments. The new algorithm is a synthesis of two standard methods used to solve combinatorial optimization problems arising from various large-scale constrained active sensor planning applications. The algorithm allows constituent techniques to operate in domains where they perform better. Both constituent methods, Integer Linear Programming (ILP) Relaxation and a Batch-Greedy algorithm, are elaborated in detail. A very powerful feature of the overall approach is that an upper bound on the gap between the found sub-optimal solution and the unknown optimal solution is available. The ILP-relaxation algorithm provides an optimal but physically unrealizable solution, so if realizable performance approaches that of the ILP-relaxation solution, then the sub-optimal solution is very nearly optimal. A visual inspection problem for SSA, which lies in the strongly NP-hard class, is considered and it has been shown that the mixed method yields very nearly optimal solutions in polynomial time. Simulation results confirm the effectiveness of the proposed planning method on different orbits, including Low Earth and geosynchronous orbits.

Satellites are subject to harsh lighting conditions which make visual inspection difficult. Automated systems which detect changes in the appearance of a satellite can generate false positives in the presence of intense shadows and specular reflections. This paper presents a new algorithm which can detect visual changes to a satellite in the presence of these lighting conditions. The position and orientation of the satellite with respect to the camera, or pose, is estimated using a new algorithm. Unlike many other pose estimation algorithms which attempt to reduce image reprojection error, this algorithm minimizes the sum of the weighted 3-dimensional error of the points in the image. Each inspection image is compared to many different views of the satellite, so that pose may be estimated regardless of which side of the satellite is facing the camera. The features in the image used to generate the pose estimate are chosen automatically using the scale-invariant feature transform. It is assumed that a good 3-dimensional model of the satellite was recorded prior to launch. Once the pose between the camera and the satellite have been estimated, the expected appearance of the satellite under the current lighting conditions is generated using a raytracing system and the 3-dimensional model. Finally, this estimate is compared with the image obtained from the camera. The ability of the algorithm to detect changes in the external appearance of satellites was evaluated using several test images exhibiting varying lighting and pose conditions. The test images included images containing shadows and bright specular reflections.

Abstract : The University of Wyoming has formed a robotics initiative consisting of three distinct parts. Biomimetic Vision Sensor, (AFRL-RX-TY-TR-2011-0096-01) develops a novel computer vision sensor based upon the biological vision system of the common housefly, Musca domestica. Lightweight, Low Power Robust Means of Removing Image Jitter (AFRL-RX-TY-TR-2011-0096-02) develops an optimal platform stabilization mechanism for motion detection and target tracking using recent advances in the area of Parallel Kinematic Machines (PKMs). Unification of Control and Sensing for More Advanced Situational Awareness, (AFRL-RX-TY-TR-2011-0096-03) develops a multi-purpose planning scheme that effectively solves patrolling and constrained sensor planning problems for a large-scale multi-agent system.

For any robotic system, fault tolerance is a desirable property. This paper uses a comparative approach to investigate fault tolerance and the associated problem of reduced manipulability of robots. It is shown that for a certain class of manipulators, the mean squared relative manipulability over all possible cases of a given number of actuator failures is always constant irrespective of the geometry of the manipulator. In this context, optimal fault tolerant manipulability is quantified. The theory is applied to a special class of parallel manipulators called orthogonal Gough-Stewart platforms (orthogonal GSPs or OGSPs). A class of symmetric OGSPs that inherently provide for optimal fault tolerant manipulability under a single failure is developed.

The position regulation problem of an eye-in-hand type of parallel robot based pointing systems (PRBPS) is considered in this paper. A fuzzy logic system is first designed to compensate for the uncertainties of the parallel robot and the uncertainty of the image Jacobian, then a hybrid controller (HC) including the image-based nonlinear controller and the adaptive supervisory fuzzy logic controller (ASFLC) is derived by using the Lyapunov direct method to realize the position regulation (PR). The stability of the closed-loop system in the Lyapunov sense is proven theoretically. The fuzzy scaling matrix is combined with the HC to improve the performance of the control system. The simulation results demonstrate that the PRBPS realizes PR with very good robustness to the parameter uncertainties, and the control input torques and settling time are reduce greatly in the case of large initial feature errors.