The determination of a crop's growth-stage is critical information for precision agriculture. Estimates of the growth-stage are used to guide irrigation and the application of agrochemicals. Of particular importance is the use of fertilizers, however, growth-stage estimates may also suggest further investigation of potential crop infections and infestations. Traditionally, the growth-stage is based upon a manual random sample of a very small number of plants that are then analyzed to produce an estimate for the entire crop (up to thousands of acres). In order to increase the sample size (and thus accuracy) and to enable precision agriculture to address non-uniform crop development across a field, we present an analysis methodology that facilitates the automated growth-stage analysis of dense point clouds that are derived from drone imagery. Our method utilizes a standard camera drone and does not use specialized sensors or geo-spatial tagging. We propose a multi-stage unsupervised method, which provides information about the individual plant locations in a field plot with a high probability. The method also produces a measure of individual plant heights, which along with their location are critical for later growth-stage estimation and necessary for robotic precision application. We confirm our method's efficacy with experimental results on corn fields in Minnesota.

The segmentation and tracking of visual patterns, particularly those patterns related to natural imagery, have sparked renewed interest in the computer vision and image processing communities. Applications in robotics, automated systems, geographical information systems, etc. require efficient and accurate methods for processing visual data. Prior work in textural analysis has led to systems with promising accuracy, but poor efficiency. Work on fractal snakes provided both accuracy and efficiency, but at the loss of orientation with respect to the texture. In many applications, resolving orientation is an important piece of information. We have built upon our work in fractal snakes to expand our snake models from a purely isotropic measure of surface roughness to an orientation sensitive model: the Anisotropic Fractal Snake.

Studies have shown that visually-impaired people, particularly those with recent impairment, suffer from reduced social interaction. Technological innovations for these people have overlooked social interaction in favor of environmental mapping and mobility. Our work develops an Enhanced Social Awareness network. This network assists visually-impaired people in identifying friends and provides them with the cues necessary to initiate social contact. We experimentally verify our work and discuss the future implications and needs of such networks.

The next generation of Wireless Sensor Networks (WSNs) will need to be sustainable in the field. Sustainability presents a number of management difficulties. Among these challenges is field programmability. Future WSNs will consist of nodes with different architectures, capabilities, and operating systems. Current solutions for reprogramming WSNs either require homogenous networks or substantial overhead. Our node management layer, Trigger, can wirelessly reconfigure heterogeneous nodes when only a few parameters need to be changed. Our method has a manageable processor overhead while significantly decreasing network traffic. Trigger provides field-programmability to the WSN. This reconfiguration can simply rectify node-management tasks like sensor drift in a node. Furthermore, Trigger contains an abstraction of node context. This means Trigger can make local decisions about policy (like power management). Trigger is an easy, lightweight method to change program and node parameters and to manage node policy.

In this work we present a moving target segmentation technique and apply it to a vision-based robot following problem. The capability to do autonomous multi-robot following is useful for many robot-team applications; however, the problem becomes very challenging when the robots can carry only a small camera or when they exhibit unpredictable motion. The ability to segment a moving target while the camera is also in motion is critical to the solution of this problem and is the focus of our work. Our contributions include: (i) Matching targets using feature-based covariance matrices; (ii) Enhancing matching performance by using features based upon the Fourier transform; and (Hi) Initializing a target model for cases without a known target model. We compare the proposed method with the scale-invariant feature transform and existing covariance matching methods. We then validate our proposed segmentation method through real-robot experiments.

The natural environments that robotic applications often encounter can present difficult problems for image-based task execution. Prior efforts have used both grayscale and color as statistical appearance descriptors in these applications. In the case of natural environments, the statistical measures of luminosity and chromaticity are often ineffective due to relatively constant shades and colors of soil, flora, and fauna. Texture can provide an alternative/additional appearance descriptor in many of these environments; however the common approaches to textural segmentation are computationally intensive and cannot be used for real-time robotic visual servoing. We present a technique for textural segmentation and tracking that can discriminate between natural textures that are otherwise similar in color and brightness. The technique builds upon earlier work in fractal imaging and in statistical deformable models (a.k.a. snakes) to provide a simple and efficient method for extracting target shape and location from an initial textural example. We then give results from the application of this technique on standard texture test patterns. We then demonstrate the effectiveness of the method on natural imagery. Finally, we show how the technique can be applied to challenging robotic applications.

The popular Expectation Maximization technique suffers a major drawback when used to approximate a density function using a mixture of Gaussian components; that is the number of components has to be a priori specified. Also, Expectation Maximization by itself cannot estimate time-varying density functions. In this paper, a novel stochastic technique is introduced to overcome these two limitations. Kernel density estimation is used to obtain a discrete estimate of the true density of the given data. A Stochastic Learning Automaton is then used to select the number of mixture components that minimizes the distance between the density function estimated using the Expectation Maximization and discrete estimate of the density. The validity of the proposed approach is verified using synthetic and real univariate and bivariate observation data.

This paper introduces a new approach to active deformable models. The paper starts with a brief review of the relevant approaches. We then use the Expectation Maximization and Parzen Window techniques to develop a more robust formulation for pressure snakes. The generic nature of the proposed model allows it to perform well on complex colored images. We show, by several examples, the enhanced performance of the snake under the new formulation, compared to the performance of other known snake formulations.

Active contour models, more commonly known as snakes, have received considerable attention for more than a decade since their introduction by Kass et al. Snakes are energy minimizing contours. The energy of the snake depends upon its shape and location within the image. These models segment and/or track target areas in the images, that have certain characteristics. Active contour Models have been used in a wide range of applications. In vision-guided robotics, snakes have been used for object tracking, object grasping and object disambiguation. They have also been used for tumor segmentation in medical imaging applications. In Human-Computer Interaction (HCI), active contours have been used for non-intrusive eye tracking. Snakes attracted much of this attention because of several characteristics. They can segment objects with a reasonable computational cost, compared to other techniques. They also give a piecewise linear description of the contour of the object with no additional processing. On the other hand, classical snakes suffered from two major problems. First, active contours tended to fail in images with weak gradient fields. Also, classical snakes were limited to segmenting simple colored objects. Several formulations have been proposed trying to solve these problems. Most of the proposed formulations try to do this by adding more energy terms to the snake in order to control its evolution. Even though these approaches enhance the segmentation accuracy, they do so by increasing the computational complexity of the snake. In this dissertation, a new active contour formulation is presented. The new formulation alleviates the need for strong gradient field, while providing the low computational cost. The proposed formulation is based on estimating the probability density functions (PDF) of the target and the background. The PDFs are estimated using either Expectation Maximization (EM) or kernel estimators and then Bayesian decision theory is employed to drive the snake. Experimental results show that the proposed approach can be effectively used for both target tracking and target segmentation. A fuzzy-sets approach to active contours is also presented, along with experimental results, to show how to integrate other classification mechanisms into active contours to form a framework for object segmentation and tracking. Also, as a by-product of this research, and to bridge the gap between parametric and nonparametric density estimation, a Stochastic Learning Automata-based (SLA) approach for a nonparametric Expectation Maximization is presented.

Developing a robust, non-intrusive, inexpensive eye tracker is quite a challenge in the computer vision field. This paper introduces a new eye tracking approach based on an enhanced formulation of the statistical pressure snakes. Using a head-mounted camera, to eliminate the effect of head movements, we can detect and track the eye pupil robustly. Experimental results on video sequences for normal and blinking eyes are presented with high accuracy.

This papers introduces a new approach to active deformable models. We use elements of pattern recognition theory to develop a more robust formulation for pressure snakes. The generic nature of the proposed model allows it to perform well on complex colored images. We show, by several examples, the enhanced performance of the snake under the new formulation, compared to the performance of other known snake formulations.

Grasping is one of the most fundamental and challenging tasks in robotics. Applications range from space missions (e.g., collection of rock samples) to industrial automation. In this work, we use a camera mounted on the end-effector of a manipulator to grasp an unknown object in the workspace. A novel deformable contour model is used to determine plausible grasp axes of the target object. Potential grasp point pairs are generated, ranked based upon measurements taken from the contour, and a vision-guided grasp of the object using the highest ranked grasp point pair is executed. Several experimental results are presented

Object grasping is one of the basic functions required for many manipulator tasks. In particular, the grasping of unknown objects is often a desired functionality in manipulator system applications ranging from space exploration to factory automation. Due to the amount of object and environment data typically required to execute an unknown object grasp, computer vision is the sensing modality of choice. This paper presents a method for the automatic determination of plausible grasp points on unknown objects using an eye-in-hand robotic system and active deformable contour models. The system finds potential grasp point pairs, ranks all the possible pairs based upon measurements taken from the contour, and executes a vision-guided grasp of the object using the highest ranked grasp point. The paper also presents initial experimental results.

Many research efforts have turned to sensing, and in particular computer vision, to create more flexible robotic systems. Computer vision is often required to provide data for the grasping of a target. Using a vision system for grasping of static or moving objects presents several issues with respect to sensing, control, and system configuration. This paper presents some of these issues in concept with the options available to the researcher and the trade‐offs to be expected when integrating a vision system with a robotic system for the purpose of grasping objects. The paper includes a description of our experimental system and contains experimental results from a particular configuration that characterize the type and frequency of errors encountered while performing various vision‐guided grasping tasks. These error classes and their frequency of occurrence lend insight into the problems encountered during visual grasping and into the possible solution of these problems.

Robotic systems require the use of sensing to enable flexible operation in uncalibrated or partially calibrated environments. Recent work combining robotics with vision has emphasized an active vision paradigm where the system changes the pose of the camera to improve environmental knowledge or to establish and preserve a desired relationship between the robot and objects in the environment. Much of this work has concentrated upon the active observation of objects by the robotic agent. We address the problem of robotic visual grasping (eye-in-hand configuration) of static and moving rigid targets. The objective is to move the image projections of certain feature points of the target to effect a vision-guided reach and grasp. An adaptive control algorithm for repositioning a camera compensates for the servoing errors and the computational delays that are introduced by the vision algorithms. Stability issues along with issues concerning the minimum number of required feature points are discussed. Experimental results are presented to verify the validity and the efficacy of the proposed control algorithms. We then address an adaptation to the control paradigm that focuses upon the autonomous grasping of a static or moving object in the manipulator’s workspace. Our work extends the capabilities of an eye-in-hand system beyond those as a ‘pointer’ or a ‘camera orienter’ to provide the flexibility required to robustly interact with the environment in the presence of uncertainty. The proposed work is experimentally verified using the Minnesota Robotic Visual Tracker (MRVT) [7] to automatically select object features, to derive estimates of unknown environmental parameters, and to supply a control vector based upon these estimates to guide the manipulator in the grasping of a static or moving object.

This paper presents techniques for visual sensing in uncalibrated environments for intelligent vehicle-highway system applications. In particular, it describes a robust scheme for detection of transportation objects, focusing on the computational issues that occur when developing real-time IVHS applications. It then highlights a feature window selection scheme that automatically determines which features are worthwhile for use in visual tracking. Next, it discusses tracking the visual motion of a vehicle or a pedestrian. The paper concludes with a presentation of results from feasibility experiments for the selected IVHS applications.

The system proposed in this paper uses active deformable models to track pedestrians moving in dynamic real-world scenes. First, figure pixels are separated from a fixed or slowly evolving ground image. Then, an initial segmentation process identifies interesting pixel blobs for tracking. The output of the segmentation process is used to choose the starting position of the control points of the active deformable model. Once tracking has begun, the control points are updated at frame rates by minimizing an energy function involving the relative position of model points, image data, and the characteristics of figure pixels.

Flexible operation of a robotic agent in an uncalibrated environment requires the ability to recover unknown or partially known parameters of the workspace through sensing. Of the sensors available to a robotic agent, visual sensors provide information that is richer and more complete than other sensors. In this paper we present robust techniques for the derivation of depth from feature points on a target's surface and for the accurate and high-speed tracking of moving targets. We use these techniques in a system that operates with little or no a priori knowledge of the object-related parameters present in the environment. The system is designed under the controlled active vision framework and robustly determines parameters such as velocity for tracking moving objects and depth maps of objects with unknown depths and surface structure. Such determination of intrinsic environmental parameters is essential for performing higher level tasks such as inspection, exploration, tracking grasping, and collision-free motion planning. For both applications, we use the Minnesota Robotic Visual Tracker (a single visual sensor mounted on the end-effector of a robotic manipulator combined with a real-time vision system) to automatically select feature points on surfaces, to derive an estimate of the environmental parameter in question, and to apply a control vector based upon these estimates to guide the manipulator. The paper concludes with applications of these techniques to transportation problems such as vehicle tracking.<>

Most early research in robotic visual tracking, especially prior to 1990, separated the vision processing and robot control aspects of the system. Recent attempts to solve the problem close the control loop by incorporating the output of the vision processing as an input to the control subsystem. The Controlled Active Vision framework describes one such approach wherein dynamic target, camera, and environmen- tal factors are incorporated via adaptive controllers that uti- lize the Sum-of-Squared Dif ferences (SSD) optical flow measurements as an input to the control loop. This paper describes recent work at the University of Minnesota' s Artifi- cial Intelligence, Robotics, and Vision Laboratory in devel- oping the Minnesota Robotic Visual Tracker (MR VT), a Controlled Active Vision robotic testbed. In addition, enhancements to the basic SSD algorithm ar e presented that produce order-of-magnitude improvements over previously reported results.