Robotic vision is widely used to provide feedback for the calibration and operation of autonomous robots. In many situations, the automation of the robot requires tracking one or more points of interest on the robot or in its surroundings. In this work, we developed and tested a camera-based vision system that can detect multiple points of interest and discriminate them into up to three groups. This is achieved by exploring the Red, Green, and Blue color layers in a single video feed. The experimental results show the proposed system's ability to accurately locate and distinguish three points of interest mounted on an industrial robot.

Despite years of research in the area of robotics, the vast majority of industrial robots are still used in “teach-repeat” mode. This requires that the workpiece be in exactly the same position and orientation every time. In many high-volume robotics applications, this is not a problem, since the parts are likely to be fixtured anyway. However, in small to medium lot applications, this can be a significant limitation. The motivation for this project was a corporation who wanted to explore the use of visual control of a manipulator to allow for automated teaching of robot tasks for parts that are run in small lot sizes.

This paper presents work done to enable a mobile manipulator to autonomously place, its tool with high accuracy and reliability, relative to a visually distinctive target. The work is novel in that the cameras are not calibrated a priori, rather, the system calibrates the cameras by moving the manipulator through the field of view, and the algorithm combines motion of the mobile base and the manipulator in order to achieve the task. Although not creating a globally improved camera calibration, the method provides very high precision in positioning a mobile manipulator relative to a visually selected target. The work was motivated by a desire to increase the precision and efficiency of the Mars exploration rovers (MER), allowing more science to be carried out in the same span of time. In addition to the algorithm, the paper describes a large number of experiments used to show the effectiveness of the method. For the experiments described in this paper, the starting distance of the rover relative to the point of interest ranged from about 2 to 8 m. Depending on the distance of traverse required, the rover had to use one to three sets of stereo cameras. Over a large range of distances, and many experiments, the system was shown to be robust and accurate. The paper further breaks down the sources of error and examines their importance based on a large number of experiments. © 2012 Wiley Periodicals, Inc.

The National Academies report “Engineering in K-12 Education: Understanding the Status and Improving the Prospects” gives recommendations describing the importance of a necessary, systematic change in the incorporation of engineering within the K-12 education system. Existing efforts to introduce engineering into K-12 typically consist of in-service activities for teachers and summer camp experiences and/or single day events in classrooms. The effectiveness of reaching out to teachers and students as individuals is debatable, but these methods are certainly not sustainable. Systematic change will require a new paradigm - teachers who have a fundamental understanding of engineering will provide the most effective, sustainable solution for the implementation of K-12 engineering education. Ohio Northern University (ONU) has developed and introduced a Bachelor of Science degree with a major in Engineering Education. This degree provides the graduate with a foundation in engineering, mathematics and education, qualifying the graduate for licensure as a secondary math teacher in the state of Ohio. The degree is similar to a General Engineering degree offered by some other Universities, expanding potential career opportunities to general engineering (sales, training, etc.) and unique opportunities in venues such as Science and Technology museums. This paper describes the fundamental structure of the degree program and the vision for those graduating with this major.

Manipulation systems for planetary exploration operate under severe limitations due to power and weight restrictions and extreme environmental conditions. Typically such systems employ carefully calibrated stereo cameras and carefully calibrated manipulators to achieve precision on the order of ten millimeters with respect to instrument placement activities. The environmental and functional restrictions under which these systems are used limit the operational accuracy of these approaches. This paper presents a novel approach to stereo-based manipulation designed to robustly achieve high precision levels despite the aforementioned limitations. The basic principle of the approach, known as Hybrid Image Plane/Stereo (HIPS) Manipulation, is the generation of camera models through direct visual sensing of the manipulator's end-effector. The HIPS method estimates and subsequently uses these models to position the manipulator at a target location specified in the image-planes of a stereo camera pair using stereo correlation and triangulation. In-situ estimation and adaptation of the manipulator/camera models in this method accounts for changes in the system configuration, thus ensuring consistent precision for the life of the mission. The end result is a increase in positioning precision by a factor of approximately two for a limited version of HIPS, and an order of magnitude increase in positioning precision for the full on-line version of HIPS.

Robotic Construction Crew (RCC) is a heterogeneous multi-robot system for autonomous acquisition, transport, and precision mating of components in construction tasks. RCC minimizes use of resources constrained by a space environment such as computation, power, communication, and sensing. A behavior-based architecture provides adaptability and robustness despite low computational requirements. RCC successfully performs several construction related tasks in an emulated outdoor environment despite high levels of uncertainty in motions and sensing. This paper provides quantitative results for formation keeping in component transport, precision instrument placement, and construction tasks.

This article has described in detail the Mars exploration rover's instrument positioning system and the use of this subsystem to carryout in situ operations of the Martian surface and subsurface. All told, the instrument deployment device (IDD) has served as an exceptional robotic mechanism for performing robust and reliable in situ science. The ability to carry out high precision mobile manipulation functions provided by the rover and the IDD has been critical to gaining a fundamental understanding of the water processes at work at both the Spirit and Opportunity landing sites. As such, the MER's IPS has paved the way for the use of future robotic devices that advance NASA's capabilities in autonomous manipulation, sample acquisition, and in situ science investigations

On January 24, 2004, the Mars Exploration Rover named Opportunity successfully landed in the region of Mars known as Meridiani Planum, a vast plain dotted with craters where orbiting spacecraft had detected the signatures of minerals believed to have formed in liquid water. The first pictures back from Opportunity revealed that the rover had landed in a crater roughly 20 meters in diameter - the only sizeable crater within hundreds of meters - which became known as Eagle Crater. And in the walls of this crater just meters away was the bedrock MER scientists had been hoping to find, which would ultimately prove that this region of Mars did indeed have a watery past. Opportunity explored Eagle Crater for almost two months, then drove more than 700 meters in one month to its next destination, the much larger Endurance Crater. After surveying the outside of Endurance Crater, Opportunity drove into the crater and meticulously studied it for six months. Then it went to examine the heat shield that had protected Opportunity during its descent through the Martian atmosphere. More than a year since landing, Opportunity is still going strong and is currently en route to Victoria Crater - more than six kilometers from Endurance Crater. Opportunity has driven more than four kilometers, examined more than eighty patches of rock and soil with instruments on the robotic arm, excavated four trenches for subsurface sampling, and sent back well over thirty thousand images of Mars - ranging from grand panoramas to up close microscopic views. This paper details the experience of driving Opportunity through this alien landscape from the point of view of the Rover Planners, the people who tell the rover where to drive and how to use its robotic arm.

Spirit is one of two rovers that landed on Mars in January 2004 as part of NASA's Mars Exploration Rover mission. As of July 2005, Spirit has traveled over 4.5 kilometers across the Martian surface while investigating rocks and soils, digging trenches to examine subsurface materials, and climbing hills to reach outcrops of bedrock. Originally designed to last 90 sols (Martian days), Spirit has survived over 500 sols of operation and continues to explore. During the mission, we achieved increases in efficiency, accuracy, and traverse capability through increasingly complex command sequences, growing experience, and updates to the on-board and ground-based software. Safe and precise mobility on slopes and in the presence of obstacles has been a primary factor in development of new software and techniques.

During Mars Exploration Rover (MER) surface operations, the scientific data gathered by the in situ instrument suite has been invaluable with respect to the discovery of a significant water history at Meridiani Planum and the hint of water processes at work in Gusev Crater. Specifically, the ability to perform precision manipulation from a mobile platform (i.e., mobile manipulation) has been a critical part of the successful operation of the Spirit and Opportunity rovers. As such, this paper describes the MER instrument positioning system that allows the in situ instruments to operate and collect their important science data using a robust, dexterous robotic arm combined with visual target selection and autonomous software functions.

This paper describes the operations of the 5 degree-of-freedom instrument deployment device (IDD), a dexterous robotic manipulator on the Mars Exploration Rovers, spirit and opportunity. The unprecedented flawless operations of the IDD enabled precise and reliable placement of at least 3 in situ instruments in sequential order on a designated target position on Martian rock/soil any time during the Martian diurnal cycle (day or night). These placements demonstrated a repeatability of /spl sim/1 mm in position and /spl sim/1 degree in orientation. This operations breakthrough is underappreciated, but it alone enabled the scientist to characterize a wide range of rocks and soils in a timely manner in the hunt for geological clues that revealed that the planet was once rich in water. In this paper we describe the IDD planning and command sequence generation process used to place and hold in situ instruments directly against rock and soil targets of interest within the IDD work volume.

NASA’s Vision for Space Exploration calls for an extended human presence in space and development of large-scale orbital structures. To reduce risk, it is essential to minimize astronaut exposure by limiting EVA and providing habitat infrastructure prior to arrival. Efficient assembly of space structures requires autonomous robotic teams with only high-level human supervision. Tasks will include component transport, precision component mating, structure inspection and analysis, and site surveying and clearing for surface structures. JPL is developing many of the required technologies for assembly and servicing to determine the challenges and required capabilities and to produce flight-relevant prototypes for maturing and testing these technologies in space-relevant environments.

The NASA Mars Exploration Rovers mission (MER) involves two robotic vehicles used to explore the geology of two surface regions on Mars in 2004. Prior to its launch, Earth-based field tests and operations campaigns were conducted in the deserts of southwestern USA to physically simulate the mission operations approach planned for MER. A prototype Mars rover called FIDO was used to conduct these field trials in complex geological settings with terrain analogous to the Martian Surface at the MER landing sites. This paper provides a high-level overview of the last major field operations test conducted in 2002 leading up to the MER mission. Objectives, approach, general results, and lessons learned are discussed.

Vision-aided flexible link robot positoning using the Camera Space Manipulation (CSM) method is developed. The primary motivation for this work is to use an autonomous vision-aided robotic system to pick-up and accurately move a flexible object that it encounters. The work consists of analytical and experimental investigation of the performance of CSM for a kinematic model of the PUMA manipulator with a flexible structure at the wrist which accounts for the gravitation. Trade-offs between camera view parameters and axial deflection model parameters were investigated. View parameter reestimation and maneuvering resulted a very accurate placement of the end-effector at the target.

Applications of vision-based remotely operated robotic systems range from planetary exploration to hazardous waste remediation. For space applications, where communication time lags are large, the target selection and robot positioning tasks may be performed sequentially, differing from conventional telerobotic maneuvers. For these point-and-move systems, the desired target must be defined in the image plane of the cameras either by an operator or through image processing software. Ambiguity of the target specification will naturally lead to end-effector positioning errors. In this paper, the target specification error covariance is shown to transform linearly to the end-effector positioning error. In addition, a methodology for optimal estimation of camera-view parameters of a vision-based robotic system based on target specification errors is presented. The proposed strategy is based on minimizing the end-effector error covariance matrix. Experimental results are presented demonstrating an increase in end-effector positioning, compared to traditional view parameter estimation by up to 32.