In dieser Arbeit werden Verfahren der Bildverarbeitung und die Fahigkeit humanoider Roboter, mit ihrer Umgebung physisch zu interagieren, in engem Zusammenspiel eingesetzt, um unbekannte Objekte zu identifizieren, sie vom Hintergrund und anderen Objekten zu trennen, und letztendlich zu greifen. Im Verlauf dieser interaktiven Exploration werden auserdem Eigenschaften des Objektes wie etwa sein Aussehen und seine Form ermittelt.

Autonomous grasping and manipulation of unknown objects is a central skill for humanoid robots. This is particularly challenging, as shape information needs to be obtained from sensory data which is often noisy and incomplete. However, object shape information is usually a key prerequisite for grasp and manipulation planning and thus needs to be estimated even if the available sensor data is limited. We propose a method for implicit surface modeling based on sparse contact information, as it arises e.g. from haptic exploration. Surfaces are locally defined using the contact points and their normals, and the object shape is extrapolated by integrating this partial information. For each contact contributing to the estimation, the local convexity or concavity is determined depending on its neighbors and their respective normals. Taking into account contact positions, normals and local convexities or concavities, the Implicit Shape Potential of the overall surface is generated. In contrast to popular methods based on Gaussian Processes, this representation allows for local details like edges and corners, without losing the ability to interpolate in the case of noise. In addition, it provides information to guide iterative exploration algorithms. The proposed method is evaluated on a set of various 3D shapes that possess flat and curved surface regions as well as convex and concave edges.

Autonomous grasping and manipulation of unknown objects is a central skill for humanoid robots. This is particularly challenging, as shape information needs to be obtained from sensory data which is often noisy and incomplete. However, object shape information is usually a key prerequisite for grasp and manipulation planning and thus needs to be estimated even if the available sensor data is limited. We propose a method for implicit surface modeling based on sparse contact information, as it arises e.g. from haptic exploration. Surfaces are locally defined using the contact points and their normals, and the object shape is extrapolated by integrating this partial information. For each contact contributing to the estimation, the local convexity or concavity is determined depending on its neighbors and their respective normals. Taking into account contact positions, normals and local convexities or concavities, the Implicit Shape Potential of the overall surface is generated. In contrast to popular methods based on Gaussian Processes, this representation allows for local details like edges and corners, without losing the ability to interpolate in the case of noise. In addition, it provides information to guide iterative exploration algorithms. The proposed method is evaluated on a set of various 3D shapes that possess flat and curved surface regions as well as convex and concave edges.

We present an approach for determining suitable locations for human-robot interaction tasks. Therefore, we introduce the task specific Interaction Workspace as a representation of the workspace that can be accessed by both agents, i.e. the robot and the human. We show how the Interaction Workspace can be efficiently determined for a specific situation by making use of precomputed workspace representations of robot and human. By considering several quality measures related to dexterity and comfort, the Interaction Workspace provides valuable information about potential targets for human robot interaction (e.g. for object handover tasks). We evaluate the online performance of building appropriate data structures and show how the approach can be applied in a realistic hand-over use case with the humanoid robot ARMAR-III.

Object shape information is essential for robot manipulation tasks, in particular for grasp planning and collision-free motion planning. But in general a complete object model is not available, in particular when dealing with unknown objects. We propose a method for completing shapes that are only partially known, which is a common situation when a robot perceives a new object only from one direction. Our approach is based on the assumption that most objects used in service robotic setups have symmetries. We determine and rate symmetry plane candidates to estimate the hidden parts of the object. By finding possible supporting planes based on its immediate neighborhood, the search space for symmetry planes is restricted, and the bottom part of the object is added. Gaps along the sides in the direction of the view axis are closed by linear interpolation. We evaluate our approach with real-world experiments using the YCB object and model set [1].

This study describes the transfer of object grasping skills between two different humanoid robots with different software frameworks. We realize such a knowledge and skill transfer between the humanoid robots iCub and ARMAR-III. These two robots have different kinematics and are programmed using different middlewares, YARP and ArmarX. We developed a bridge system to allow for the execution of grasping skills of ARMAR-III on iCub. As the embodiment differs, the known feasible grasps for the one robot are not always feasible for the other robot. We propose a reactive correction behavior to detect failure of a grasp during its execution, to correct it until it is successful, and thus adapt the known grasp definition to the new embodiment.

We present an approach for autonomous interactive object segmentation by a humanoid robot. The visual segmentation of unknown objects in a complex scene is an important prerequisite for e.g. object learning or grasping, but extremely difficult to achieve through passive observation only. Our approach uses the manipulative capabilities of humanoid robots to induce motion on the object and thus integrates the robots manipulation and sensing capabilities to segment previously unknown objects. We show that this is possible without any human guidance or pre-programmed knowledge, and that the resulting motion allows for reliable and complete segmentation of new objects in an unknown and cluttered environment. We extend our previous work, which was restricted to textured objects, by devising new methods for the generation of object hypotheses and the estimation of their motion after being pushed by the robot. These methods are mainly based on the analysis of motion of color annotated 3D points obtained from stereo vision, and allow the segmentation of textured as well as non-textured rigid objects. In order to evaluate the quality of the obtained segmentations, they are used to train a simple object recognizer. The approach has been implemented and tested on the humanoid robot ARMAR-III, and the experimental results confirm its applicability on a wide variety of objects even in highly cluttered scenes.

We present an approach for visually detecting collisions between a robot's hand and an object during grasping. This allows to detect unintended premature collisions between parts of the hand and the object which might lead to failure of the grasp if they went unnoticed. Our approach is based on visually perceiving that the object starts to move, and is thus a good complement for force-based contact detection which fails e.g. in the case of grasping light objects that don't resist the applied force but are just pushed away. Our visual collision detection approach tracks the hand in the robot's camera images and analyzes the optical flow in its vicinity. When a collision is perceived, the most probable part of the hand to have caused it is estimated, and a corrective motion is executed. We evaluate the detection together with different reaction strategies on the humanoid robot ARMAR-III. The results show that the detection of failures during grasp execution and their correction allow the robot to successfully finish the grasp attempts in almost all of the cases in which it would otherwise have failed.

Humanoid robotics research has made significant progress and will continue to play central role in robotics research and many applications of the 21st cen-tury. In our research we investigate the engineering of humanoid robots able to act in the real world, learn from human observation, interact and collaborate with humans. In this paper we present recent advances in humanoid grasping and manipulation. We describe the grasping and manipulation system implemented on the humanoid robots ARMAR-IIIa and ARMAR-IIIb [1] to endow these robots with the capability of grasping and manipulating known and unknown objects in real world tasks. Further, we present a new grasping representation in the task space which can be learned from human grasping demonstration and adapted for execution on the robot while taking the grasping task and object specific constraints into consideration.

Humans can effortlessly perceive an object they encounter for the first time in a possibly cluttered scene and memorize its appearance for later recognition. Such performance is still difficult to achieve with artificial vision systems because it is not clear how to define the concept of objectness in its full generality. In this paper we propose a paradigm that integrates the robot’s manipulation and sensing capabilities to detect a new, previously unknown object and learn its visual appearance. By making use of the robot’s manipulation capabilities and force sensing, we introduce additional information that can be utilized to reliably separate unknown objects from the background. Once an object has been identified, the robot can continuously manipulate it to accumulate more information about it and learn its complete visual appearance. We demonstrate the feasibility of the proposed approach by applying it to the problem of autonomous learning of visual representations for viewpoint-independent object recognition on a humanoid robot.

A major strength of humanoid robotics platforms consists in their potential to perform a wide range of manipulation tasks in human-centered environments thanks to their anthropomorphic design. Further, they offer active head-eye systems which allow to extend the observable workspace by employing active gaze control. In this work, we address the question where to look during manipulation tasks while exploiting these two key capabilities of humanoid robots. We present a solution to the gaze selection problem, which takes into account constraints derived from manipulation tasks. Thereby, three different subproblems are addressed: the representation of the acquired visual input, the calculation of saliency based on this representation, and the selection of the most suitable gaze direction. As representation of the visual input, a probabilistic environmental model is discussed, which allows to take into account the dynamic nature of manipulation tasks. At the core of the gaze selection mechanism, a novel saliency measure is proposed that includes accuracy requirements from the manipulation task in the saliency calculation. Finally, an iterative procedure based on spherical graphs is developed in order to decide for the best gaze direction. The feasibility of the approach is experimentally evaluated in the context of bimanual manipulation tasks on the humanoid robot ARMAR-III.

Learning the visual appearance and physical properties of unknown objects is an important capability for humanoid robots that are supposed to be working in an open environment. We present an approach that enables a robot to discover new, unknown objects, segment them from the background and grasp them. This gives the robot full control over the object and allows its further multimodal exploration.

Learning about new objects that a robot sees for the first time is a difficult problem because it is not clear how to define the concept of object in general terms. In this paper we consider as objects those physical entities that are comprised of features which move consistently when the robot acts upon them. Among the possible actions that a robot could apply to a hypothetical object, pushing seems to be the most suitable one due to its relative simplicity and general applicability. We propose a methodology to generate and apply pushing actions to hypothetical objects. A probing push causes visual features to move, which enables the robot to either confirm or reject the initial hypothesis about existence of the object. Furthermore, the robot can discriminate the object from the background and accumulate visual features that are useful for training of state of the art statistical classifiers such as bag of features.

This paper reports on a new approach for segmentation and learning of new, unknown objects with a humanoid robot. No prior knowledge about the objects or the environment is needed. The only necessary assumptions are firstly, that the object has a (partly) smooth surface that contains some distinctive visual features and secondly, that the object moves as a rigid body. The robot uses both its visual and manipulative capabilities to segment and learn unknown objects in unknown environments. The segmentation algorithm is based on pushing hypothetical objects by the robot, which provides a sufficient amount of information to distinguish the object from the background. In the case of a successful segmentation, additional features are associated with the object over several pushing-and-verification iterations. The accumulated features are used to learn the appearance of the object from multiple viewing directions. We show that the learned model, in combination with the proposed segmentation process, allows robust object recognition in cluttered scenes.

The autonomous acquisition of object representations which allow recognition, localization and grasping of objects in the environment is a challenging task, which has shown to be difficult. In this paper, we present a systems for autonomous acquisition of visual object representations, which endows a humanoid robot with the ability to enrich its internal object representation and allows the realization of complex visual tasks. More precisely, we present techniques for segmentation and modeling of objects held in the five-fingered robot hand. Multiple object views are generated by rotating the held objects in the robot's field of view. The acquired object representations are evaluated in the context of visual search and object recognition tasks in cluttered environments. Experimental results show successful implementation of the complete cycle from object exploration to object recognition on a humanoid robot.