Abstract Humanoid robot has the potential to achieve various manipulation strategies such as lifting, pushing, pivoting and rolling. Deciding which strategies to apply for the object manipulation is difficult because the appropriate manipulation motion varies greatly depending on the object property. We propose the planning method of whole-body manipulation for various manipulati on strategies. In order to plan the manipulation in the global scope, we generate the graph representing all the possible object poses and operation contacts. By searching the feasible path in the graph, we plan the sequence of object states satisfying the kinematics and dynamics conditions. Finally, the robot postures for the whole-body manipulation are generated from the object states. The proposed method is efficient because the tractable object configuration is first planned in the global scope and then the complicated robot configuration is calculated. We show the effectiveness of the proposed method by planning the humanoid robot motion for various manipulation strategies automatically from the object and robot models. Graphical Abstract

In order to enable humanoids to pick a unstable object, it is necessary to track the position and pose of the object in real time by the image and point cloud from camera.But the conventional methods to track objects by using 3D Optical flow are able to track only the position of objects moving by translational motion.In this paper, we propose the real time object position and pose tracking method using segmentation of point cloud in addtion to using 3D optical flow. We applied this method to control the picking motion of humanoid robot and realize picking unstable objects.

To achieve tasks in unknown environments with high reliability, highly accurate localization during task execution is necessary for humanoid robots. In this paper, we discuss a localization system which can be applied to a humanoid robot when executing tasks in the real world. During such tasks, humanoid robots typically do not possess a referential to a constant horizontal plane which can in turn be used as part of fast and cost efficient localization methods. We solve this problem by first computing an improved odometry estimate through fusing visual odometry, feedforward commands from gait generator and orientation from inertia sensors. This estimate is used to generate a 3D point cloud from the accumulation of successive laser scans and such point cloud is then properly sliced to create a constant height horizontal virtual scan. Finally, this slice is used as an observation base and fed to a 2D SLAM method. The fusion process uses a velocity error model to achieve greater accuracy, which parameters are measured on the real robot. We evaluate our localization system in a real world task execution experiment using the JAXON robot and show how our system can be used as a practical solution for humanoid robots localization during complex tasks execution processes.

This paper presents the development of high-speed and high-power humanoid robot research platform JAXON. Researchers have studied humanoid robots widely and people expect humanoid robots to help various tasks such as housework, entertainment, and disaster-relief. Therefore it is important to develop humanoid research platform available in various fields. We considered the following as design requirements necessary to utilize humanoids for various uses. 1) Robots have humanlike body proportion to work in infrastructure matched to human body structure. 2) Robots have the same degree of physical performance as humans. 3) Robots have energy sources such as batteries and act without tethers. 4) Robots walk with two legs or four limbs and continue to work without fatal damage in unexpected rollover. JAXON satisfied these requirements. Then we demonstrates the performance of JAXON through the experiment of getting out of a vehicle, stepping over walls, squatting with heavy barbels, walking with four limbs, and operating on batteries. Further more, we assesses the performance of the strong armor and the shock absorbing structure through a backward over-turning accident.

It is necessary for a robot system to control accuracy and region-of-interest according to safety, specified time limit, and available computer resources. If enough duration for a task is provided, robot can use more accurate perception and planners producing higher rate of success. In the research and development of robot systems, these relationship between accuracy and computation time has been tuned carefully by hand. Appropriate sensors, algorithms and parameters are necessary to be chosen by system implementers. In this paper, a constitution method being able to control and evaluate perceptions and actions for robot systems is proposed towards implementing robot systems with automatic parameter controls of perceptions and planners corresponding to specified time limits and success rate.

Groping behavior based on contact sensors is necessary for manipulation in an unknown environment. For those situations, it is effective for a robot to accumulate contact information as an environment map, and to plan the motions for executing the safe trial motion. We first propose a method of updating the occupancy grid map of the manipulation region from the contact information by introducing the contact sensor model. Using this map, we propose a method of sampling-based motion planning that enables the execution of the safe trial motion based on the criteria of feasibility and safety. To verify the effectiveness, we show the experimentally obtained results, showing that a real robot plans and executes the manipulation with groping behavior in the occluded environment.

Whole-body holding manipulation is effective for carrying the handleless large object. In order to keep the object stability, the dexterous transition motion is necessary. From geometric and physical conditions of object manipulation, we propose the general method of generating the transition graph, which represents the object pose and grasp contact. By searching the path on the graph, the transition motion is planned automatically with considering the object motion and contact switching simultaneously. By generating and modifying the whole-body holding motion, the planned object motion is achieved stably. We show the effectiveness of the proposed method by the experiments, in which robot lifts up a large object with whole-body contact by the planned transition motion.

This paper presents a high level system architecture of the work done by Team NEDO-JSK for DRC (DARPA Robotics Challenge), which was held on 5th-6th, June, 2015, California, USA. DRC was a competition which included driving vehicle, manipulating objects and locomotion over rough terrain. Communication between robot and operators were limited in a similar way to teleoperating situation.The system introduced in this paper was strongly motivated by DRC competition, however system architecture is useful for generic teleoperating tasks. User interface and sensor visualization is the central element of our teleoperating system and operators' performance greatly depends on them. The goal of our system is to provide system to operate robot with less effort of operators and to be able to handle unexpected situation. The key features of the system are: 1) supervised system architecture based on simple command from operators. 2) low-level sensor visualization and high-level teleoperating user interface capable to deal with emergencies. 3) system integration over unreliable and narrow network.This paper summarizes our technical approach and implementation focusing on communication over narrow and unreliable network and user interface for operators. As evaluation of our system, this paper summarizes actual performance through DRC competition.

In recent years, humanoids have been expected to play an important part in disaster response due to safety concerns. For disaster response, humanoids should do tasks in unknown and unstructured environments possibly with limited communications. Firstly this paper presents a robot operating system in which complementary integration of autonomous and manual functions is achieved. In our system operator can change the level of automation depending on the situation: operator can modify the result of recognition in 3D Viewer, and can transfer from auto motion generating mode to manual control mode at any time with inheriting some motion parameters. Secondly for the purpose of overcoming communication-limit in disaster site, we propose the method of generating robot motion with little communication between operator and robot. Even when communication is limited, our System can convey necessary information to user by processing past data, always transferring small important data, and showing future motion plans.

This paper presents Team NEDO-JSK's approach to the development of novel humanoid platform for disaster response through participation to DARPA Robotics Challenge Finals. This development is a part of the project organized by New Energy and Industrial Technology Development Organization. Technology for this robot is based on the recent research of high-speed and high-torque motor driver with water-cooling system, RTM-ROS inter-operation for intelligent robotics, and generation of full-body fast dancing motion, due to the generic 10 year's research of HRP-2 as a platform humanoid robot. Development target is the robot support in a variety of unsafe human tasks teleoperated by humans in case of a disaster response, equipped with body structure capability for use of human devices and tools in human environment, performance for dynamic full-body actions covering human-sized speed and power, and basic function for intelligent and integrated robot platform system for performing various tasks independently. we also describes NEDO-JSK team's approach to design methodology for robot hardware and architecture of software system and user interface for DRC Finals as a test case of disaster response.

In unknown enviromnent like disaster site, humanoids have to be able to handle tasks flexibly. We first propose the robot operating system for handling unknown object assisted with our shape recognition method. Secondory, we propose reusing handling information that humanoids have done with integration of two different recognition system.

In the wake of the DARPA Robotics Challenge, the task for robots to drive vehicles has been expected to be a method for robots to transport themselves to the disaster site where it is hazardous for humans to approach. In the driving task, it is important for the robot to estimate the path of the vehicle and select an appropriate path for navigation through unknown obstacles, even under limited communication with an operator. It is also necessary for robots to suggest the estimated path of vehicle to an operator to deal with unforeseen circumstances. Therefore, we propose a recognition guided teleoperated driving system for robots to drive vehicles in disaster sites with estimated vehicle path based on steering angle and vehicle model. First, we show model based steering and pedaling strategy to achieve the target steering angle for desired path. Next, we propose a vehicle path estimation and a local planner that can suggest a traveling path according to the surroundings. We integrated them into a teleoperation system for bandwidth limited environments as recognition guidance. Finally, we show the effectiveness of our driving system by conducting field driving experiments with three different robots: JAXON, STARO and HRP-2.

This paper describes a practical method to construct real-time controllers to achieve locomotion and manipulation tasks with a humanoid robot. We propose a method to insert emergency stop functionality to each layer to avoid robot's falling down and joint overloads even if recognition and planning error exist. We explain implementation of multilayered real-time controllers on HRP2 robot and application to several manipulation and locomotion tasks. Finally, we evaluate emergency stop functionality in several manipulation tasks.

A daily assistant robot which performs tasks in living environment should be interrupted by human at any time in order to prevent failure of task execution and improve robot's behavior. This paper provides the model of a preemptive robot action management system which is integrated with an existing distributed robot control system. This model consists of three modules which is called when a human interrupts a robot action: a preemptive task management, dynamic action state management and motion modification.

System for using a humanoid robot is very complex. Evaluation platform through humanoid robot design to high level software application is required. In this paper, Gazebo (robot dynamics simulator), OpenRTM and ROS (robot middle ware) are adopted as a evaluation platform. Robot model conversion system and connecting Gazebo and OpenRTM/ROS as holding convertibility between a real robot and a simulator are provided as open source software. And, method for transparent integrating software is discussed.

Biped robots are expected to explore in the unknown environment which has undulate floors and a lot of obstacles on the floors. Such kind of the environment are difficult for wheeled mobile robots. In addition to planning and recognition functionality, interruption from operators to the systems are important to change behavior of navigation system. In order to achieve the tasks safely, as the system detects the unexpected situation, operators need to direct suitable goal and parameters to the planner. In this paper, we show a navigation system capable of interruption by operators. The communication between the system and operators are not continuous instruction but interruption. The keys of the system are: 1) system integration across different machines according to supervised autonomy framework. 2) fast vision based environment modeling and 3-D footstep planning to support operator to direct goals. 3) interruptible scheduling of footstep execution to cancel walking according to monitoring environment and operator's instructions. We implement the system on a real robot, HRP-2, and show effectiveness of interruptible system integration and 3-D footstep planning through experiments.

A computational method for optimal control problems of descriptor systems is focused on. Practical systems usually contain dynamic constraints and static constraints, and generally are expressed in a descriptor fashion. However, there is little research about the optimal control design with descriptor systems. In this paper, based on the so called second variation algorithms, we propose an iterative algorithm for numerical solutions in a descriptor fashion, using descriptor-type Riccati equations.

This paper describes a textureless object segmentation approach for autonomous service robots acting in human living environments. The proposed system allows a robot to effectively segment textureless objects in cluttered scenes by leveraging its manipulation capabilities. In our pipeline, the cluttered scenes are first statically segmented using state-of-theart classification algorithm and then the interactive segmentation is deployed in order to resolve this possibly ambiguous static segmentation. In the second step the RGBD (RGB + Depth) sparse features, estimated on the RGBD point cloud from the Kinect sensor, are extracted and tracked while motion is induced into a scene. Using the resulting feature poses, the features are then assigned to their corresponding objects by means of a graph-based clustering algorithm. In the final step, we reconstruct the dense models of the objects from the previously clustered sparse RGBD features. We evaluated the approach on a set of scenes which consist of various textureless flat (e.g. box-like) and round (e.g. cylinder-like) objects and the combinations thereof.

Many countries around the world face three major issues associated with their aging societies: a declining population, an increasing proportion of seniors, and an increasing number of single-person households. To explore assistive technologies that can help solve the problems faced by aging societies, we have tested several information and robot technologies. This paper introduces research on a home-assistant robot, which improves the ease and productivity of home activities. For people who work hard outside the home, the assistant robot performs chores in their home environment while they are away. A case study of a life-sized robot with a humanlike functional body performing daily chores is presented. An integrated software system incorporating modeling, recognition, and manipulation skills, as well as a motion generation approach based on the software system, is explained. Moreover, because housekeepers perform chores one after another in their daily environment, we also aim to develop a system for continuously performing a series of tasks by including failure detection and recovery.

Humanoid robots working in a household environment need 3D geometric shape models of objects for recognizing and managing them properly. In this paper, we make humanoid robots creating models by themselves with dual-arm re-grasping (Fig.1). When robots create models by themselves, they should know how and where they can grasp objects, how their hands occlude object surfaces, and when they have seen every surface on an object. In addition, to execute efficient observation with less failure, it is important to reduce the number of re-grasping. Of course when the shape of objects is unknown, it is difficult to get a sequence of grasp positions which fulfills these conditions. This determination problem of a sequence of grasp positions can be expressed through a graph search problem. To solve this graph, we propose a heuristic method for selecting the next grasp position. This proposed method can be used for creating object models when 3D shape information is updated on-line. To evaluate it, we compare the result of the re-grasping sequence from this method with the optimal sequence coming out of breadth first search which use 3D shape information. Also, we propose an observation system with dual-arm re-grasping considering the points when humanoid robots execute observation in the real world. Finally, we show the experiment results of construction of 3D shape models in the real world using the heuristic method and the observation system.