In this paper, we propose a real-time color-based target person learning and Bayesian person re-identification algorithm for target person tracking of a mobile robot in environments of varying illumination. We also propose an effective framework for simultaneous tracking of a target person and multiple non-target persons (STnT), by use of both color and lidar where the target person can be tracked without being affected by dynamic objects. To show the validity of the proposed tracking method, we collect datasets from challenging environments that feature varying illumination and various occlusions. In addition, we develop an evaluation system to measure the error between the ground truth pose and the pose predicted by our system. Our proposed tracking algorithm was successfully deployed on a service robot, Servinggo, at the international exhibition ROBOTWORLD Korea, and showed good tracking performances in very dynamic exhibition environments.

Efficiency and performance are significant challenges in applying Machine Learning (ML) to robotics, especially in energy-constrained real-world scenarios. In this context, Hyperdimensional Computing offers an energy-efficient alternative but has been underexplored in robotics. We introduce ReactHD, an HDC-based framework tailored for perception-action-based learning for sensorimotor controls of robot tasks. ReactHD employs hypervectors to encode sensory inputs and learn the suitable high-dimensional pattern for robot actions. It also integrates two HD-based lightweight symbolic learning paradigms: HDC-based supervised learning by demonstration (HDC-IL) and HD-Reinforcement Learning (HDC-RL). It renders robots to show precisely situated reactive behaviors in complex environments. Our empirical evaluations show that ReactHD achieves robust and accurate learning outcomes comparable to state-of-the-art deep learning while substantially improving the performance and energy consumption efficiency by 14.2× and 15.3×. To the best of our knowledge, ReactHD is the first HDC-based framework deployed in real-world settings.

In this letter, we present an autonomous navigation system for goal-driven exploration of unknown environments through deep reinforcement learning (DRL). Points of interest (POI) for possible navigation directions are obtained from the environment and an optimal way point is selected, based on the available data. Following the waypoints, the robot is guided towards the global goal and the local optimum problem of reactive navigation is mitigated. Then, a motion policy for local navigation is learned through a DRL framework in a simulation. We develop a navigation system where this learned policy is integrated into a motion planning stack as the local navigation layer to move the robot between waypoints towards a global goal. The fully autonomous navigation is performed without any prior knowledge while a map is recorded as the robot moves through the environment. Experiments show that the proposed method has an advantage over similar exploration methods, without reliance on a map or prior information in complex static as well as dynamic environments.

본 연구는 둘 이상의 물체가 서로 겹치거나 인접하여 로봇 파지 포인트를 파악하기 어려운 클러터 환경(cluttered environment)에서 역학 예측 신경망(Dynamics Prediction Network, DPN)을 이용하여 작업환경과 로봇 행동 사이의 역학 관계를 학습하고 파지 작업을 수행하는 강화학습 방법을 제안한다. 클러터 환경에서 파지 포인트를 찾을 수 없는 상황을 해결하기 위해 비파지 동작을 활용하며, 제안하는 역학 예측 신경망을 이용한 강화학습 방법은 파지뿐 아니라 비파지 동작 학습에서도 효과적임을 보여준다. 학습 환경을 추계(stochastic world)가 아닌 결정계(deterministic world)로 가정하여 전이 확률(state transition probability)을 1로 고정함으로써 DPN은 현재 상태(current state), 현재 행동(current action), 다음 상태(next state)의 일대일 대응관계를 학습한다. 학습 단계에서 추가적으로 진행되는 DPN으로 인한 학습 부담을 줄이기 위하여 매개변수 공유(parameter sharing)를 학습 모델에 적용하였다. 제안하는 방법은 매개변수 공유와 DPN으로 학습에서의 부담뿐 아니라 동작모델의 가중치 메모리 크기를 절반으로 줄이면서도 블록 테스트 환경에서 8%의 파지 성공률 향상을 보였다. 실세계 물체를 3D 스캔하여 구성한 새로운 테스트 환경 실험에서 23%의 파지 성공률 향상을 보여 물리적 상호작용 학습으로 일반화(generalization)가 잘 이루어졌음을 보였다.

For smart manufacturing, an automated robotic assembly system built upon an autoprogramming environment is necessary to reduce setup time and cost for robots that are engaged in frequent task reassignment. This article presents an approach to the autoprogramming of robotic assembly tasks with minimal human assistance. The approach integrates "robotic learning of assembly tasks from observation" and "robotic embodiment of learned assembly tasks in the form of skills." In the former, robots observe human assembly operations to learn a sequence of assembly tasks, which is formalized into a human assembly script. The latter transforms the human assembly script into a robot assembly script in which a sequence of robot-executable assembly tasks are defined based on action planning supported by workspace modeling and simulated retargeting. The assembly tasks, in the form of the robot assembly script, are then implemented via pretrained robot skills. These skills aim to enable robots to execute difficult tasks that involve inherent uncertainties and variations. We validate the proposed approach by building a prototype of the automated robotic assembly system for a power breaker and an electronic set-top box. The results verify that the proposed automated robotic assembly system is not only feasible but also viable, as it is associated with a dramatic reduction in the human effort required for automating robotic assembly.

In this paper, we present a goal-driven autonomous mapping and exploration system that combines reactive and planned robot navigation. First, a navigation policy is learned through a deep reinforcement learning (DRL) framework in a simulated environment. This policy guides an autonomous agent towards a goal while avoiding obstacles. We develop a navigation system where this learned policy is integrated into a motion planning stack as the local navigation layer to move the robot towards the intermediate goals. A global path planner is used to mitigate the local optimum problem and guide the robot towards the global goal. Possible intermediate goal locations are extracted from the environment and used as local goals according to the navigation system heuristics. The fully autonomous navigation is performed without any prior knowledge while mapping is performed as the robot moves through the environment. Experiments show the capability of the system navigating in previously unknown surroundings and arriving at the designated goal.

In this study, we present a method to grasp diverse unseen real-world objects using an off-policy actor-critic deep reinforcement learning (RL) with the help of a simulation and the use of as little real-world data as possible. Actor-critic deep RL is unstable and difficult to tune when a raw image is given as an input. Therefore, we use state representation learning (SRL) to make actor-critic RL feasible for visual grasping tasks. Meanwhile, to reduce visual reality gap between simulation and reality, we also employ a typical pixel-level domain adaptation that can map simulated images to realistic ones. In our method, as the SRL model is a common preprocessing module for simulated and real-world data, we perform SRL using real and adapted images. This pixel-level domain adaptation enables the robot to learn grasping skills in a real environment using small amounts of real-world data. However, the controller trained in the simulation should adapt to the real world efficiently. Hence, we propose a method combining a typical pixel-level domain adaptation and the proposed SRL model, where we perform SRL based on a feature-level domain adaptation. In evaluations of vision-based robotics grasping tasks, we show that the proposed method achieves a substantial improvement over a method that only employs a pixel-level or domain adaptation.

In recent years, vehicle networks require high bandwidth due to the increasing complexity of electronic control devices for vehicles due to the demand for advanced driving aids, infotainment and V2X communication. Therefore, Ethernet protocol was introduced and vehicle gateway system was newly introduced in vehicle system. The vehicle gateway system provides an interface for continuously connecting and exchanging vehicle data in different communication environments between the Controller Area Network (CAN) protocol and the Ethernet network protocol. There are two types of data exchange in the vehicle gateway. These are direct routing and indirect routing. Vehicle gateways are connected to external networks, which can lead to security vulnerabilities. So it needs security function to ensure message integrity. Recently, vehicle manufacturers have introduced gateway systems with security features that operate to verify the integrity of messages using cipher-based message authentication codes (CMAC). But applying security functions to a gateway system introduces delays in performing security functions. Therefore, in this paper, to design stable and efficient the vehicle gateway system with security function, we have tested and evaluated the latency time that can occur according to the routing methods which are direct routing and indirect routing.

In this paper, we propose a novel heuristics function for evaluating and selecting intermediate points in a goal-driven autonomous exploration and mapping system where navigation is performed by a learned neural network. The function calculation takes into consideration the training setting of the deep reinforcement learning-based network and combines it with distance information towards the global goal and the map information. The candidate with the minimum score is selected as the current intermediate goal. This allows the navigation system to be guided towards the global goal in an informed manner. Experiments in simulation and real-world settings show the benefit of the proposed approach over similar heuristic candidate point evaluation methods.

We developed a new framework to generate hand and finger grasping motions. The proposed framework provides online adaptation to the position and orientation of objects and can generate grasping motions even when the object shape differs from that used during motion capture. This is achieved by using a mesh model, which we call primitive object grasping (POG), to represent the object grasping motion. The POG model uses a mesh deformation algorithm that keeps the original shape of the mesh while adapting to varying constraints. These characteristics are beneficial for finger grasping motion synthesis that satisfies constraints for mimicking the motion capture sequence and the grasping points reflecting the shape of the object. We verify the adaptability of the proposed motion synthesizer according to its position/orientation and shape variations of different objects by using motion capture sequences for grasping primitive objects, namely, a sphere, a cylinder, and a box. In addition, a different grasp strategy called a three‐finger grasp is synthesized to validate the generality of the POG‐based synthesis framework.

For robotic grasping tasks with diverse target objects, some deep learning-based methods have achieved state-of-the-art results using direct visual input. In contrast, actor-critic deep reinforcement learning (RL) methods typically perform very poorly when applied to grasp diverse objects, especially when learning from raw images and sparse rewards. To render these RL techniques feasible for vision-based grasping tasks, we used state representation learning (SRL), in which we encode essential information for subsequent use in RL. However, typical representation learning procedures are unsuitable for extracting pertinent information for learning grasping skills owing to the high complexity of visual inputs for representation learning, in which a robot attempts to grasp a target object. We found that the proposed preprocessed input image is the key to capturing effectively a compact representation. This enables deep RL to learn robotic grasping skills from highly varied and diverse visual inputs. Further, we demonstrate the effectiveness of the proposed approach with varying levels of preprocessing in a realistic simulated environment. We also describe how the resulting model can be transferred to a real-world robot and also demonstrate a 68% success rate on real-world grasp attempts.

PURPOSE:Childhood obesity is a major global issue that causes a variety of health problems and high social costs. Many previous studies have investigated childhood obesity using cross-sectional data, but few longitudinal cohort studies have been performed, especially in the Korean population. METHODS:We analyzed the incidence and prevalence of obesity and overweight in a Korean prospective cohort study of children that were followed-up from age 7 to age 36. The study eventually recruited a total of 1216 participants, with 16 follow-up surveys over 30 years (1986-2017). RESULTS:The annual incidence of obesity showed a small peak (2.1%) at age 13 when the cohort entered middle school, but a rapid increase (6.4%) when participants reached the age of 20. The prevalence of obesity and overweight at age 8 was 0.8% and 0.9%, respectively, and increased rapidly from age 12 (obesity 2.2%, overweight 4.6%), reaching 9.5% and 15.9%, respectively, at age 20. The prevalence of obesity and overweight was consistently higher in girls than in boys during the childhood period, but this trend reversed in adulthood. CONCLUSIONS:Incidence and prevalence of obesity and overweight increased markedly after the final grades of elementary school in females, but after adolescence in males.

We present a novel feature matching algorithm that systematically utilizes the geometric properties of image features such as position, scale, and orientation, in addition to the conventional descriptor vectors. In challenging scenes, in which repetitive structures and large view changes are present, it is difficult to find correct correspondences using conventional approaches that only use descriptors, as the descriptor distances of correct matches may not be the least among the candidates. The feature matching problem is formulated as a Markov random field (MRF) that uses descriptor distances and relative geometric similarities together. Assuming that the layout of the nearby features does not considerably change, we propose the bidirectional transfer measure to gauge the geometric consistency between the pairs of feature correspondences. The unmatched features are explicitly modeled in the MRF to minimize their negative impact. Instead of solving the MRF on the entire features at once, we start with a small set of confident feature matches, and then progressively expand the MRF with the remaining candidate matches. The proposed progressive approach yields better feature matching performance and faster processing time. Experimental results show that the proposed algorithm provides better feature correspondences in many challenging scenes, i.e., more matches with higher inlier ratio and lower computational cost than those of the state-of-the-art algorithms. The source code of our implementation is open to the public.

Semantic segmentation has a wide array of applications such as scene understanding, autonomous driving, and robot manipulation tasks. While existing segmentation models have achieved good performance using bottom-up deep neural processing, this paper describes a novel deep learning architecture that integrates top-down and bottom-up processing. The resulting model achieves higher accuracy at a relatively low computational cost. In the proposed model, higher-level top-down information is transmitted to the lower layers through recurrent connections in an encoder and a decoder, and the recurrent connection weights are trained using backpropagation. Experiments on several benchmark datasets demonstrate that this use of top-down information improves the mean intersection over union by more than 3% compared with a state-of-the-art bottom-up only network using the CamVid, SUN-RGBD and PASCAL VOC 2012 benchmark datasets. Additionally, the proposed model is successfully applied to a dataset designed for robotic grasping tasks.

We propose a framework based on imitation learning and self-learning to enable robots to learn, improve, and generalize motor skills. The peg-in-hole task is important in manufacturing assembly work. Two motor skills for the peg-in-hole task are targeted: “hole search” and “peg insertion”. The robots learn initial motor skills from human demonstrations and then improve and/or generalize them through reinforcement learning (RL). An initial motor skill is represented as a concatenation of the parameters of a hidden Markov model (HMM) and a dynamic movement primitive (DMP) to classify input signals and generate motion trajectories. Reactions are classified as familiar or unfamiliar (i.e., modeled or not modeled), and initial motor skills are improved to solve familiar reactions and generalized to solve unfamiliar reactions. The proposed framework includes processes, algorithms, and reward functions that can be used for various motor skill types. To evaluate our framework, the motor skills were performed using an actual robotic arm and two reward functions for RL. To verify the learning and improving/generalizing processes, we successfully applied our framework to different shapes of pegs and holes. Moreover, the execution time steps and path optimization of RL were evaluated experimentally.

Energy bills are one of the most significant regular payments made by any household. The accurate estimation of the energy bills can help people make better decisions on their energy use. However, it is difficult for people to predict their monthly energy bills because of many influencing factors, such as weather and progressive rates. In this paper, we propose a deep learning-based prediction method of the energy bills for individual households, considering the weather. First, we proposed a gated recurrent unit-based model that can incorporate weather and date information and predict energy usage of electricity, water, and gas. Then, a weighted mean squared error is adopted for training of the proposed model for better accuracy. Finally, the predicted energy usage using the proposed model are converted to a monthly energy bill. The proposed approach is verified on the dataset consisting of energy usage data of 2,234 households and weather data of the Korea Meteorological Administration. The results show that our approach can accurately predict the energy bills of individual households with a small average error of 3,892 won.

In this paper, we present a supervised learning-based mixed-input sensor fusion neural network for autonomous navigation on a designed track referred to as Fusion Drive. The proposed method combines RGB image and LiDAR laser sensor data for guided navigation along the track and avoidance of learned as well as previously unobserved obstacles for a low-cost embedded navigation system. The proposed network combines separate CNN-based sensor processing into a fully combined network that learns throttle and steering angle labels end-to-end. The proposed network outputs navigational commands with similar learned behavior from the human demonstrations. Performed experiments with validation data-set and in real environment exhibit desired behavior. Recorded performance shows improvement over similar approaches.

This paper presents a deep reinforcement learning approach to learn robot navigation in continuous action space with a motion behavior based on human proxemics. We extended a deep deterministic policy gradient network to include convolutional layers for dealing with motion over multiple timesteps. A proxemics-based cost function for the robot to obtain the desired socially aware navigation behavior was developed and implemented in the learning stage, which respects the personal and intimate space of a human. The performed experiments in the simulated and real environments exhibited the desired behavior. Furthermore, the intrusions into the proxemics zones of a human were significantly reduced compared to similar learned robot navigation approachers.

In this paper, we propose a goal-oriented obstacle avoidance navigation system based on deep reinforcement learning that uses depth information in scenes, as well as goal position in polar coordinates as state inputs. The control signals for robot motion are output in a continuous action space. We devise a deep deterministic policy gradient network with the inclusion of depth-wise separable convolution layers to process the large amounts of sequential depth image information. The goal-oriented obstacle avoidance navigation is performed without prior knowledge of the environment or a map. We show that through the proposed deep reinforcement learning network, a goal-oriented collision avoidance model can be trained end-to-end without manual tuning or supervision by a human operator. We train our model in a simulation, and the resulting network is directly transferred to other environments. Experiments show the capability of the trained network to navigate safely around obstacles and arrive at the designated goal positions in the simulation, as well as in the real world. The proposed method exhibits higher reliability than the compared approaches when navigating around obstacles with complex shapes. The experiments show that the approach is capable of avoiding not only static, but also dynamic obstacles.