Learning diverse and flexible locomotion strategies in uncertain environments has been a longstanding challenge for quadruped robots. Although recent progress in domain randomization has partially tackled this difficulty by training policies on a wide range of potential factors, there is still a great need for improving efficiency. In this paper, we propose a novel framework for adaptive quadruped robot locomotion learning in uncertain environments. Our method is based on data-efficient reinforcement learning and learns simulation parameters iteratively. We also propose a novel Sampling-Interval-Adaptive Identification (SIAI) strategy that uses historical parameters to optimize sampling distribution and then improve identification accuracy. Final evaluations based on multiple robotic locomotion tasks showed superiority of our method over baselines.

As an intelligent device integrating a series of advanced technologies, mobile robots have been widely used in the field of defense and military affairs because of their high degree of autonomy and flexibility. They can independently track and attack dynamic targets. However, traditional tracking attack algorithms are sensitive to the changes of the external environment, and does not have mobility and expansibility, while deep reinforcement learning can adapt to different environments because of its good learning and exploration ability. In order to pursuit target accurately and robust, this paper proposes a solution based on deep reinforcement learning algorithm. In view of the low accuracy and low robustness of traditional dynamic target pursuit, this paper models the dynamic target tracking and attack problem of mobile robots as a Partially Observable Markov Decision Process (POMDP), and proposes a general-purpose end-to-end deep reinforcement learning framework based on dual agents to track and attack targets accurately in different scenarios. Aiming at the problem that it is difficult for mobile robots to accurately track targets and evade obstacles, this paper uses partial zero-sum game to improve the reward function to provide implicit guidance for attackers to pursue targets, and uses asynchronous advantage actor critic (A3C) algorithm to train models in parallel. Experiments in this paper show that the model can be transferred to different scenarios and has good generalization performance. Compared with the baseline method, the attacker’s time to successfully destroy the target is reduced by 44.7

Automated Guided Vehicle (AGV) has been widely used in automated warehouses and flexible manufacture systems for material delivery. As a flexible robot, AGV can finish automatic transportation of raw materials in different locations. The proper AGV scheduling strategy can effectively reduce the overall delivery time. To eliminate the large scheduling overhead from the centralized methods, we propose a multi-AGV distributed scheduling scheme in this paper. In particular, we design a Hierarchical Intrinsic Reward Mechanism (HIRM) for the multi-agent reinforcement learning to improve the convergence speed and the final policy level. Based on it, we propose the HIRM Bidirectionally-Coordinated Network (HIRM-BiCNet) based multi-AGV distributed scheduling scheme, to improve the scheduling success rate. The proposed scheme avoids the dependence on the global information and explicit communication. Experiment results demonstrate that our approach achieves impressive results at increase in scheduling success rate (30.75%) and decrease in scheduling time (16 time steps) compared to existing schemes.

Crowd-inspired intelligent manufacturing space (CrowdIM) aims to leverage the aggregated power of heterogeneous human–machine–things (HMT) agents for improving the efficiency of intelligent manufacturing. A significant scientific problem in CrowdIM is how to improve individual skills and crowd intelligence through cooperation, complementation, competition, and confrontation among HMT agents. The emergence mechanism of biological crowd intelligence provides an inspiration to address this challenge. This article explores the mapping mechanisms between natural crowd intelligence and CrowdIM, from the aspects, such as collective dynamics, self-adaptive mechanism, crowd intelligence optimization, graph structure mapping model, evolutionary game dynamics, multiagent learning, and so on. We further propose a general model of CrowdIM and expound it through a typical case study.

Automated Guided Vehicles (AGVs) have been widely used for material handling in flexible shop floors. Each product requires various raw materials to complete the assembly in production process. AGVs are used to realize the automatic handling of raw materials in different locations. Efficient AGVs task allocation strategy can reduce transportation costs and improve distribution efficiency. However, the traditional centralized approaches make high demands on the control center’s computing power and real-time capability. In this paper, we present decentralized solutions to achieve flexible and self-organized AGVs task allocation. In particular, we propose two improved multi-agent reinforcement learning algorithms, MAD-DPG-IPF (Information Potential Field) and BiCNet-IPF, to realize the coordination among AGVs adapting to different scenarios. To address the reward-sparsity issue, we propose a reward shaping strategy based on information potential field, which provides stepwise rewards and implicitly guides the AGVs to different material targets. We conduct experiments under different settings (3 AGVs and 6 AGVs), and the experiment results indicate that, compared with baseline methods, our work obtains up to 47% task response improvement and 22% training iterations reduction.

Recently unmanned aerial vehicle (UAV) has been widely applied in military and civil fields due to its strong autonomic and adaptability. Compared with the manned vehicles, UAV has significant advantages in carrying out the dangerous work by keeping human life away from risks. Although UAV provides notable benefits to practical applications, it gives rise to a complex path planning problem. The optimal flying path of a UAV should be obtained such that the flight length and time cost can be reduced as much as possible. In this paper, we study the path planning problem and propose an improved A* algorithm to solve the problem. First, with the models of UAVs and regions, an exact formulation based on mixed integer linear programming (MILP) is introduced to completely search the solution space. Then, by improving the evaluation function and the node selection strategy, an improved A* algorithm is presented to produce an optimal flight path for UAVs. Experimental results show that the approach proposed is more effective to solve the path planning problem than the traditional algorithms.