Cooperative target search (CTS) technology is highly desirable in various multi-autonomous aerial vehicle (AAV) applications. However, searching for unknown targets in a dynamic threatening environment is a challenging problem, especially for AAVs with limited sensing range and communication capabilities. Besides, traditional searching methods lack scalability and efficient collaboration among the AAV swarm in dynamic environments. In this work, a digital twin (DT)-enabled distributed CTS approach was presented for AAV swarms and achieving sim-to-real transfer. Specifically, a new scalable multi-agent reinforcement learning (MARL) based algorithm called SAMARL is adopted to improve effectiveness and adaptability, combining a multi-head attention mechanism. In SAMARL, a scalable observation space with graph representation and an environmental cognition map is designed to thoroughly consider the target search rate, area coverage, and safety assurance. Then, a DT-driven training framework is proposed to facilitate the continuous evolution of MARL models and address the tradeoff between training speed and environment fidelity. Furthermore, we innovatively develop a distributed AAV swarm digital twin cooperative target search validation system, including real flight control, communication simulation tools, and a 3D physics engine. Extensive simulations validate its superiority compared to state-of-the-art strategies. More importantly, we also conduct real-world flight experiments on different scale mission areas and AAV swarms, further demonstrating the generalization and scalability of trained models.

As an advanced framework, the digital twin network (DTN) provides effective management and decision support for autonomous aerial vehicle (AAV) swarms and has become a recent research hotspot. The effectiveness of many DTN applications relies on the assumption that high-fidelity digital twin (DT) models exist and are readily available. However, constructing such high-fidelity DT models of AAV swarms is a challenging task, especially in complex and dynamic environments. Despite its importance, there is a notable lack of research focused on the construction of high-fidelity DT models specifically for AAV swarms. This study proposes a dynamic data-driven approach for constructing and calibrating DT models of AAV swarms to achieve long-term consistency with real-world AAV behaviors. The method leverages parameter identification to estimate key parameters of DT models and data assimilation to refine and calibrate the model. It can provide high-fidelity DT AAV models for artificial intelligence model training and facilitate AAV swarm DTN from concept to real application. Additionally, this article developed a DT simulation platform for AAV swarms, validating the proposed method through software-in-the-loop simulations and physical testing. Results indicate that the optimal subpattern assignment metric decreases by an average of 79.2% after calibration, significantly improving the DT model's fidelity.

Multi-agent deep reinforcement learning (MADRL) has become a typical paradigm for the flocking motion of UAV swarm in dynamic, stochastic environments. However, sim-to-real problems, such as reality gap, training efficiency, and safety issues, restrict the application of MADRL in flocking motion scenarios. To address these problems, we first propose a digital twin (DT)-enabled training framework. With the assistance of high-fidelity digital twin simulation, effective policies can be efficiently trained. Based on the multi-agent proximal policy optimization (MAPPO) algorithm, we then design the learning approach for flocking motion with matching observation space, action space, and reward function. Afterward, we employ a distributed flocking center estimation algorithm based on position consensus. The estimated center is used as a policy input to improve the aggregation behavior. Moreover, we introduce a repulsion scheme, which applies an additional repulsion force to the action to prevent UAVs from colliding with neighbors and obstacles. Simulation results show that our method performs well in maintaining flocking formation and avoiding collisions, and has better decision-making ability in near-realistic environments.

Unmanned aerial vehicles (UAVs) deployed as a swarm can offer a flexible and cost-efficient solution for surveillance missions in large-scale adversarial environments. UAV swarms possess superior real-time surveillance capabilities in detecting time-sensitive targets with speed and security, surpassing alternative techniques such as satellites and ground-based sensors. However, the area surveillance problem in adversarial scenarios using UAV swarms necessitates careful consideration of factors such as swarm positioning, flight attitudes, antenna directionality, the threat posed by adversary detection systems (ADSs), and swarm network topology. To address this challenge, we propose the Integrated Low Detection Probability (ILDP) deployment method for UAV swarms, which incorporates the Danger Avoidance Distributed Motion control algorithm (DADM) for swarm collaboration and the Low Detection Probability Topology Control algorithm (LDPTC) for swarm networking. The DADM algorithm facilitates swarm cooperation in achieving area coverage and evading ADSs by leveraging neighboring, environmental, and threat information in adversarial scenarios. Furthermore, the LDPTC algorithm establishes a topology optimization model that comprehensively considers swarm distribution and the impact of directional antenna sidelobes to reduce transmitting energy leakage on the ground. Our strategy significantly decreases the detection probability of UAV swarms by ADSs, ensuring the operational effectiveness of UAV swarms in dynamic adversarial scenarios. Extensive simulations validate the superiority of our proposed ILDP method, demonstrating considerably lower detection probabilities compared to other approaches in static and dynamic adversary environments across various swarm scales. Moreover, our method excels in real-time surveillance capabilities with lower computational complexity, freeing up computing resources for UAVs to fulfill additional tasks.

Unmanned aerial vehicle (UAV) swarm networking and collaboration have significant prospects in both civilian and military applications, due to its remarkable properties in cooperative efficiency, reduced risks, and operational cost. Traditional algorithms have challenging issues of high computational complexity and low efficiency in UAV swarm networking and collaboration, while computational intelligence (CI) has attracted increasing attention since it has advantages in solving complex optimization problems. The networking of UAV swarms serves as an essential foundation for collaboration, and intelligent collaboration is a crucial means of enhancing the performance of UAV swarm systems. To date, extensive CI-based algorithms have been proposed to improve the networking and collaboration capabilities of UAV swarms, and several relevant surveys have also been presented. However, existing surveys either review networking or collaboration. To the best of our knowledge, there is no survey that simultaneously concentrates on CI-based UAV swarm networking and collaboration. In this survey, we provide a comprehensive overview of CI-based networking and collaboration algorithms from six typical aspects including channel access, network routing, cooperative task assignment, cooperative path planning, cooperative search, and cooperative jamming. More importantly, to help researchers choose appropriate algorithms to satisfy the requirements of different missions, we classify CI-based algorithms into four categories, namely heuristic behavior search-based algorithms, policy design-based algorithms, policy learning-based algorithms, and hybrid algorithms. Finally, we discuss open issues and future directions that may influence future research on UAV swarm intelligence networking and collaboration. This review may provide new insights and valuable references for researchers in this field.

This paper considers the cooperative search for stationary targets by multiple unmanned aerial vehicles (UAVs) with limited sensing range and communication ability in a dynamic threatening environment. The main purpose is to use multiple UAVs to find more unknown targets as soon as possible, increase the coverage rate of the mission area, and more importantly, guide UAVs away from threats. However, traditional search methods are mostly unscalable and perform poorly in dynamic environments. A new multi-agent deep reinforcement learning (MADRL) method, DNQMIX, is proposed in this study to solve the multi-UAV cooperative target search (MCTS) problem. The reward function is also newly designed for the MCTS problem to guide UAVs to explore and exploit the environment information more efficiently. Moreover, this paper proposes a digital twin (DT) driven training framework “centralized training, decentralized execution, and continuous evolution” (CTDECE). It can facilitate the continuous evolution of MADRL models and solve the tradeoff between training speed and environment fidelity when MADRL is applied to real-world multi-UAV systems. Simulation results show that DNQMIX outperforms state-of-art methods in terms of search rate and coverage rate.

Over the past decades, unmanned aerial vehicles (UAVs) have been widely used in both military and civilian fields. In these applications, flocking motion is a fundamental but crucial operation of multi-UAV systems. Traditional flocking motion methods usually designed for a specific environment. However, the real environment is mostly unknown and stochastic, which greatly reduces the practicality of these methods. In this article, deep reinforcement learning (DRL) is used to realize the flocking motion of multi-UAV systems. Considering that the sim-to-real problem restricts the application of DRL to the flocking motion scenario, a digital twin (DT)-enabled DRL training framework is proposed to solve this problem. The DRL model can learn from DT and be quickly deployed on the real-world UAV with the help of DT. Under this training framework, this article proposes an actor–critic DRL algorithm, named behavior-coupling deep deterministic policy gradient (BCDDPG), for the flocking motion problem, which is inspired by the flocking behavior of animals. Extensive simulations are conducted to evaluate the performance of BCDDPG. Simulation results show that BCDDPG achieves a higher average reward and performs better in terms of arrival rate and collision rate compared with the existing methods.

With high mobility, low cost and outstanding maneuverability properties, unmanned aerial vehicle (UAV) swarm has attracted worldwide attentions in both academia and industry. Nevertheless, the complex and coherent characteristics of the intelligent cooperation of UAV swarm greatly restrict its wide application. The recent development of artificial intelligence provides new methodologies for intelligent cooperation of UAV swarm. However, these methods are resource-in-tensive that cannot be directly applied in the computation and storage constrained UAVs. In this article, we propose a novel digital twin (DT)-based intelligent cooperation framework of UAV swarm. In the framework, a digital twin model is established to reflect the physical entity (i.e., UAV swarm) with high-fidelity and monitors its whole life cycle. Next, the decision model that integrates a machine learning algorithm is built to explore the global optimal solution and controls the behaviors of UAV swarm. To demonstrate the effectiveness of our proposed framework, a case study on intelligent network reconstruction is introduced, and simulation results are presented. Finally, a representative application provided by the framework is discussed.

To address the problems of high latency and low channel utilization in underwater acoustic sensor networks,this paper proposes a new multi-link transmission Medium Access Control(MAC) protocol.In this protocol,nodes exchange the delay information and transmission plans through the RTS/CTS handshake protocol to implement multi-link data transmission.When the sink node receives data from multiple nodes,the sink node plans the sending order of nodes according to the delay between nodes and the size of the received data frames,avoiding the energy loss and the low channel utilization rate caused by transmission conflicts.At the same time,in order to obtain the maximum network throughput,the theoretical expression of the optimal backoff window value is given.Simulation results show that compared with the RC-FAMA,S-FAMA protocols,this protocol can improve the throughput of underwater acoustic sensor networks and reduce transmission conflicts,thus prolonging the lifetime of underwater nodes.

Nowadays unmanned aerial vehicle (UAV) has gradually entered people's vision because of its huge application potential, like target tracking, area coverage, remote sensing, etc. In the considered FANETs, one crucial problem is how to decrease the probability of being discovered by an adversary’s detection system, while achieving efficient deployment and maintaining the connectivity of the network. To solve this problem, we propose a distributed virtual force motion control (DVFMC) algorithm. This algorithm requires no global information, and every node only finds and communicates with its neighbors automatically. They automatically maintain distance with high-risk areas and perform power control to decrease the energy leakage and therefore have a low probability of detection (LPD) coverage. Our comparative simulation results validate that our algorithm can reduce greatly the probability of being discovered by the adversary’s detection system.

Orthogonal circular orbit constellation combines the advantages of polar orbit constellation and equatorial orbit constellation to achieve continuous global coverage,improving the coverage performance of pure polar orbit constellation at low and middle latitudes to some extent.However,the traditional design method of orthogonal circular orbit constellation is to divide the coverage of polar orbit constellation and equatorial orbit constellation by latitude lines,which is too rough to fully utilize geometric properties of two constellations.To solve this problem,according to the motion features and coverage features of the polar orbit constellation satellites,this paper analyzes the features of the coverage gaps generated in the areas that do not meet the continuous coverage requirements,and integrates the characteristics of the equatorial orbit constellation satellites to propose a design method of orthogonal circular orbit constellation based on geometric analysis.By using the analytical method,the parameters of the half-width angle for minimum ground coverage of the equatorial satellite is determined.The typical constellation scheme is given to compare with the traditional design method of orthogonal circular orbit constellation,and the effectiveness of the proposed method is demonstrated by the STK simulations.

The reconstruction of routing schemes constitutes an important quality of service (QoS) support for UAV swarms to keep their applications and services stable and active. In this paper, we develop a forwarding distance and available bandwidth (AB) estimation based opportunistic routing (FD-ABOR) protocol for high dynamic UAV swarms. We first improve the AB estimation algorithm for multi-hop UAV ad hoc networks by taking full account of the disparity between the sending and receiving ability and reconsidering the bandwidth consumption induced by hidden nodes. Secondly, we propose the scheme that the sender broadcast the RTS frame piggybacked with available bandwidth information for route request. Based on available bandwidth and forwarding distance, neighbour node decides it to be the candidate forwarder and computes the forwarding priority, then competes with each other to forward data packets. The simulation results show that our method outperforms AODV and DSR in terms of throughput and packet delivery ratio.

Opportunistic medium access control (MAC) protocol design is a challenging issue for multi-channel cognitive radio networks (CRN). This paper provides a novel traffic-priority-based opportunistic MAC (TPO-MAC) protocol in multi-channel CRN. An important mechanism in TPO-MAC is that whether the primary users (PUs) can access the licensed channels immediately depends on their traffic priority, which alleviates the collision between secondary users (SUs) and PUs. In order to investigate the performance of TPO-MAC, a Markov chain model is further proposed to derive the saturation throughput of TPO-MAC. We validate our model by extensive simulations and compare the performance of TPO-MAC with the results from other literature. The results show that TPO-MAC performs better than the existing opportunistic MAC protocols.

In wireless ad hoc networks, a suitable MAC protocol can provide better QoS (Quality of Service) guarantees. However, in modern warfare environment, the combat missions change rapidly. A fixed MAC protocol may be not suitable for the highly changeable environments and combat missions. To solve this problem, we propose a neural network-based MAC protocol-selection mechanism to select the optimal MAC protocol for new missions based on the real-time environment information of the battlefield, i.e., the intelligent selection (IS) algorithm. In IS, we prepare the empirical data set with massive mission data, and construct the learning model through making use of the neural network. Experimental results demonstrate that the prediction accuracy of the proposed IS algorithm is over 0.93.

Directional antennas have numerous advantages over omnidirectional antennas in CSMA/CA ad hoc networks. However, estimating the available bandwidth of a flow in the medium access process in such a network is very challenging. In this paper, we present a passive available bandwidth estimation algorithm for directional ad hoc networks, termed PABE-D. The estimation process consists of two phases, i.e., the preliminary and refined estimation phases. In the preliminary phase, the available transmission/reception duration in each beam of the node and the available bandwidth of the directional link are obtained by analyzing the available duration of both the sender and receiver. In the refined phase, we analyze the effect of the directional hidden terminal and deafness problems to further improve the estimate accuracy. The performance of our proposed PABE-D algorithm is evaluated in two typical topologies, i.e., the parallel and random grid topologies. The simulation results demonstrate that our algorithm can effectively estimate the available bandwidth in directional ad hoc networks.

This paper analyzes the limitation of IEEE 802.11 protocol using a single power for node communication. On the basis of IEEE 802.11protocol and the basic power backoff protocol which introduces the idea of power backoff, a CSMATPB protocol with time and space backoff is proposed. The protocol increases the time competition window, while making the power backoff reduce the interference between nodes. Then, the three dimensional Markov chain model is established for this protocol. Through the results of numerical calculation and simulation analysis, it is proved that the protocol mechanism can effectively solve the problem that the lower space reuse rate of IEEE 802.11 protocol and the problem that the basic power backoff protocol discards the packet prematurely. So CSMATPB protocol can effectively improve the saturation throughput and energy utilization of the network.