Q-learning based energy-efficient and void avoidance routing protocol for underwater acoustic sensor networks

The routing in underwater acoustic sensor networks (UASNs) has become a challenging issue due to several problems. First, in UASN, the distance between the nodes changes due to their mobility with the water current, thus increasing the network’s energy consumption. Second problem in UASNs is the occurrence of the void hole, which affects the network’s performance. Because nodes are unable to deliver data towards the destination due to the absence of forwarder nodes (FNs) in the network. Thus, the objective of routing in UASNs is to overcome the issues mentioned earlier to prolong the network’s lifetime. Therefore, a Q-learning based energy-efficient and balanced data gathering (QL-EEBDG) routing protocol is proposed in this paper. In QL-EEBDG, the FNs are selected according to their residual energy and grouped according to their neighboring nodes’ energies. Using energy as the main selection parameter assures efficient energy consumption in the network. Moreover, efficient selection of the FNs increases the lifetime of the network. However, the void node recovery process fails when the topology of the network is changed. Therefore, to avoid void holes in QL-EEBDG, a QL-EEBDG adjacent node (QL-EEBDG-ADN) scheme is proposed. It finds alternate neighbor routes for packet transmission and ensures continuous communication in the network. Extensive simulations are carried out for the performance evaluation of the proposed technique with existing baseline protocols, namely efficient balanced energy consumption based data gathering (EBDG), enhanced EBDG (EEBDG) and QELAR. The performance parameters used in the simulations are network lifetime, energy tax, network stability period and packet delivery ratio (PDR). The simulation results depict that the proposed QL-EEBDG-ADN outperforms the baseline protocols by approximately 11% better PDR and 25% better energy tax.