This paper investigates the task how to achieve fairness in distributed dynamic spectrum access (DSA). Specifically, we consider a cognitive radio network scenario with multiple primary users (PUs) and secondary users (SUs). Each PU operates in a licensed channel. We assume that there is no coordination between PUs and SUs, and no coordination among SUs. The key challenges for SUs are to: (1) avoid collisions with PUs, (2) avoid collisions with other SUs, (3) fair access of spectrum resources in an uncoordinated system, (4) deal with different PU activity patterns, (5) deal with spectrum sensing errors. To address these challenges, we propose a deep reinforcement learning (DRL) approach and an associated reward function to achieve fair access to spectrum resources. Specifically, we use the method of Dueling Double Deep Q-Networks with Prioritised Experience Replay (D3QN-PER) as DRL algorithm for each SU. In our simulation experiments, we demonstrate that the proposed approach performs better than existing DRL methods.

In this paper, we propose to use deep reinforcement learning (DRL) for the task of cooperative spectrum sensing (CSS) in a cognitive radio network. We selected a recently proposed offline DRL method called conservative Q-learning (CQL) due to its ability to learn complex data distributions efficiently. The task of CSS is performed as follows. Each secondary user (SU) performs local sensing and using CQL algorithm, determines the presence of licensed user for current and $k$ -1 future timeslots. These results are forwarded to the fusion centre where another CQL algorithm is operating that generates a global decision for the current and $k$ -1 future timeslots. Then, SUs do not perform sensing for the next $k$ -1 timeslots to save energy. The proposed CSS mechanism can significantly increase the licensed user detection accuracy and the data transmissions by SUs. In addition, it reduces the sensing results transmission overhead. The proposed solution is tested with a stochastic traffic load model for different activity patterns. Our simulation results show that the proposed problem formulation using the CQL algorithm can achieve similar detection accuracy as other state-of-the-art methods for CSS while significantly reducing the computation time.

In this paper, we investigate the task of quality of service (QoS) routing in software defined networks (SDN). We consider delay, bandwidth, loss, and cost as QoS parameters. We propose a new deep reinforcement learning solution for greedy online QoS routing in SDN and call it Deep Q-Routing (DQR). DQR utilises a dueling deep Q-network with prioritised experience replay to compute a path for any source-destination pair request in the presence of multiple QoS metrics. In contrast to existing DRL-based routing methods, the proposed DQR method regards the task of routing as a discrete control problem and uses a reward function comprising weighted QoS parameters. Our simulation results show that DQR substantially improves end-to-end throughput compared to other existing learning based methods.

With the introduction of communication infrastructure into the traditional power grids, smart power grids are emerging to meet the future electricity demands. In smart grid, advanced metering infrastructure (AMI) is one of the main components that enables bi-directional communication between home area networks and utility providers. In an AMI network, one of the crucial operations is to update the firmware of the smart meters. In this paper, we propose a new forwarding strategy for the firmware updates in AMI network. Our simulation results show that the completion time of the smart meter firmware update process can be reduced significantly by using the proposed new strategy.

A wireless sensor network (WSN) is a deployment of several small-sized chips called nodes, equipped with sensors to monitor physical or environmental conditions, such as temperature, sound, and pressure, and then cooperatively send back their data through the network to a central location for processing. WSNs are emerging rapidly and have a vast field of research. WSNs are cheap, smaller in size, and have intelligent nodes; due to these characteristics, a number of emerging technologies are based on them. These emerging technologies include the Internet of Things (IoT), cognitive radio sensor networks, cloud computing, cyber physical systems, smart grid, and vehicular ad hoc sensor networks. To the best of our knowledge, no work has been done in the literature so far to combine all these emerging technologies in a single source. In this chapter, our goal is to combine WSN-based emerging technologies as a single source so that reviewers can form opinions regarding these technologies.