In this paper, we study efficient rate control schemes for delay sensitive communications over wireless fading channels based on reinforcement learning. Our objective is to find a rate control scheme that optimizes the link layer performance, specifically, maximizes the system throughput subject to a fixed bit error rate (BER) constraint and longterm average power constraint. We assume the buffer at the transmitter is finite; hence packet drop happens when the buffer is full. We assume the fading channel under our study can be modeled as a finite state Markov chain, however the transition probability of channel states is not known, and the only information available about the wireless channel is the instantaneous channel gain, which is estimated and fed back from receiver side to the transmitter side on the fly. In this paper, we use reinforcement learning approach to learn the time-varying channel environment and search for the optimal control policy on line. Simulation results show that starting from an arbitrary control policy, the learning agent gradually modifies its estimation about the system model and adjusts the control policy to its optimality.

Cooperative diversity is an effective technique for transmission performance improvement over fading channels. In this paper, through the use of a virtual antenna array, a time division multiple access (TDMA) based cooperative media access control (MAC) protocol, Cooperative TDMA (C-TDMA), is proposed to enable cooperative transmissions in Rayleigh fading channels to improve the probability of correct packet reception and system throughput. A C-TDMA throughput expression is derived using a Markov chain model. Numerical results show that the throughput of conventional TDMA can be improved by over 40% through the use of the cooperative MAC designs.

This paper proposes a method to protect the communication band through machine learning in cognitive networks. A machine learning cognitive radio (MLCR) extracts features from the signal waveforms received from various radios. A machine learning radio user (MLRU) assigns the states, i.e., unauthorized/authorized, and the associated actions, i.e., interfering/no interfering, to each waveform. The MLCR learns through a proposed hierarchical neural network to classify the signal states based on their features. The {signal, action} pairs are stored in the knowledge base and can be retrieved by MLCR automatically based on its prediction of the signal state related to the presented signal waveform. A case study of protecting the band of a legacy radio using our proposed method is provided to validate the effectiveness of this work.

In this paper, an incremental self-organizing map integrated with hierarchical neural network (ISOM-HNN) is proposed as an efficient approach for signal classification in cognitive radio networks. This approach can effectively detect unknown radio signals in the uncertain communication environment. The adaptability of ISOM can improve the real-time learning performance, which provides the advantage of using this approach for on-line learning and control of cognitive radios in many real-world application scenarios. Furthermore, we propose to integrate the ISOM with the hierarchical neural network (HNN) to improve the learning and prediction accuracy. Detailed learning algorithm and simulation results are presented in this work to demonstrate the effectiveness of this approach.