Real-time Video Data Traffic Management for Publish-Subscribe Based Messaging System.

There is a growing need to transmit video data over IoT infrastructures in the same processing pipeline with sensor edge devices. As the number of devices in the same pipeline increases, consumers experience more frequent message delays while getting data from multiple sources. In this study, we propose and analyze three priority queue management algorithms for publish-subscribe based Kafka framework to utilize realtime video streaming. The algorithms are implemented on a real test-bed by deploying multiple sensors with different Quality of Service (QoS) requirements. The results reveal that the proposed solutions enhance QoS provisioning when compared to the default Kafka queue model. Moreover, the Updated Weighted Fair Kafka Queue (UWFKQ) algorithm performs better than the Weighted Fair Kafka Queue (WFKQ) and the Priority Kafka Queue algorithms for relieving starvation as well as long wait times. Besides, the slider mechanism proposed for UWFKQ algorithm enables the system to utilize unused poll counts so that congestion between packet transmission flows can be controlled. We have also shown that several default Kafka parameters can successfully be tuned to optimize the throughput rate.