Frequent Pattern Mining for IoT Big Streaming Data Based on Short-Pattern Graph

Internet of things (IoT) technology is important in the era of big data, and in most IoT scenarios, data are constantly arriving in the form of streams. Streaming data have a fast frequency and a large volume, which, over time, has no upper limit. Streaming data contain much valuable information, such as the behaviors of IoT objects, which can be mined through frequent pattern mining algorithms. Due to high spatial complexity, such methods must adopt window mechanisms to reduce spatial and temporal costs. However, the statistics of frequent patterns based on window patterns are local, and for global data, some important patterns may be overlooked. Therefore, we propose a frequent pattern mining method for streaming data based on a new data structure, the short-pattern graph (SP-graph), aiming to significantly improve the upper limit of the data volume. The proposed algorithm does not need to set windows, or set sufficiently large windows to avoid missing important patterns. We first define the SP-graph data structure, and propose a frequent pattern mining algorithm based on it, as well as a real-time object behavior prediction algorithm and pattern-containing mining algorithm. We experimentally compare our method with classical methods on a real streaming dataset, and verify its much lower spatial complexity, greater data capacity, ability to flexibly adjust the frequency threshold, and offline and online mining efficiency, which enable real-time performance.