Research on High-dimensional Data Mining Based on Big Data

In the era of big data, with the rapid development of the Internet, the feature vector dimensions of most data are very high, and their huge dimensions will cause dimensional disasters, posing great obstacles to data processing and storage. Traditional data processing methods face new and severe challenges, and high-dimensional data mining technology has become the main research direction. Starting from the development challenges of big data, this article summarizes the characteristics of high-dimensional data and its impact on data mining, focusing on high-dimensional data mining technology as the research object, a mathematical model based on three-dimensional matrices was established, the transformation between traditional databases and three-dimensional matrices was analyzed, and a three-dimensional model of big data was established, the redundancy problem of association rules was solved by minimizing support's degree and confidence level, resulting in high-quality association rules, taking multi-dimensional data of ocean as an example, extracting high-dimensional data themes from big data, this article analyzes the clustering algorithm based on hypergraph using the implementation of HMETIS algorithm. These methods, models, and algorithms all have certain scalability and have positive research significance and practical value for subsequent data mining work.