(Fuzzy) Database management systems aim to provide tools for data storage and ing. Based on the stored information, systems can offer analytical functionalities in order to deliver decisional database environments. In many application areas, fuzzy systems have proven to be efficient for modeling, reasoning, and predicting with imprecise information. However, expanding the frontiers of such areas or exploring new domains is often limited when facing real world data: as the space to search get bigger, more computation time and memory space are required. In this chapter, we discuss how the parallelization of fuzzy algorithms is crucial to tackle the problem of scalability and optimal performance in the context of fuzzy database mining. More precisely, we present the parallelization of fuzzy database mining algorithms on multi-core architectures of two knowledge discovery paradigms, namely fuzzy gradual pattern mining and fuzzy tree mining (for example in the case of XML databases). We also present a review of other two related problems, namely fuzzy association rule mining and fuzzy clustering.
Gradual patterns aim at describing co-variations of data such as the higher the size, the higher the weight. In recent years, such patterns have been studied more and more from the data mining point of view. The extraction of such patterns relies on efficient and smart orderings that can be built among data, for instance, when ordering the data with respect to the size, then the data are also ordered with respect to the weight. However, in many application domains, it is hardly possible to consider that data values are crisply ordered. When considering gene expression, it is not true from the biological point of view that Gene 1 is more expressed than Gene 2, if the levels of expression only differ from the tenth decimal. We thus consider fuzzy orderings and fuzzy gamma rank correlation. In this paper, we address two major problems related to this framework: (i) the high memory consumption and (ii) the precision, representation and efficient storage of the fuzzy concordance degrees versus the loss or gain of computing power. For this purpose, we consider multi-precision matrices represented using sparse matrices coupled with parallel algorithms. Experimental results show the interest of our proposal.
Mining gradual patterns invokes a number of iterations for generating, adjusting, measuring, and comparing gradual tendencies between numeric attributes of imprecise or uncertain databases. Gradual tendencies are complex correlations of the form {The hight/lower X, the hight/lower Y}. Automatic extraction of such gradual patterns involves huge amounts of processing time, load balance, and high memory consumption. When managing large databases, taking this into account is challenging. In this paper, we show a framework and an algorithm based on rank correlation and fuzzy orderings for mining gradual patterns from imprecise or uncertain data. We also present an approach to improve performance of the algorithm using the parallel programming model of OpenMP and the Yale Sparse Matrix Format to reduce memory consumption. Through an experimental study, we show the performance of our approach with respect to the number of attributes of the databases and the number of cores available.
Mining gradual patterns plays a crucial role in many real world applications where huge volumes of complex numerical data must be handled, e.g., biological databases, survey databases, data streams or sensor readings. Gradual patterns highlight complex order correlations of the form "The more/less X, the more/less Y". Only recently algorithms have appeared to mine efficiently gradual rules. However, due to the complexity of mining gradual rules, these algorithms cannot yet scale on huge real world datasets. In this paper, we thus propose to exploit parallelism in order to enhance the performances of the fastest existing one (GRITE) on multicore processors. Through a detailed experimental study, we show that our parallel algorithm scales very well with the number of cores available.
Dans cet article, nous proposons un cadre pour traiter deux grands problemes lors de l'extraction de motifs graduels basee sur les ordres flous et sur le coefficient de correlation de rang gamma flou. Les problematiques abordees sont i) la consommation memoire et ii) la precision, la representation, et le stockage efficace des degres de concordance floue de chaque paire d'indices (i, j) par rapport a la perte ou le gain de puissance de calcul. Dans ce contexte, notre approche implique l'utilisation d'une technique dediee au traitement des matrices creuses (afin d'eviter le stockage des valeurs zero) et une vaste gamme de representations de precision variable (de 1 a 64 bits).
In this paper, we study the mining of gradual patterns in the presence of numeric attributes belonging to data sets. The field of gradual pattern mining have been recently proposed to extract covariations of attributes, such as: {the higher the age, the higher the salary}. This gradual pattern denoted as {size≥salary≥} means that the age of people increases together with their salary. Actually, the analysis of such correlations is very memory consuming. When managing huge databases, issue is very challenging. In this context, we focus on the use of fuzzy orderings to take this into account and we propose techniques in order to optimize the computation. These techniques are based on a matrix representation of fuzzy concordance degrees C(i; j) and the Yale Sparse Matrix Format.
Data aggregation is one of the key features used in databases, especially for Business Intelligence (e.g., ETL, OLAP) and analytics/data mining. When considering SQL databases, aggregation is used to prepare and visualize data for deeper analyses. However, these operations are often impossible on very large volumes of data regarding memory-and-time-consumption. In this paper, we show how NoSQL databases such as MongoDB and its key-value stores, thanks to the native MapReduce algorithm, can provide an efficient framework to aggregate large volumes of data. We provide basic material about the MapReduce algorithm, the different NoSQL databases (read intensive vs. write intensive). We investigate how to efficiently modelize the data framework for BI and analytics. For this purpose, we focus on read intensive NoSQL databases using MongoDB and we show how NoSQL and MapReduce can help handling large volumes of data.
Gradual patterns highlight complex order correlations of the form "The more/less X, the more/less Y". Only recently algorithms have appeared to mine efficiently gradual rules. However, due to the complexity of mining gradual rules, these algorithms cannot yet scale on huge real world datasets. In this paper, we propose to exploit parallelism in order to enhance the performances of the fastest existing one (GRITE). Through a detailed experimental study, we show that our parallel algorithm scales very well with the number of cores available.
Fuzzy tree mining has been recently introduced in order to extract frequent subtrees from databases of labeled trees. It has many applications, especially for handling semi-structured data (e.g., XML). In this framework, soft approaches have been proposed, also known as fuzzy tree mining. They allow the methods to better recognize patterns that are embedded in the database, even if the patterns are only partially present. However, such soft methods have to cope with the problem of remaining scalable on huge volumes of data, regarding both time and memory consumption. It is thus interesting to take advantage of the new generations of computers with multi-core architectures. We thus propose an original method for parallelizing fuzzy tree mining. This paper presents our approach and discusses the main problems addressed and solutions proposed, based on the experimental results.
Many basic computations can be done by means of iterative neighborhood-based calculations, including threshold, optimum, distance transform, contour closing, mathematical morphology, etc. Some of them can be performed using rows-per-rows scans (A. Rosenfeld and J.-L. Pfaltz, 1966) (G. Borgefors, 1986). Such regular computations allow to optimize the use of caches on standard architecture, and to achieve computations in good times. However, these basic computations are also useful inside the regions of the images. When applied on some regions instead of the whole image, more scans could be necessary because of the irregular shape of the regions. In this paper, we show that row-per-row scans can be used for a large class of operators, so-called idempotent r-operators (including the previously cited computations). Moreover, we give conditions on the use of scans to perform neighborhood-based computations inside any regions. Among other results, we show that only two scans allow to compute a distance transform in every regions used in classical split and merge algorithms (S.L. Horowitz and T. Pavlidis, 1976), and only three scans are sufficient on Voronoi regions (M. Tuceryan and A.K. Jain, 1990). These results extend the use of row per row scans to more cases, and improve many algorithms that rely on some neighborhood-based computations inside regions.
Several hard problems have to be addressed in order to parallelize image analysis algorithms. Indeed, at the region level, these algorithms handle irregular (and sometimes strongly dynamic) data-structures. Moreover, they often lead to an unbalanced amount of computations, which is quite impossible to foresee offline. This paper focus on the parallelization of the ANET image analysis programming environment. Thanks to graph related data structures and efficient computing primitives, ANET allows rapid image algorithm prototyping. But in return, these primitives are difficult to parallelize. We present a solution for powerful implicit parallelization of the ANET environment, without any change in the application programming interface. The ANET API is summarized and illustrated with some examples. Several parallelization experimentations are reported. The solution we propose is detailed, and results are given on complete image analysis applications. ANET appears as a powerful environment, both for its expressiveness that allows rapid prototyping and for its implicit parallelization that allows good computation time.
In this paper we present the programming environment Anet for image analysis, that aims to bridge the gap between programmability requirements and parallel efficiency. It is based on the graph based associative nets computing model, and allows irregular data manipulation. As it is intrinsically a parallel model, parallel execution can be quite naturally considered, and as the number of primitives is small, effective parallelization requires an initial limited effort and can be reused by a large set of programs.
Pascal Poncelet合作论文数University Montpellier 2 - LIRMM1
Benjamin Négrevergne合作论文数Grenoble University, France1