Given the increasing size and dimension of streaming data made available by for example in industrial sensors or wireless sensor networks (WSNs), it is an important and worthwhile task to monitor not only the data itself but also the relationships between data sources. To solve this task, we present DIMID, an online algorithm to monitor dependencies in high dimensional streaming data. DIMID uses an entropy-based measure that generalizes to non-linear as well as complex functional types of relationships, is non-parametric and can be computed incrementally. To deal with the streaming, possibly infinite data, DIMID contains a dimensionality reducing projection method and an estimator for entropy that uses the local density of data points. This also allows the algorithm to update the current relationships with new data as it becomes available, instead of recomputing on the complete batch after every update. Comparisons to three state-of-the-art other algorithms for dependency-monitoring on a variety of time series data sets with linear and non-linear dependencies showed significant ( p < 0.01) improvements in the AUC- and F1-measure. We also achieve a reduction in run-time from linear to logarithmic in the number of observed samples.

Intelligent production in smart factories or wearable devices that measure our activities produce on an ever growing amount of sensor data. In these environments, the validation of measurements to distinguish sensor flukes from significant events is of particular importance. We developed an algorithm that detects dependencies between sensor readings. These can be used for instance to verify or analyze large scale measurements. An entropy based approach allows us to detect dependencies beyond linear correlation and is well suited to deal with high dimensional and high volume data streams. Results show statistically significant improvements in reliability and on-par execution time over other stream monitoring systems.

Efficient learning algorithms gain importance in the time of Big Data and decentralized data processing (e.g. Internet of Things). The thesis evaluates one class of such efficient algorithms: the incremental linear model trees. The algorithms are systematically compared on stationary Big Data and are improved in their usability by the new pruning extension GuIP and the new data stream processing system PAFAS.

Increasingly, data streams are generated from a growing number of small, cheap sensors that monitor, e.g., personal activities, industrial facilities or the natural environment. In these settings, there are often rapid changes in input-to-target relations and we are concerned with tree-structured models that can rapidly adapt to these changes. Based on our new algorithms accuracy and tracking behavior is improved, which we demonstrate for a number of popular tree based-classifiers with over state-of-the-art change detection using five data sets and two different settings. The key novel idea is the representation of record values as distributions rather than point-values in the stream setting, covering a larger part of the instance space early on, and resulting in an often smaller, more flexible classification model.

Incremental linear model trees with approximate lookahead are fast, but produce overly large trees. This is due to non-optimal splitting decisions boosted by a possibly unlimited number of examples obtained from a data source. To keep the processing speed high and the tree complexity low, appropriate incremental pruning techniques are needed. In this paper, we introduce a pruning technique for the class of incremental linear model trees with approximate lookahead on stationary data sources. Experimental results show that the advantage of approximate lookahead in terms of processing speed can be further improved by producing much smaller and consequently more explanatory, less memory consuming trees on high-dimensional data. This is done at the expense of only a small increase in prediction error. Additionally, the pruning algorithm can be tuned to either produce less accurate model trees at a much higher processing speed or, alternatively, more accurate trees at the expense of higher processing times.

The existence of massive datasets raises the need for algorithms that make efficient use of resources like memory and computation time. Besides well-known approaches such as sampling, online algorithms are being recognized as good alternatives, as they often process datasets faster using much less memory. The important class of algorithms learning linear model trees online (incremental linear model trees or ILMTs in the following) offers interesting options for regression tasks in this sense. However, surprisingly little is known about their performance, as there exists no large-scale evaluation on massive stationary datasets under equal conditions. Therefore, this paper shows their applicability on massive stationary datasets under various parameter settings. To reduce biases arising from the choice of a programming language or programming skills, all algorithms were reimplemented within the same framework and tested under the same conditions. Results on real-world datasets indicate that for massive stationary datasets parameter settings leading to complex models do not pay off, as there is at most a small accuracy gain at a much larger running time. Experimental evidence suggests that simple and fast algorithms perform best.

The growing number of time-labeled datasets in science and industry increases the need for algorithms that automatically induce process models. Existing methods are capable of identifying process models that typically only work on single attribute events. We propose a new model type to address the problem of mining multi-attribute events, meaning that each event is described by a vector of attributes. The model is based on timed automata, includes expressive descriptions of states and can be used for making predictions. A probabilistic real time automaton is created, where each state is annotated by a profile of events. To identify the states of the automaton, similar events are combined by a clustering approach. The method was implemented and tested on a synthetic, a medical and a biological dataset. Its prediction accuracy was evaluated on a medical dataset and compared to a combined logistic regression, which is considered a standard in this application domain. Moreover, the method was experimentally compared to Multi-Output HMMs and Petri nets learned by standard process mining algorithms. The experimental comparison suggests that the automaton-based approach performs favorably in several dimensions. Most importantly, we show that meaningful medical and biological process knowledge can be extracted from such automata.

Mining data streams has become an important topic within the last decade, and numerous approaches have addressed various issues in this domain. However, little work has been devoted to the setting where high-speed data streams are potentially faster than the underlying learning algorithm, and nevertheless a prediction needs to be given promptly for each unlabeled instance in the stream. Additionally, a model has still to be trained on the labeled instances of the stream in parallel. To solve this problem, we propose a framework and controller-based approach that ensures proper training of a model and a prediction for each unlabeled instance in the stream. The approach can also be used for data streams with altering stream speeds. We show that the method performs well under heavy system load on both synthetic and real-world data using different learning algorithms. We believe that this type of realtime machine learning, the synthesis of machine learning and real-time systems, raises a number of interesting questions for machine learning research.

The classification of wounds into healing states depending on their absorption spectrum of visible and near infrared light remains an important task in dermatology. Moreover, a reduction of the spectrum that is used in the classification task to fewer but important wavelengths is desirable, as each measured wavelength increases the examination costs without necessarily providing further information to the classification of wound healing states. This paper addresses two aspects: First the improvement of the classification of wounds into healing states and second, a cost reduction by choosing only important wavelengths. Standard Data Mining methods are evaluated for their classification accuracy (CA) and compared to their performance when applying feature selection techniques that are used to reduce the amount of necessary wavelengths. The results indicate that the 1‐nearest‐neighbor approach (IB1 algorithm) comes up with the best CA, while only relying on a fraction (4%) of the standard wavelength spectrum.

Classification of different types of dementia commonly involves examination from several perspectives, e.g., medical images, neuropsychological tests, etc. Thus, dementia classification should lend itself to so-called multi-view learning. Instead of simply combining several views, we use stacking to make the most of the information from the various views (PET scans, MMSE, CERAD and demographic variables). In the paper, we not only show the performance of stacked multi-view learning on classifying dementia data, we also try to explain the factors contributing to its performance. More specifically, we show that the correlation of views on the base and the meta level should be within certain ranges to facilitate successful stacked multi-view learning.

One of the goals of medical research in the area of dementia is to correlate images of the brain with clinical tests. Our approach is to start with the images and explain the differences and commonalities in terms of the other variables. First, we cluster Positron emission tomography (PET) scans of patients to form groups sharing similar features in brain metabolism. To the best of our knowledge, it is the first time ever that clustering is applied to whole PET scans. Second, we explain the clusters by relating them to non-image variables. To do so, we employ RSD, an algorithm for relational subgroup discovery, with the cluster membership of patients as target variable. Our results enable interesting interpretations of differences in brain metabolism in terms of demographic and clinical variables. The approach was implemented and tested on an exceptionally large data collection of patients with different types of dementia. It comprises 10 GB of image data from 454 PET scans, and 42 variables from psychological and demographical data organized in 11 relations of a relational database. We believe that explaining medical images in terms of other variables (patient records, demographic information, etc.) is a challenging new and rewarding area for data mining research.

We present a new and comprehensive approach to inductive databases in the relational model. The main contribution is a new inductive query language extending SQL, with the goal of supporting the whole knowledge discovery process, from pre-processing via data mining to post-processing. A prototype system supporting the query language was developed in the SINDBAD (structured inductive database development) project. Setting aside models and focusing on distance-based and instance-based methods, closure can easily be achieved. An example scenario from the area of gene expression data analysis demonstrates the power and simplicity of the concept. We hope that this preliminary work will help to bring the fundamental issues, such as the integration of various pattern domains and data mining techniques, to the attention of the inductive database community.