An open question in ensemble-based active learning is how to choose one classifier type, or appropriate combinations of multiple classifier types, to construct ensembles for a given task. While existing approaches typically choose one classifier type, this paper presents a method that trains and adapts multiple instances of multiple classifier types toward an appropriate ensemble during active learning. The method is termed adaptive heterogeneous ensembles (henceforth referred to as AHE). Experimental evaluations show that AHE constructs heterogeneous ensembles that outperform homogeneous ensembles composed of any one of the classifier types, as well as bagging, boosting and the random subspace method with random sampling. We also show in this paper that the advantage of AHE over other methods is increased if (1) the overall size of the ensemble also adapts during learning; and (2) the target data set is composed of more than two class labels. Through analysis we show that the AHE outperforms other methods because it automatically discovers complementary classifiers: for each data instance in the data set, instances of the classifier type best suited for that data point vote together, while instances of the other, inappropriate classifier types disagree, thereby producing a correct overall majority vote.

Generating models from large data sets-and determining which subsets of data to mine-is becoming increasingly automated. However, choosing what data to collect in the first place requires human intuition or experience, usually supplied by a domain expert. This paper describes a new approach to machine science which demonstrates for the first time that nondomain experts can collectively formulate features and provide values for those features such that they are predictive of some behavioral outcome of interest. This was accomplished by building a Web platform in which human groups interact to both respond to questions likely to help predict a behavioral outcome and pose new questions to their peers. This results in a dynamically growing online survey, but the result of this cooperative behavior also leads to models that can predict the user's outcomes based on their responses to the user-generated survey questions. Here, we describe two Web-based experiments that instantiate this approach: The first site led to models that can predict users' monthly electric energy consumption, and the other led to models that can predict users' body mass index. As exponential increases in content are often observed in successful online collaborative communities, the proposed methodology may, in the future, lead to similar exponential rises in discovery and insight into the causal factors of behavioral outcomes.

Counting craters in remotely sensed images is the only tool that provides relative dating of remote planetary surfaces. Surveying craters requires counting a large amount of small subkilometer craters, which calls for highly efficient automatic crater detection. In this article, we present an integrated framework on autodetection of subkilometer craters with boosting and transfer learning. The framework contains three key components. First, we utilize mathematical morphology to efficiently identify crater candidates , the regions of an image that can potentially contain craters. Only those regions occupying relatively small portions of the original image are the subjects of further processing. Second, we extract and select image texture features, in combination with supervised boosting ensemble learning algorithms, to accurately classify crater candidates into craters and noncraters. Third, we integrate transfer learning into boosting, to enhance detection performance in the regions where surface morphology differs from what is characterized by the training set. Our framework is evaluated on a large test image of 37,500 × 56,250 m 2 on Mars, which exhibits a heavily cratered Martian terrain characterized by nonuniform surface morphology. Empirical studies demonstrate that the proposed crater detection framework can achieve an F1 score above 0.85, a significant improvement over the other crater detection algorithms.

Orthogonal Nonnegative Matrix Tri-Factorization (ONMTF), a dimension reduction method using three small matrices to approximate an input data matrix, clusters the rows and columns of an input data matrix simultaneously However, ONMTF is computationally expensive due to an intensive computation of the Lagrangian multipliers for the orthogonal constraints In this paper, we introduce Fast Orthogonal Nonnegative Matrix Tri-Factorization (FONT), which uses approximate constants instead of computing the Lagrangian multipliers As a result, FONT reduces the computational complexity significantly Experiments on document datasets show that FONT outperforms ONMTF in terms of clustering quality and running time Moreover, FONT is further accelerated by incorporating Alternating Least Squares, and can be much faster than ONMTF.

An ensemble is a set of learned models that make decisions collectively. Although an ensemble is usually more accurate than a single learner, existing ensemble methods often tend to construct unnecessarily large ensembles, which increases the memory consumption and computational cost. Ensemble pruning tackles this problem by selecting a subset of ensemble members to form subensembles that are subject to less resource consumption and response time with accuracy that is similar to or better than the original ensemble. In this paper, we analyze the accuracy/diversity trade-off and prove that classifiers that are more accurate and make more predictions in the minority group are more important for subensemble construction. Based on the gained insights, a heuristic metric that considers both accuracy and diversity is proposed to explicitly evaluate each individual classifier's contribution to the whole ensemble. By incorporating ensemble members in decreasing order of their contributions, subensembles are formed such that users can select the top $p$ percent of ensemble members, depending on their resource availability and tolerable waiting time, for predictions. Experimental results on 26 UCI data sets show that subensembles formed by the proposed EPIC (Ensemble Pruning via Individual Contribution ordering) algorithm outperform the original ensemble and a state-of-the-art ensemble pruning method, Orientation Ordering (OO).

Object The authors describe the artificial neural network (ANN) as an innovative and powerful modeling tool that can be increasingly applied to develop predictive models in neurosurgery. They aimed to demonstrate the utility of an ANN in predicting survival following traumatic brain injury and compare its predictive ability with that of regression models and clinicians. Methods The authors designed an ANN to predict in-hospital survival following traumatic brain injury. The model was generated with 11 clinical inputs and a single output. Using a subset of the National Trauma Database, the authors “trained” the model to predict outcome by providing the model with patients for whom 11 clinical inputs were paired with known outcomes, which allowed the ANN to “learn” the relevant relationships that predict outcome. The model was tested against actual outcomes in a novel subset of 100 patients derived from the same database. For comparison with traditional forms of modeling, 2 regression models were developed using the same training set and were evaluated on the same testing set. Lastly, the authors used the same 100-patient testing set to evaluate 5 neurosurgery residents and 4 neurosurgery staff physicians on their ability to predict survival on the basis of the same 11 data points that were provided to the ANN. The ANN was compared with the clinicians and the regression models in terms of accuracy, sensitivity, specificity, and discrimination. Results Compared with regression models, the ANN was more accurate (p < 0.001), more sensitive (p < 0.001), as specific (p = 0.260), and more discriminating (p < 0.001). There was no difference between the neurosurgery residents and staff physicians, and all clinicians were pooled to compare with the 5 best neural networks. The ANNs were more accurate (p < 0.0001), more sensitive (p < 0.0001), as specific (p = 0.743), and more discriminating (p < 0.0001) than the clinicians. Conclusions When given the same limited clinical information, the ANN significantly outperformed regression models and clinicians on multiple performance measures. While this paradigm certainly does not adequately reflect a real clinical scenario, this form of modeling could ultimately serve as a useful clinical decision support tool. As the model evolves to include more complex clinical variables, the performance gap over clinicians and logistic regression models will persist or, ideally, further increase.

Identifying impact craters on planetary surfaces is one fundamental task in planetary science. In this paper, we present an embedded framework on auto-detection of craters, using feature selection and boosting strategies. The paradigm aims at building a universal and practical crater detector. This methodology addresses three issues that such a tool must possess: (i) it utilizes mathematical morphology to efficiently identify the regions of an image that can potentially contain craters; only those regions, defined as crater candidates, are the subjects of further processing; (ii) it selects Haar-like image texture features in combination with boosting ensemble supervised learning algorithms to accurately classify candidates into craters and non-craters; (iii) it uses transfer learning, at a minimum additional cost, to enable maintaining an accurate auto-detection of craters on new images, having morphology different from what has been captured by the original training set. All three aforementioned components of the detection methodology are discussed, and the entire framework is evaluated on a large test image of 37,500 x 56,250$ m2 on Mars, showing heavily cratered Martian terrain characterized by nonuniform surface morphology. Our study demonstrates that this methodology provides a robust and practical tool for planetary science, in terms of both detection accuracy and efficiency.

Previous chapter Next chapter Full AccessProceedings Proceedings of the 2010 SIAM International Conference on Data Mining (SDM)Adaptive Informative Sampling for Active LearningZhenyu Lu, Xindong Wu, and Josh BongardZhenyu Lu, Xindong Wu, and Josh Bongardpp.894 - 905Chapter DOI:https://doi.org/10.1137/1.9781611972801.78PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAboutAbstract Many approaches to active learning involve periodically training one classifier and choosing data points with the lowest confidence. An alternative approach is to periodically choose data instances that maximize disagreement among the label predictions across an ensemble of classifiers. Many classifiers with different underlying structures could fit this framework, but some ensembles are more suitable for some data sets than others. The question then arises as to how to find the most suitable ensemble for a given data set. In this work we introduce a method that begins with a heterogeneous ensemble composed of multiple instances of different classifier types, which we call adaptive informative sampling (AIS). The algorithm periodically adds data points to the training set, adapts the ratio of classifier types in the heterogeneous ensemble in favor of the better classifier type, and optimizes the classifiers in the ensemble using stochastic methods. Experimental results show that the proposed method performs consistently better than homogeneous ensembles. Comparison with random sampling and uncertainty sampling shows that the algorithm effectively draws informative data points for training. Previous chapter Next chapter RelatedDetails Published:2010ISBN:978-0-89871-703-7eISBN:978-1-61197-280-1 https://doi.org/10.1137/1.9781611972801Book Series Name:ProceedingsBook Code:PR136Book Pages:1-953

Many approaches to active learning involve training one classifier by periodically choosing new data points about which the classifier has the least confidence, but designing a confidence measure without bias is nontrivial. An alternative approach is to train an ensemble of classifiers by periodically choosing data points that cause maximal disagreement among them. Many classifiers with different underlying structures could fit this framework, but some classifiers are more suitable for different data sets than others. The question then arises as to how to find the most suitable classifier for a given data set. In this work, an evolutionary algorithm is proposed to address this problem. The algorithm starts with a combination of artificial neural networks and decision trees, and iteratively adapts the ratio of the classifier types according to a replacement strategy. Experiments with synthetic and real data sets show that when the algorithm considers both fitness and classifier type for replacement, the population becomes saturated with accurate instantiations of the more suitable classifier type. This allows the algorithm to perform consistently well across data sets, without having to determine a priori a suitable classifier type.

One common approach to active learning is to iteratively train a single classifier by choosing data points based on its uncertainty, but it is nontrivial to design uncertainty measures unbiased by the choice of classifier. Query by committee suggests that given an ensemble of diverse but accurate classifiers, the most informative data points are those that cause maximal disagreement among the predictions of the ensemble members. However the method for finding ensembles appropriate to a given data set remains an open question. In this paper, the random subspace method is combined with active learning to create multiple instances of different classifier types, and an algorithm is introduced that adapts the ratio of different classifier types in the ensemble towards better overall accuracy. Here we show that the proposed algorithm outperforms C4.5 with uncertainty sampling, Naive Bayes with uncertainty sampling, bagging, boosting and the random subspace method with random sampling. To the best of our knowledge, our work is the first to adapt the ratio of classifiers in a heterogeneous ensemble for active learning.

Pattern matching is fundamental in applications such as biological sequence analysis and text indexing. A wildcard gap matches any subsequence with a length between two user specified integers, therefore introducing much adaptability to patterns. However, most existing works require that gaps in a pattern be the same. In this paper, we define a new pattern matching problem where gaps are independently specified. The objective is to compute the number of all matches. Since this number isexponential with respect to the maximal gap flexibility and the pattern length, counting matches one by one is computationally infeasible. We develop an efficient algorithm, named Pattern mAtching with Independent wildcard Gaps (PAIG) for this problem, and propose two approaches to enhance its performance further. For the final version, the time complexity is O(Ll2W2), where L is the sequence length, l is the pattern length, and W is the maximal gap flexibility. The space complexity is O(lW), making PAIG easy to run in a Java Applet. Experimental results validate the efficiency of PAIG and confirm our analysis about its different versions.

Selective sampling is a form of active learning which can reduce the cost of training by only drawing informative data points into the training set. This selected training set is expected to contain more information for modeling compared to random sampling, thus making modeling faster and more accurate. We introduce a novel approach to selective sampling, which is derived from the Estimation-Exploration Algorithm (EEA). The EEA is a coevolutionary algorithm that uses model disagreement to determine the significance of a training datum, and evolves a set of models only on the selected data. The algorithm in this paper trains a population of Artificial Neural Networks (ANN) on the training set, and uses their disagreement to seek new data for the training set. A medical data set called the National Trauma Data Bank (NTDB) is used to test the algorithm. Experiments show that the algorithm outperforms the equivalent algorithm using randomly-selected data and sampling evenly from each class. Finally, the selected training data reveals which features most affect outcome, allowing for both improved modeling and understanding of the processes that gave rise to the data.