One of the most powerful, popular and accurate classification techniques is support vector machines (SVMs). In this work, we want to evaluate whether the accuracy of SVMs can be further improved using training set selection (TSS), where only a subset of training instances is used to build the SVM model. By contrast to existing approaches, we focus on wrapper TSS techniques, where candidate subsets of training instances are evaluated using the SVM training accuracy. We consider five wrapper TSS strategies and show that those based on evolutionary approaches can significantly improve the accuracy of SVMs. (C) 2015 Elsevier B.V. All rights reserved.

One of the most known and effective methods in supervised classification is the k-nearest neighbors classifier. Several approaches have been proposed to enhance its precision, with the fuzzy k-nearest neighbors (fuzzy-kNN) classifier being among the most successful ones. However, despite its good behavior, fuzzy-kNN lacks of a method for properly defining several mechanisms regarding the representation of the relationship between the instances and the classes of the classification problems. Such a method would be very desirable, since it would potentially lead to an improvement in the precision of the classifier.In this work we present a new approach, evolutionary fuzzy k-nearest neighbors classifier using interval-valued fuzzy sets (EF-kNN-IVFS), incorporating interval-valued fuzzy sets for computing the memberships of training instances in fuzzy-kNN. It is based on the representation of multiple choices of two key parameters of fuzzy-kNN: one is applied in the definition of the membership function, and the other is used in the computation of the voting rule. Besides, evolutionary search techniques are incorporated to the model as a self-optimization procedure for setting up these parameters. An experimental study has been carried out to assess the capabilities of our approach. The study has been validated by using nonparametric statistical tests, and remarks the strong performance of EF-kNN-IVFS compared with several state of the art techniques in fuzzy nearest neighbor classification. (C) 2015 Elsevier Inc. All rights reserved.

Nonparametric statistical tests are useful procedures that can be applied in a wide range of situations, such as testing randomness or goodness of fit, one-sample, two-sample and multiple-sample analysis, association between bivariate samples or count data analysis. Their use is often preferred to parametric tests due to the fact that they require less restrictive assumptions about the population sampled. In this work, JavaNPST, an open source Java library implementing 40 nonparametric statistical tests, is presented. It can be helpful for programmers and practitioners interested in performing nonparametric statistical analyses, providing a quick and easy way of running these tests directly within any Java code. Some examples of use are also shown, highlighting some of the more remarkable capabilities of the library.

Several recent approaches use distributional similarity for making symbolic reasoning more flexible. While an important step in the right direction, the use of similarity has a number of inherent limitations. We argue that similarity-based reasoning should be complemented with commonsense reasoning patterns such as interpolation and a fortiori inference. We show how the required background knowledge for these inference patterns can be obtained from distributional models.

This paper reviews the fingerprint classification literature looking at the problem from a double perspective. We first deal with feature extraction methods, including the different models considered for singular point detection and for orientation map extraction. Then, we focus on the different learning models considered to build the classifiers used to label new fingerprints. Taxonomies and classifications for the feature extraction, singular point detection, orientation extraction and learning methods are presented. A critical view of the existing literature have led us to present a discussion on the existing methods and their drawbacks such as difficulty in their reimplementation, lack of details or major differences in their evaluations procedures. On this account, an experimental analysis of the most relevant methods is carried out in the second part of this paper, and a new method based on their combination is presented.

The K-Nearest Neighbors (k-NN) classifier has become a well-known, successful method for pattern classification tasks. In recent years, many enhancements to the original algorithm have been proposed. Fuzzy sets theory has been the basis of several proposed models towards the enhancement of the nearest neighbors rule, being the Fuzzy K-Nearest Neighbors (FuzzyKNN) classifier the most notable procedure in the field.In this work we present a new approach to the nearest neighbor classifier based on the use of interval valued fuzzy sets. The use and implementation of interval values facilitates the membership of the instances and the computation of the votes in a more flexible way than the original FuzzyKNN method, thus improving its adaptability to different supervised learning problems. An experimental study, contrasted by the application of nonparametric statistical procedures, is carried out to ascertain whether the Interval Valued K-Nearest Neighbor (IV-KNN) classifier proposed here is significantly more accurate than k-NN, FuzzyKNN and other fuzzy nearest neighbor classifiers. We conclude that the IV-KNN is indeed significantly more accurate than the rest of classifiers analyzed.

Commonsense reasoning patterns such as interpolation and a fortiori inference have proven useful for dealing with gaps in structured knowledge bases. An important difficulty in applying these reasoning patterns in practice is that they rely on fine-grained knowledge of how different concepts and entities are semantically related. In this paper, we show how the required semantic relations can be learned from a large collection of text documents. To this end, we first induce a conceptual space from the text documents, using multi-dimensional scaling. We then rely on the key insight that the required semantic relations correspond to qualitative spatial relations in this conceptual space. Among others, in an entirely unsupervised way, we identify salient directions in the conceptual space which correspond to interpretable relative properties such as ‘more fruity than’ (in a space of wines), resulting in a symbolic and interpretable representation of the conceptual space. To evaluate the quality of our semantic relations, we show how they can be exploited by a number of commonsense reasoning based classifiers. We experimentally show that these classifiers can outperform standard approaches, while being able to provide intuitive explanations of classification decisions. A number of crowdsourcing experiments provide further insights into the nature of the extracted semantic relations.

Place types taxonomies tend to have a shallow structure, which limits their predictive value. Although existing place type taxonomies could in principle be refined, the result would inevitably be highly subjective and application-specific. Instead, in this paper, we propose a methodology to enrich place types taxonomies with a ternary betweenness relation derived from Flickr. In particular, we first construct a semantic space of place types by applying dimensionality reduction methods to tag co-occurrence data obtained from Flickr. Our hypothesis is that natural properties of place types should correspond to convex regions in this space. Specifically, knowing that places P 1,...,P n have a given property, we could then induce that all places which are located in the convex hull of {P 1,...,P n } in the semantic space are also likely to have this property. To avoid relying on computationally expensive convex hull algorithms, we propose to derive a ternary betweenness relation from the semantic space, and to approximate the convex hull at the symbolic level based on this relation. We present experimental results which support the usefulness of our approach.

Statistical validation of results for supporting the conclusions achieved in an experimental study is more and more demanded in research results. Although statistics are usually used in the analysis of results for comparing the performance of several algorithms, they could be used in other tasks, such as proper selection of parameters's value or study of the trend of a parameter. In this short paper, we describe a non-parametric test, the Page test, which can be used for predicting the order of experimental conditions. We include an illustrative example for using it on classification problems taking the well-known k-nearest neighbour algorithm.

The analysis of the performance of different approaches is a staple concern in the design of Computational Intelligence experiments. Any proper analysis of evolutionary optimization algorithms should incorporate a full set of benchmark problems and state-of-the-art comparison algorithms. For the sake of rigor, such an analysis may be completed with the use of statistical procedures, supporting the conclusions drawn. In this paper, we point out that these conclusions are usually limited to the final results, whereas intermediate results are seldom considered. We propose a new methodology for comparing evolutionary algorithms’ convergence capabilities, based on the use of Page’s trend test. The methodology is presented with a case of use, incorporating real results from selected techniques of a recent special issue. The possible applications of the method are highlighted, particularly in those cases in which the final results do not enable a clear evaluation of the differences among several evolutionary techniques.

Various applications, such as critique-based recommendation systems and analogical classifiers, rely on knowledge of how different entities relate. In this paper, we present a methodology for identifying such semantic relationships, by interpreting them as qualitative spatial relations in a conceptual space. In particular, we use multi-dimensional scaling to induce a conceptual space from a relevant text corpus and then identify directions that correspond to relative properties such as "more violent than" in an entirely unsupervised way. We also show how a variant of FOIL is able to learn natural categories from such qualitative representations, by simulating a fortiori inference, an important pattern of commonsense reasoning.

The Nearest Neighbor rule is one of the most successful classifiers in machine learning but it is very sensitive to noisy data, which may cause its performance to deteriorate. This contribution proposes a new feature weighting classifier that tries to reduce the influence of noisy features. The computation of the weights is based on combining imputation methods and non-parametrical statistical tests. The results obtained show that our proposal can improve the performance of the Nearest Neighbor classifier dealing with different types of noisy data.

In recent years, many nearest neighbor algorithms based on fuzzy sets theory have been developed. These methods form a field, known as fuzzy nearest neighbor classification, which is the source of many proposals for the enhancement of the k nearest neighbor classifier. Fuzzy sets theory and several extensions, including fuzzy rough sets, intuitionistic fuzzy sets, type-2 fuzzy sets and possibilistic theory are the foundations of these hybrid techniques, designed to tackle some of the drawbacks of the nearest neighbor rule.In this paper the most relevant approaches to fuzzy nearest neighbor classification are reviewed, as are applications and theoretical works. Several descriptive properties are defined to build a full taxonomy, which should be useful as a future reference for new developments. An experimental framework, including implementations of the methods, datasets, and a suggestion of a statistical methodology for results assessment is provided. A case of study is included, featuring a comparison of the best techniques with several state of the art crisp nearest neighbor classifiers. The work concludes with the suggestion of some open challenges and ways to improve fuzzy nearest neighbor classification as a machine learning technique.

The Nearest Neighbor rule is one of the most successful classifiers in machine learning. However, it is very sensitive to noisy, redundant and irrelevant features, which may cause its performance to deteriorate. Feature weighting methods try to overcome this problem by incorporating weights into the similarity function to increase or reduce the importance of each feature, according to how they behave in the classification task. This paper proposes a new feature weighting classifier, in which the computation of the weights is based on a novel idea combining imputation methods – used to estimate a new distribution of values for each feature based on the rest of the data – and the Kolmogorov–Smirnov nonparametric statistical test to measure the changes between the original and imputed distribution of values. This proposal is compared with classic and recent feature weighting methods. The experimental results show that our feature weighting scheme is very resilient to the choice of imputation method and is an effective way of improving the performance of the Nearest Neighbor classifier, outperforming the rest of the classifiers considered in the comparisons.

This paper presents a Genetic algorithm based association rule mining in which multi fitness functions are used. Genetic algorithm is used for performing global search. This proposed algorithm generates intersecting association rules from dataset. A fitness function with parameter support is defined for generating frequent itemsets and then other parameters like confidence, lift, leverage etc are used for defining second fitness function for generating association rules. The proposed algorithm is compared with classical Apriori algorithm and also with existing Genetic algorithm for association rule mining on the basis of metrics Support Count, Confidence count, and rule accuracy. Comparisons are also made on different generations.

Typically, comparisons among optimization algorithms only considers the results obtained at the end of the search process. However, there are occasions in which is very interesting to perform comparisons along the search. This way, algorithms could also be categorized depending on its convergence performance, which would help when deciding which algorithms perform better among a set of methods that are assumed as equal when only the results at the end of the search are considered. In this work, we present a procedure to perform a pairwise comparison of two algorithms' convergence performance. A non-parametric procedure, the Page test, is used to detect significant differences between the evolution of the error of the algorithms as the search continues. A case of study has been also provided to demonstrate the application of the test.

The nearest neighbor classifier is one of the most used and well-known techniques for performing recognition tasks. It has also demonstrated itself to be one of the most useful algorithms in data mining in spite of its simplicity. However, the nearest neighbor classifier suffers from several drawbacks such as high storage requirements, low efficiency in classification response, and low noise tolerance. These weaknesses have been the subject of study for many researchers and many solutions have been proposed. Among them, one of the most promising solutions consists of reducing the data used for establishing a classification rule (training data) by means of selecting relevant prototypes. Many prototype selection methods exist in the literature and the research in this area is still advancing. Different properties could be observed in the definition of them, but no formal categorization has been established yet. This paper provides a survey of the prototype selection methods proposed in the literature from a theoretical and empirical point of view. Considering a theoretical point of view, we propose a taxonomy based on the main characteristics presented in prototype selection and we analyze their advantages and drawbacks. Empirically, we conduct an experimental study involving different sizes of data sets for measuring their performance in terms of accuracy, reduction capabilities, and runtime. The results obtained by all the methods studied have been verified by nonparametric statistical tests. Several remarks, guidelines, and recommendations are made for the use of prototype selection for nearest neighbor classification.

Prototype generation techniques have arisen as very competitive methods for enhancing the nearest neighbor classifier through data reduction. Within the prototype generation methodology, the methods of adjusting the prototypes' positioning have shown an outstanding performance. Evolutionary algorithms have been used to optimize the positioning of the prototypes with promising results. However, these results can be improved even more if other data reduction techniques, such as prototype selection and feature weighting, are considered. In this paper, we propose a hybrid evolutionary scheme for data reduction, incorporating a new feature weighting scheme within two different prototype generation methodologies. Specifically, we will focus on a self-adaptive differential evolution algorithm in order to optimize feature weights and the placement of the prototypes. The results are contrasted with nonparametric statistical tests, showing that our proposal outperforms previously proposed methods, thus showing itself to be a suitable tool in the task of enhancing the performance of the nearest neighbor classifier.

Cooperative coevolution is a successful trend of evolutionary computation which allows us to define partitions of the domain of a given problem, or to integrate several related techniques into one, by the use of evolutionary algorithms. It is possible to apply it to the development of advanced classification methods, which integrate several machine learning techniques into a single proposal. A novel approach integrating instance selection, instance weighting, and feature weighting into the framework of a coevolutionary model is presented in this paper. We compare it with a wide range of evolutionary and nonevolutionary related methods, in order to show the benefits of the employment of coevolution to apply the techniques considered simultaneously. The results obtained, contrasted through nonparametric statistical tests, show that our proposal outperforms other methods in the comparison, thus becoming a suitable tool in the task of enhancing the nearest neighbor classifier.