Supervised classification techniques are known to exploit physical information of the analysed data, such as similarity, distribution and other low level features. Despite the relevance of such features, recent works have showed that a higher variety of patterns can be detected by combining low level and high level features. In this paper, it is proposed a supervised classification technique which applies limiting probabilities of the random walk theory over underlying networks constructed from input labeled data. The appealing feature of the proposed approach is that the adjacency matrix which carries both physical and structural information about the data. Structural information are given by features extracted from network connections. The class of a given unlabeled sample is estimated by a heuristic called ease of access, which is measured by the random walk process over the adjacency matrix. Such approach makes the technique quite general as one can put distinct data measures of interest in the connection matrix of the underlying data network to guide the random walker. Specifically, we show examples of combining low and high level features in the proposed classification scheme. Simulation results using artificial and real data sets suggest that the proposed technique is not only competitive with current and established classification techniques, but it also can reveal intrinsic structural patterns formed by the input data. (C) 2017 Elsevier Ltd. All rights reserved.

Graph-based dimensionality reduction has attracted a lot of attention in recent years. Such methods aim to exploit the graph representation in order to catch some structural information hidden in data. They usually consist of two steps: graph construction and projection. Although graph construction is crucial to the performance, most research work in the literature has focused on the development of heuristics and models to the projection step, and only very recently, attention was paid to network construction. In this work, graph construction is considered in the context of supervised dimensionality reduction. To be specific, using a nature-inspired optimization framework, this work investigates if an optimized graph is able to provide better projections than well-known general-purpose methods. The proposed method is compared with widely used graph construction methods on a range of real-world image classification problems. Results show that the optimization framework has achieved considerable dimensionality reduction rates as well as good predictive performance.

Semantic Role Labeling (SRL) is a natural language processing task that detects the arguments of predicates (usually verbs) and their semantic roles. Such roles characterize semantic relationships between an event and its participants, as who did what to whom, where, when and how, which is very useful to improve a wide range of tasks, such as information extraction and plagiarism dectection to name a few. Commonly, a supervised classifier is trained over large English annotated resources in order to perform the prediction of unlabeled sentences. However, most part of non-English languages suffers from scarcity of annotated data, as the labeling process is expensive, time consuming and requires the efforts of human annotators. Although such limitation makes harder the training of supervised methods for those languages, it indicates an appropriate scenario to apply semi-supervised learning (SSL) methods, which are able to learn not only from labeled data, but also from the unlabeled ones. In this article, we investigate SSL methods in the classification of semantic roles for the Brazilian Portuguese, a relatively resource-poor language. Specifically, a representative set of SSL methods based on low density separation, graphs and self-training are considered. Experiments on the PropBank-br, a Brazilian Portuguese corpus built with text from Brazilian newspapers, were performed varying the number of labeled arguments. Additionaly, the SSL methods were compared against state-of-the-art SRL methods. The results demonstrated that self-training heuristic outperforms other SSL and supervised methods, even when the latter are trained on a high number of labeled arguments.

In this paper we propose a new technique for Semi-Supervised Learning based on attraction forces. The main idea behind the SSL paradigm is to perform a classification task taking into account a few labeled instances and the information provided by many unlabeled instances. Essentially, the proposed technique considers each data instance as dimensionless points on a n-dimensional space and performs their dynamics accordingly to the resultant forces. The labeled points act as fixed attraction points whereas the unlabeled ones move towards them, whereby the unlabed instances are labeled through a label propagation mechanism when they approximate a defined neighborhood region around a fixed attraction point. The technique mainly takes into account two important SSL assumptions: smoothness and cluster. The results obtained from simulations performed on artificial datasets exhibit the effectiveness of the proposed method.

We propose a new supervised classification technique which considers the ease of access of unlabeled instances to training classes through an underlying network. The training data set is used to construct a network, in which instances (nodes) represent the states that a random walker visits, and the network link structure is modified by performing a link weight composition between the unlabeled instance bias and the initial network link weights. Different from traditional classification heuristics, which divide the training data set into subspaces, the proposed scheme uses random walk limiting probabilities to measure the limiting state transitions among training nodes. An unlabeled instance receives the label of the class that is most easily reached by the random walker, that is, the limiting transition to that class is large. Simulation results suggest that the proposed technique is comparable to some well-known classification techniques.

In machine learning, dimensionality reduction aims at reducing the dimension of the input data in order to achieve a small set of features that keeps the most important original relationships among data samples. In this paper, we investigate the usage of a non-parametric network formation algorithm into a graph embedding framework to perform supervised dimensionality reduction. Specifically, our technique maps data into networks and constructs two network adjacency matrices which convey information about intra-class components and inter-class penalty connections. Both matrices are inserted into an optimization framework in order to achieve a projection vector that is used to project high-dimension data samples into a low-dimensional space. One advantage of the technique is that no parameter is required, that is, there is no need to select a model for the input data. Computer simulations on real-world data sets have been performed to compare the proposed technique to some classical network formation methods such as k-NN and e-radius, and to well-known dimensionality reduction algorithms such as PCA and LDA. Statistical tests have shown that our approach outperforms those algorithms.

We present a nature-inspired semi-supervised learning technique based on the flocking formation of certain living species like birds and fishes. Each data item is treated as an individual in the flock. Starting from random directions, each data item moves according to its surrounding items, by getting closer to them (but not too much close) and taking the same direction of motion. Labeled items play special roles, ensuring that data from different classes will belong to different, distant flocks. Experiments on both artificial and benchmark datasets were performed and show its classification accuracy. Despite the rich behavior, we argue that this technique has a sub-quadratic asymptotic time complexity, thus being feasible to be used on large datasets. In order to achieve such performance, a space-partitioning technique is introduced. We also argue that the richness behind this dynamic, self-organizing model is quite robust and may be used to do much more than simply propagating the labels from labeled to unlabeled data. It could be used to determine class overlapping, wrong labeling, etc.

Recently, many network-based methods have been developed and successfully applied to cluster data. Once the underlying network has been constructed, a clustering method can be applied over its vertices and edges. In this paper, the concept of pinning control in complex networks is applied to cluster data. Firstly, an adaptive method for constructing sparse and connected networks is proposed. Secondly, a dissimilarity measure is computed via a dynamic system in which vertices are expected to reach a consensus state regarding a reference trajectory. The reference is forced into the system by pinning control. A theoretical analysis was carried out to prove the convergence of the dynamic system under certain parameter constraints. The results using real data sets have showed that the proposed method performs well in the presence of clusters with different sizes and shapes comparing to some well-known clustering methods.

This paper deals with the classification of multiple pattern observations sets. A set of observations consists of different transformations, possibly including rotation, perspectives and projections. Each set belongs to a single pattern, that is, the pattern is considered invariant under such transformations. The method uses a network representation of the input data to take advantage of the topological relations between the patterns revealed by a low-dimensional manifold. A measurement called modularity is computed to numerically indicate the topological characteristics of the constructed networks. Simulations were carried out in real image data sets, and results have showed that the proposed method outperforms some recent and state-of-the-art techniques.

This paper presents a new network-based classification technique using limiting probabilities from random walk theory. Instead of using a traditional heuristic to classify data relying on physical features such as similarity or density distribution, it uses a concept called ease of access. By means of an underlying network, in which nodes represent states for the random walk process, unlabeled instances are classified with the label of the most easily reached class. The limiting probabilities are used as a measure for the ease of access by taking into account the biases provided by an unlabeled instance in a specific adjacency matrix weight composition. In this way, the technique allows data classification from a different viewpoint. Simulation results suggest that the proposed scheme is competitive with current and well-known classification algorithms.

In this paper, we study a new type of competitive learning scheme realized on large-scale networks. The model consists of several agents walking within the network and competing with each other to occupy as many nodes as possible, while attempting to reject intruder agents. In the end of the process, each agent dominates a community (a strongly connected subnetwork). Here, the model is described by a stochastic dynamical system. In this paper, a mathematical analysis for uncovering the system's properties is presented. In addition, the model is applied to solve handwritten digits and letters clustering problems. An interesting feature is that the model is able to group the same digits or letters even with considerable distortions into the same cluster. Computer simulations reveal that the proposed technique presents high precision of cluster detections, as well as low computational complexity.

In this paper, we aim to study the usage of different network formation methods into a graph embedding framework to perform supervised dimensionality reduction. Images are often high-dimensional patterns, and dimensionality reduction can enhance processing and also increase classification accuracy. Specifically, our technique maps images into networks and constructs two network adjacency matrices to convey information about intra-class components and inter-class penalty connections. Both matrices are inserted into an optimization framework in order to achieve a projection vector that is used to project high-dimension data samples into a low-dimensional space. One advantage of the technique is that no parameter is required, that is, there is no need to select a model for the input data. Applications on handwritten digits recognition are performed, and the proposed technique is compared to some classical network formation methods. Numerical results show the approach is promising.

Extracting useful knowledge from data sets is a key concept in modern information systems. Consequently, the need of efficient techniques to extract the desired knowledge has been growing over time. Machine learning is a research field dedicated to the development of techniques capable of enabling a machine to "learn" from data. Many techniques have been proposed so far, but there are still issues to be unveiled specially in interdisciplinary research. In this thesis, we explore the advantages of network data representation to develop machine learning techniques based on dynamical processes on networks. The network representation unifies the structure, dynamics and functions of the system it represents, and thus is capable of capturing the spatial, topological and functional relations of the data sets under analysis. We develop network-based techniques for the three machine learning paradigms: supervised, semi-supervised and unsupervised. The random walk dynamical process is used to characterize the access of unlabeled data to data classes, configuring a new heuristic we call ease of access in the supervised paradigm. We also propose a classification technique which combines the high-level view of the data, via network topological characterization, and the low-level relations, via similarity measures, in a general framework. Still in the supervised setting, the modularity and Katz centrality network measures are applied to classify multiple observation sets, and an evolving network construction method is applied to the dimensionality reduction problem. The semi-supervised paradigm is covered by extending the ease of access heuristic to the cases in which just a few labeled data samples and many unlabeled samples are available. A semi-supervised technique based on interacting forces is also proposed, for which we provide parameter heuristics and stability analysis via a Lyapunov function. Finally, an unsupervised network-based technique uses the concepts of pinning control and consensus time from dynamical processes to derive a similarity measure used to cluster data. The data is represented by a connected and sparse network in which nodes are dynamical elements. Simulations on benchmark data sets and comparisons to well-known machine learning techniques are provided for all proposed techniques. Advantages of network data representation and dynamical processes for machine learning are highlighted in all cases.

Semi-supervised learning is a classification paradigm in which just a few labeled instances are available for the training process. To overcome this small amount of initial label information, the information provided by the unlabeled instances is also considered. In this paper, we propose a nature-inspired semi-supervised learning technique based on attraction forces. Instances are represented as points in a k-dimensional space, and the movement of data points is modeled as a dynamical system. As the system runs, data items with the same label cooperate with each other, and data items with different labels compete among them to attract unlabeled points by applying a specific force function. In this way, all unlabeled data items can be classified when the system reaches its stable state. Stability analysis for the proposed dynamical system is performed and some heuristics are proposed for parameter setting. Simulation results show that the proposed technique achieves good classification results on artificial data sets and is comparable to well-known semi-supervised techniques using benchmark data sets.

The semi-supervised learning paradigm allows that a large amount of unlabeled data be classified using just a few labeled data. To account for the minimal a priori label knowledge, the information provided by the unlabeled data is also used in the classification process. This paper describes a semi-supervised technique that uses random walk limiting probabilities to propagate label information. Each label is propagated through a network of unlabeled instances via a biased random walk. The probability of a vertex receiving a label is expressed in terms of the limiting conditions of the walk process. Simulations show that the proposed technique is competitive with benchmarked techniques.

Semi-Supervised Learning (SSL) is a learning paradigm in which the classification task is performed by taking into account just a few labeled instances. The unlabeled instances also participate in the process, but by providing additional information about the dataset. In this paper, a new semi-supervised technique based on interacting forces is proposed. Both labeled and unlabeled instances play different roles in the proposed mechanism: the labeled instances perform attraction forces over the unlabeled instances to accomplish label propagation. Inside a defined neighborhood, a label in able to propagates to an unlabeled instance. The technique mainly takes into account two important SSL assumptions: smoothness and cluster. Results obtained from simulations performed on artificial and real datasets exhibit the effectiveness of the proposed method.

Among the many machine learning methods developed for classification tasks, the network-based learning algorithms made great success. Usually, these methods consist of two stages: the construction of a network from the original vector-based data set and the learning in the constructed network. In this paper, a network concept, called vertex centrality, is used to perform pattern classification. A group of multiple invariant transformations of a same pattern is given and the network classifier must predict the pattern class the group belongs to. The prediction is based on the Katz centrality network measurement. Due to the ability of characterizing topological structure of input patterns, the method has been shown very competitive comparing to some state-of-the-art methods.

Traditional data classification techniques consider only physical features of input data in order to construct their hypotheses. On the other hand, the human (animal) brain performs both low and high order learning and it has facility to identify patterns according to the semantic meaning of input data. In this paper, we propose a data classification technique by combining the low level and the high level learning. The low level term can be implemented by any classification technique, while the high level classification is realized by the extraction of features of the underlying network constructed from the input data. Thus, the former classifies data instances by their physical features, while the latter measures the compliance to the pattern formation of the data. Our study shows that the proposed technique can not only realize classification according to the pattern formation, but it is also able to improve the performance of traditional classification techniques. An application on handwritten digits recognition is performed, revealing that higher classification rates can be obtained when we have a proper mixture of low and high level classifiers.

Competitive learning is an important mechanism for data clustering and pattern recognition. In this paper, we present a rigorous definition of a new type of competitive learning scheme realized on large scale networks. In this model, several particles walk in the network and compete with each other to occupy as many nodes as possible, while attempting to reject intruder particles. As a result, each particle will dominate a cluster of the network. Moreover, we propose an efficient method for determining the right number of clusters by using the information generated by the competition process itself, avoiding the calculation of an external evaluating index. In this work, we apply the model to handwritten data clustering. Computer simulations reveal that the proposed technique obtains satisfactory cluster detection accuracy.