338
title: Explore/Exploit Schemes for Web Content Optimization
author: Deepak Agarwal,Bee-Chung Chen,Pradheep Elango
abstract: We propose novel multi-armed bandit (explore/exploit) schemes to maximize total clicks on a content module published regularly on Yahoo! Intuitively one can ``explore'' each candidate item by displaying it to a small fraction of user visits to estimate the item's click-through rate (CTR) and then ``exploit'' high CTR items in order to maximize clicks. While bandit methods that seek to find the optimal trade-off between explore and exploit have been studied for decades existing solutions are not satisfactory for web content publishing applications where dynamic set of items with short lifetimes delayed feedback and non-stationary reward (CTR) distributions are typical. In this paper we develop a Bayesian solution and extend several existing schemes to our setting. Through extensive evaluation with nine bandit schemes we show that our Bayesian solution is uniformly better in several scenarios. We also study the empirical characteristics of our schemes and provide useful insights on the strengths and weaknesses of each. Finally we validate our results with a ``side-by-side'' comparison of schemes through live experiments conducted on a random sample of real user visits to Yahoo!
title: Connecting Sparsely Distributed Similar Bloggers
author: Nitin Agarwal,Huan Liu,Shankara Subramanya,John J. Salerno,Philip S Yu
abstract: The nature of the Blogosphere determines that the majority of bloggers are only connected with a small number of fellow bloggers and similar bloggers can be largely disconnected from each other. Aggregating them allows for cost-effective personalized services targeted marketing and exploration of new business opportunities. As most bloggers have only a small number of adjacent bloggers the problem of aggregating similar bloggers presents challenges that demand novel algorithms of connecting the non-adjacent due to the fragmented distributions of bloggers. In this work we define the problem delineate its challenges and present an approach that uses innovative ways to employ contextual information and collective wisdom to aggregate similar bloggers. A real-world blog directory is used for experiments. We demonstrate the efficacy of our approach report findings and discuss related issues and future work.
title: Rule Ensembles for Multi-target Regression
author: Timo Aho,Bernard ?enko,Sa?o D?eroski
abstract: Methods for learning decision rules are being successfully applied to many problem domains especially where understanding and interpretation of the learned model is necessary. In many real life problems we would like to predict multiple related (nominal or numeric) target attributes simultaneously. Methods for learning rules that predict multiple targets at once already exist but are unfortunately based on the covering algorithm which is not very well suited for regression problems. A better solution for regression problems may be a rule ensemble approach that transcribes an ensemble of decision trees into a large collection of rules. An optimization procedure is then used for selecting the best (and much smaller) subset of these rules and to determine their weights. Using the rule ensembles approach we have developed a new system for learning rule ensembles for multi-target regression problems. The newly developed method was extensively evaluated and the results show that the accuracy of multi-target regression rule ensembles is better than the accuracy of multi-target regression trees but somewhat worse than the accuracy of multi-target random forests. The rules are significantly more concise than random forests and it is also possible to create very small rule sets that are still comparable in accuracy to single regression trees.
title: A Local Scalable Distributed Expectation Maximization Algorithm for Large Peer-to-Peer Networks
author: Kanishka Bhaduri,Ashok N Srivastava
abstract: This paper describes a local and distributed expectation maximization algorithm for learning parameters of Gaussian mixture models (GMM) in large peer-to-peer (P2P) environments. The algorithm can be used for a variety of well-known data mining tasks in distributed environments such as clustering anomaly detection target tracking and density estimation to name a few necessary for many emerging P2P applications in bioinformatics webmining and sensor networks. Centralizing all or some of the data to build global models is impractical in such P2P environments because of the large number of data sources the asynchronous nature of the P2P networks and dynamic nature of the data/network. The proposed algorithm takes a two-step approach. In the monitoring phase the algorithm checks if the model quality is acceptable by using an efficient local algorithm. This is then used as a feedback loop to sample data from the network and rebuild the GMM when it is outdated. We present thorough experimental results to verify our theoretical claims.
title: Cross-Guided Clustering,Transfer of Relevant Supervision across Domains for Improved Clustering
author: Indrajit Bhattacharya,Shantanu Godbole,Sachindra Joshi,Ashish Verma
abstract: Lack of supervision in clustering algorithms often leads to clusters that are not useful or interesting to human reviewers. We investigate if supervision can be automatically transferred to a clustering task in a target domain by providing a relevant supervised partitioning of a dataset from a different source domain. The target clustering is made more meaningful for the human user by trading off intrinsic clustering goodness on the target dataset for alignment with relevant supervised partitions in the source dataset wherever possible. We propose a cross-guided clustering algorithm that builds on traditional k-means by aligning the target clusters with source partitions. The alignment process makes use of a cross-domain similarity measure that discovers hidden relationships across domains with potentially different vocabularies. Using multiple real-world datasets we show that our approach improves clustering accuracy significantly over traditional k-means.
title: Audio Classification of Bird Species,A Statistical Manifold Approach
author: Forrest Briggs,Raviv Raich,Xiaoli Z Fern
abstract: Our goal is to automatically identify which species of bird is present in an audio recording using supervised learning. Devising effective algorithms for bird species classification is a preliminary step toward extracting useful ecological data from recordings collected in the field. We propose a probabilistic model for audio features within a short interval of time then derive its Bayes risk-minimizing classifier and show that it is closely approximated by a nearest-neighbor classifier using Kullback-Leibler divergence to compare histograms of features. We note that feature histograms can be viewed as points on a statistical manifold and KL divergence approximates geodesic distances defined by the Fisher information metric on such manifolds. Motivated by this fact we propose the use of another approximation to the Fisher information metric namely the Hellinger metric. The proposed classifiers achieve over 90% accuracy on a data set containing six species of bird and outperform support vector machines.
title: Finding Associations and Computing Similarity via Biased Pair Sampling
author: Andrea Campagna,Rasmus Pagh
abstract: Sampling-based methods have previously been proposed for the problem of finding interesting associations in data even for low-support items. While these methods do not guarantee precise results they can be vastly more efficient than approaches that rely on exact counting. However for many similarity measures no such methods have been known. In this paper we show how a wide variety of measures can be supported by a simple biased sampling method. The method also extends to find high-confidence association rules. We demonstrate theoretically that our method is superior to exact methods when the threshold for "interesting similarity/confidence" is above the average pairwise similarity/confidence and the average support is not too low. Our method is particularly good when transactions contain many items. We confirm in experiments on standard association mining benchmarks that this gives a significant speedup on real data sets (sometimes much larger than the theoretical guarantees). Reductions in computation time of over an order of magnitude and significant savings in space are observed.
title: Beyond Banditron,A Conservative and Efficient Reduction for Online Multiclass Prediction with Bandit Setting Model
author: Guangyun Chen,Gang Chen,Jianwen Zhang,Shuo Chen,Changshui Zhang
abstract: In this paper we consider a recently proposed supervised learning problem called online multiclass prediction with bandit setting model. Aiming at learning from partial feedback of online classification results i.e. true when the predicting label is right or false when the predicting label is wrong this new kind of problems arouses much of researchers interest due to its close relations to real world internet applications and human cognitive procedure. While some algorithms have been brought forward we propose a novel algorithm to deal with such problems. First we reduce the multiclass prediction problem to binary based on Conservative one-versus-all others Reduction scheme,Then Online Passive-Aggressive Algorithm is embedded as binary learning algorithm to solve the reduced problem. Also we derive a pleasing cumulative mistake bound for our algorithm and a time complexity bound linear to the sample size. Further experimental evaluation on several real world multiclass datasets including RCV1 MNIST 20 Newsgroups and USPS shows that our method outperforms the existing algorithms with a great improvement.
title: Probabilistic Similarity Query on Dimension Incomplete Data
author: Wei Cheng,Xiaoming Jin,Jian-Tao Sun
abstract: Retrieving similar data has drawn many research efforts in the literature due to its importance in data mining database and information retrieval. This problem is challenging when the data is incomplete. In previous research data incompleteness refers to the fact that data values for some dimensions are unknown. However in many practical applications (e.g. data collection by sensor network under bad environment) not only data values but even data dimension information may also be missing which will make most similarity query algorithms infeasible. In this work we propose the novel similarity query problem on dimension incomplete data and adopt a probabilistic framework to model this problem. For this problem users can give a distance threshold and a probability threshold to specify their retrieval requirements. The distance threshold is used to specify the allowed distance between query and data objects and the probability threshold is used to require that the retrieval results satisfy the distance condition at least with the given probability. Instead of enumerating all possible cases to recover the missed dimensions we propose an efficient approach to speed up the retrieval process by leveraging the inherent relations between query and dimension incomplete data objects. During the query process we estimate the lower/upper bounds of the probability that the query is satisfied by a given data object and utilize these bounds to filter irrelevant data objects efficiently. Furthermore a probability triangle inequality is proposed to further speed up query processing. According to our experiments on real data sets the proposed similarity query method is verified to be effective and efficient on dimension incomplete data.
title: flowNet,Flow-Based Approach for Efficient Analysis of Complex Biological Networks
author: Young-Rae Cho,Lei Shi,Aidong Zhang
abstract: Biological networks having complex connectivity have been widely studied recently. By characterizing their inherent and structural behaviors in a topological perspective these studies have attempted to discover hidden knowledge in the systems. However even though various algorithms with graph-theoretical modeling have provided fundamentals in the network analysis the availability of practical approaches to efficiently handle the complexity has been limited. In this paper we present a novel flow-based approach called flowNet to efficiently analyze large-sized complex networks. Our approach is based on the functional influence model that quantifies the influence of a biological component on another. We introduce a dynamic flow simulation algorithm to generate a flow pattern which is a unique characteristic for each component. The set of patterns can be used in identifying functional modules (i.e. clustering). The proposed flow simulation algorithm runs very efficiently in sparse networks. Since our approach uses a weighted network as an input we also discuss supervised and unsupervised weighting schemes for unweighted biological networks. As experimental results in real applications to the yeast protein interaction network we demonstrate that our approach outperforms previous graph clustering methods with respect to accuracy.
title: ?-Anomica,A Fast Support Vector Based Novelty Detection Technique
author: Santanu Das,Kanishka Bhaduri,Nikunj C. Oza,Ashok N Srivastava
abstract: In this paper we propose -Anomica a novel anomaly detection technique that can be trained on huge data sets with much reduced running time compared to the benchmark one-class Support Vector Machines algorithm. In -Anomica the idea is to train the machine such that it can provide a close approximation to the exact decision plane using fewer training points and without losing much of the generalization performance of the classical approach. We have tested the proposed algorithm on a variety of continuous data sets under different conditions. We show that under all test conditions the developed procedure closely preserves the accuracy of standard one- class Support Vector Machines while reducing both the training time and the test time by 5 ? 20 times.
title: Temporal Neighborhood Discovery Using Markov Models
author: Sandipan Dey,Vandana P. Janeja,Aryya Gangopadhyay
abstract: Temporal data which is a sequence of data tuples measured at successive time instances is typically very large. Hence instead of mining the entire data we are interested in dividing the huge data into several smaller intervals of interest which we call temporal neighborhoods. In this paper we propose an approach to generate temporal neighborhoods through unequal depth discretization. We describe two novel algorithms (a) Similarity based Merging (SMerg) and (b) Stationary distribution based Merging (StMerg). These algorithms are based on the robust framework of Markov models and the Markov Stationary distribution respectively. We identify temporal neighborhoods with distinct demarcations based on unequal depth discretization of the data. We discuss detailed experimental results in both synthetic and real world data. Specifically we show (i) the efficacy of our approach through precision and recall of labeled bins (ii) the ground truth validation in real world datasets and (iii) knowledge discovery in the temporal neighborhoods such as global anomalies. Our results indicate that we are able to identify valuable knowledge based on our ground truth validation from real world traffic data.
title: Active Learning with Generalized Queries
author: Jun Du,Charles X Ling
abstract: Active learning can actively select or construct examples to label to reduce the number of labeled examples needed for building accurate classifiers. However previous works of active learning can only ask specific queries. For example to predict osteoarthritis from a patient dataset with 30 attributes specific queries always contain values of all these 30 attributes many of which may be irrelevant. A more natural way is to ask "generalized queries" with don't-care attributes such as "are people over 50 with knee pain likely to have osteoarthritis?" (with only two attributes,age and type of pain). We assume that the oracle (and human experts) can readily answer those generalized queries by returning probabilistic labels. The power of such generalized queries is that one generalized query may be equivalent to many specific ones. However overly general queries may receive highly uncertain labels from the oracle and this makes learning difficult. In this paper we propose a novel active learning algorithm that asks generalized queries. We demonstrate experimentally that our new method asks significantly fewer queries compared with the previous works of active learning. Our method can be readily deployed in real-world tasks where obtaining labeled examples is costly.
title: Conditional Models for Non-smooth Ranking Loss Functions
author: Avinava Dubey,Jinesh Machchhar,Chiranjib Bhattacharyya,Soumen Chakrabarti
abstract: Learning to rank is an important area at the interface of machine learning information retrieval and Web search. The central challenge in optimizing various measures of ranking loss is that the objectives tend to be non-convex and discontinuous. To make such functions amenable to gradient based optimization procedures one needs to design clever bounds. In recent years boosting neural networks support vector machines and many other techniques have been applied. However there is little work on directly modeling a conditional probability Pr(y|x_q) where y is a permutation of the documents to be ranked and x_q represents their feature vectors with respect to a query q. A major reason is that the space of y is huge,n! if n documents must be ranked. We first propose an intuitive and appealing expected loss minimization objective and give an efficient shortcut to evaluate it despite the huge space of ys. Unfortunately the optimization is non-convex so we propose a convex approximation. We give a new efficient Monte Carlo sampling method to compute the objective and gradient of this approximation which can then be used in a quasi-Newton optimizer like LBFGS. Extensive experiments with the widely-used LETOR dataset show large ranking accuracy improvements beyond recent and competitive algorithms.
title: Unsupervised Class Separation of Multivariate Data through Cumulative Variance-Based Ranking
author: Andrew Foss,Osmar R. Za?ane,Sandra Zilles
abstract: This paper introduces a new extension of outlier detection approaches and a new concept class separation through variance. We show that accumulating information about the outlierness of points in multiple subspaces leads to a ranking in which classes with differing variance naturally tend to separate. Exploiting this leads to a highly effective and efficient unsupervised class separation approach especially useful in the difficult case of heavily overlapping distributions. Unlike typical outlier detection algorithms this method can be applied beyond the `rare classes' case with great success. Two novel algorithms that implement this approach are provided. Additionally experiments show that the novel methods typically outperform other state-of-the-art outlier detection methods on high dimensional data such as Feature Bagging SOE1 LOF ORCA and Robust Mahalanobis Distance and competes even with the leading supervised classification methods.
title: Execution Anomaly Detection in Distributed Systems through Unstructured Log Analysis
author: Qiang Fu,Jian-Guang Lou,Yi Wang,Jiang Li
abstract: Detection of execution anomalies is very important for the maintenance development and performance refinement of large scale distributed systems. Execution anomalies include both work flow errors and low performance problems. People often use system logs produced by distributed systems for troubleshooting and problem diagnosis. However manually inspecting system logs to detect anomalies is unfeasible due to the increasing scale and complexity of distributed systems. Therefore there is a great demand for automatic anomalies detection techniques based on log analysis. In this paper we propose an unstructured log analysis technique for anomalies detection. In the technique we propose a novel algorithm to convert free form text messages in log files to log keys without heavily relying on application specific knowledge. The log keys correspond to the log-print statements in the source code which can provide cues of system execution behavior. After converting log messages to log keys we learn a Finite State Automaton (FSA) from training log sequences to present the normal work flow for each system component. At the same time a performance measurement model is learned to characterize the normal execution performance based on the log mes-sages timing information. With these learned models we can automatically detect anomalies in newly input log files. Experiments on Hadoop and SILK show that the technique can effectively detect running anomalies.
title: Learning the Shared Subspace for Multi-task Clustering and Transductive Transfer Classification
author: Quanquan Gu,Jie Zhou
abstract: There are many clustering tasks which are closely related in the real world e.g. clustering the web pages of different universities. However existing clustering approaches neglect the underlying relation and treat these clustering tasks either individually or simply together. In this paper we will study a novel clustering paradigm namely multi-task clustering which performs multiple related clustering tasks together and utilizes the relation of these tasks to enhance the clustering performance. We aim to learn a subspace shared by all the tasks through which the knowledge of the tasks can be transferred to each other. The objective of our approach consists of two parts,(1) Within-task clustering,clustering the data of each task in its input space individually,and (2) Cross-task clustering,simultaneous learning the shared subspace and clustering the data of all the tasks together. We will show that it can be solved by alternating minimization and its convergence is theoretically guaranteed. Furthermore we will show that given the labels of one task our multi-task clustering method can be extended to transductive transfer classification (a.k.a. cross-domain classification domain adaption). Experiments on several cross-domain text data sets demonstrate that the proposed multi-task clustering outperforms traditional single-task clustering methods greatly. And the transductive transfer classification method is comparable to or even better than several existing transductive transfer classification approaches.
title: Accurate Estimation of the Degree Distribution of Private Networks
author: Michael Hay,Chao Li,Gerome Miklau,David Jensen
abstract: We describe an efficient algorithm for releasing a provably private estimate of the degree distribution of a network. The algorithm satisfies a rigorous property of differential privacy and is also extremely efficient running on networks of 100 million nodes in a few seconds. Theoretical analysis shows that the error scales linearly with the number of unique degrees whereas the error of conventional techniques scales linearly with the number of nodes. We complement the theoretical analysis with a thorough empirical analysis on real and synthetic graphs showing that the algorithm's variance and bias is low that the error diminishes as the size of the input graph increases and that common analyses like fitting a power-law can be carried out very accurately
title: A Linear-Time Graph Kernel
author: Shohei Hido,Hisashi Kashima
abstract: The design of a good kernel is fundamental for knowledge discovery from graph-structured data. Existing graph kernels exploit only limited information about the graph structures but are still computationally expensive. We propose a novel graph kernel based on the structural characteristics of graphs. The key is to represent node labels as binary arrays and characterize each node using logical operations on the label set of the connected nodes. Our kernel has a linear time complexity with respect to the number of nodes times the average number of neighboring nodes in the given graphs. The experimental result shows that the proposed kernel performs comparable and much faster than a state-of-the-art graph kernel for benchmark data sets and shows high scalability for new applications with large graphs.
title: GSML,A Unified Framework for Sparse Metric Learning
author: Kaizhu Huang,Yiming Ying,Colin Campbell
abstract: There has been significant recent interest in sparse metric learning (SML) in which we simultaneously learn both a good distance metric and a low-dimensional representation. Unfortunately the performance of existing sparse metric learning approaches is usually limited because the authors assumed certain problem relaxations or they target the SML objective indirectly. In this paper we propose a Generalized Sparse Metric Learning method (GSML). This novel framework offers a unified view for understanding many of the popular sparse metric learning algorithms including the Sparse Metric Learning framework proposed the Large Margin Nearest Neighbor (LMNN) and the D-ranking Vector Machine (D-ranking VM). Moreover GSML also establishes a close relationship with the Pairwise Support Vector Machine. Furthermore the proposed framework is capable of extending many current non-sparse metric learning models such as Relevant Vector Machine (RCA) and a state-of-the-art method proposed into their sparse versions. We present the detailed framework provide theoretical justifications build various connections with other models and propose a practical iterative optimization method making the framework both theoretically important and practically scalable for medium or large datasets. A series of experiments show that the proposed approach can outperform previous methods in terms of both test accuracy and dimension reduction on six real-world benchmark datasets.
title: GRAPE,A Graph-Based Framework for Disambiguating People Appearances in Web Search
author: Lili Jiang,Jianyong Wang,Ning An,Shengyuan Wang,Jian Zhan,Lian Li
abstract: Finding information about people using search engines is one of the most common activities on the Web. However search engines usually return a long list of Web pages which may be relevant to many namesakes especially given the explosive growth of Web data. To address the challenge caused by name ambiguity in Web people search this paper proposes a novel graph-based framework GRAPE (abbr. a Graph-based fRamework for disAmbiguating People appEarances in Web search). In GRAPE people tag information (e.g. people name organization and email address) surrounding the queried people name is extracted from the search results a graph-based unsupervised algorithm is then developed to cluster the extracted tags where a new method Cohesion is introduced to measure the importance of a tag for clustering and each final cluster of tags represents a unique people entity. Experimental results show that our proposed framework outperforms the state-of-the-art Web people name disambiguation approaches.
title: A Tree-Based Framework for Difference Summarization
author: Ruoming Jin,Yuri Breitbart,Rong Li
abstract: Understanding the differences between two datasets is a fundamental data mining question and is also ubiquitously important across many real world scientific applications. In this paper we propose a tree-based framework to provide a parsimonious explanation of the difference between two distributions based on rigorous two-sample statistical test. We develop two efficient approaches. The first one is a dynamic programming approach that finds a minimal number of data subsets that describe the difference between two data sets. The second one is a greedy approach that approximates the dynamic programming approach. We employ the well-known Friedman's MST (minimal spanning tree) statistics for two-sample statistical tests in our summarization tree construction and develop novel techniques to speedup its computational procedure. We performed a detailed experimental evaluation on both real and synthetic datasets and demonstrated the effectiveness of our tree-summarization approach.
title: TrBagg,A Simple Transfer Learning Method and its Application to Personalization in Collaborative Tagging
author: Toshihiro Kamishima,Masahiro Hamasaki,Shotaro Akaho
abstract: The aim of transfer learning is to improve prediction accuracy on a target task by exploiting the training examples for tasks that are related to the target one. Transfer learning has received more attention in recent years because this technique is considered to be helpful in reducing the cost of labeling. In this paper we propose a very simple approach to transfer learning,TrBagg which is the extension of bagging. TrBagg is composed of two stages,Many weak classifiers are first generated as in standard bagging and these classifiers are then filtered based on their usefulness for the target task. This simplicity makes it easy to work reasonably well without severe tuning of learning parameters. Further our algorithm equips an algorithmic scheme to avoid negative transfer. We applied TrBagg to personalized tag prediction tasks for social bookmarks Our approach has several convenient characteristics for this task such as adaptation to multiple tasks with low computational cost.
title: PEGASUS,A Peta-Scale Graph Mining System Implementation and Observations
author: U. Kang,Charalampos E. Tsourakakis,Christos Faloutsos
abstract: In this paper we describe PEGASUS an open source Peta Graph Mining library which performs typical graph mining tasks such as computing the diameter of the graph computing the radius of each node and finding the connected components. As the size of graphs reaches several Giga- Tera- or Peta-bytes the necessity for such a library grows too. To the best of our knowledge PEGASUS is the first such library implemented on the top of the Hadoop platform the open source version of MapReduce. Many graph mining operations (PageRank spectral clustering diameter estimation connected components etc.) are essentially a repeated matrix-vector multiplication. In this paper we describe a very important primitive for PEGASUS called GIM-V (Generalized Iterated Matrix-Vector multiplication). GIM-V is highly optimized achieving (a) good scale-up on the number of available machines (b) linear running time on the number of edges and (c) more than 5 times faster performance over the non-optimized version of GIM-V. Our experiments ran on M45 one of the top 50 supercomputers in the world. We report our findings on several real graphs including one of the largest publicly available Web Graphs thanks to Yahoo! with 6,7 billion edges.
title: Efficient Discovery of Frequent Correlated Subgraph Pairs
author: Yiping Ke,James Cheng,Jeffrey Xu Yu
abstract: The recent proliferation of graph data in a wide spectrum of applications has led to an increasing demand for advanced data analysis techniques. In view of this many graph mining techniques such as frequent subgraph mining and correlated subgraph mining have been proposed. In many applications both frequency and correlation play an important role. Thus this paper studies a new problem of mining the set of frequent correlated subgraph pairs. A simple algorithm that combines existing algorithms for mining frequent subgraphs and correlated subgraphs results in a multiplication of the mining operations the majority of which are redundant. We discover that most of the graphs correlated to a common graph are also highly correlated. We establish theoretical foundations for this finding and derive a tight lower bound on the correlation of any two graphs that are correlated to a common graph. This theoretical result leads to the design of a very effective skipping mechanism by which we skip the processing of a majority of graphs in the mining process. Our algorithm FCP-Miner is a fast approximate algorithm but we show that the missing pairs are only a small set of marginally correlated pairs. Extensive experiments verify both the efficiency and effectiveness of FCP-Miner.
title: Self-Adaptive Anytime Stream Clustering
author: Philipp Kranen,Ira Assent,Corinna Baldauf,Thomas Seidl
abstract: Clustering streaming data requires algorithms which are capable of updating clustering results for the incoming data. As data is constantly arriving time for processing is limited. Clustering has to be performed in a single pass over the incoming data and within the possibly varying inter-arrival times of the stream. Likewise memory is limited making it impossible to store all data. For clustering we are faced with the challenge of maintaining a current result that can be presented to the user at any given time. In this work we propose a parameter free algorithm that automatically adapts to the speed of the data stream. It makes best use of the time available under the current constraints to provide a clustering of the objects seen up to that point. Our approach incorporates the age of the objects to reflect the greater importance of more recent data. Moreover we are capable of detecting concept drift novelty and outliers in the stream. For efficient and effective handling we introduce the ClusTree a compact and self-adaptive index structure for maintaining stream summaries. Our experiments show that our approach is capable of handling a multitude of different stream characteristics for accurate and scalable anytime stream clustering.
title: Improving SVM Classification on Imbalanced Data Sets in Distance Spaces
author: Suzan K?knar-Tezel,Longin Jan Latecki
abstract: Imbalanced data sets present a particular challenge to the data mining community. Often it is the rare event that is of interest and the cost of misclassifying the rare event is higher than misclassifying the usual event. When the data is highly skewed toward the usual it can be very difficult for a learning system to accurately detect the rare event. There have been many approaches in recent years for handling imbalanced data sets from under-sampling the majority class to adding synthetic points to the minority class in feature space. Distances between time series are known to be non-Euclidean and nonmetric since comparing time series requires warping in time. This fact makes it impossible to apply standard methods like SMOTE to insert synthetic data points in feature spaces. We present an innovative approach that augments the minority class by adding synthetic points in distance spaces. We then use Support Vector Machines for classification. Our experimental results on standard time series show that our synthetic points significantly improve the classification rate of the rare events and in many cases also improves the overall accuracy of SVM.
title: CoCoST,A Computational Cost Efficient Classifier
author: Liyun Li,Umut Topkara,Baris Coskun,Nasir Memon
abstract: Computational cost of classification is as important as accuracy in on-line classification systems. The computational cost is usually dominated by the cost of computing implicit features of the raw input data. Very few efforts have been made to design classifiers which perform effectively with limited computational power,instead feature selection is usually employed as a pre-processing step to reduce the cost of running traditional classifiers. We present CoCoST a novel and effective approach for building classifiers which achieve state-of-the-art classification accuracy while keeping the expected computational cost of classification low even without feature selection. CoCost employs a wide range of novel cost-aware decision trees each of which is tuned to specialize in classifying instances from a subset of the input space and judiciously consults them depending on the input instance in accordance with a cost-aware meta-classifier. Experimental results on a network flow detection application show that our approach can achieve better accuracy than classifiers such as SVM and random forests while achieving 75%-90% reduction in the computational costs.
title: Semi-naive Exploitation of One-Dependence Estimators
author: Nan Li,Yang Yu,Zhi-Hua Zhou
abstract: It is well known that the key of Bayesian classifier learning is to balance the two important issues that is the exploration of attribute dependencies in high orders for ensuring a sufficient flexibility in approximating the ground-truth dependencies and the exploration of low orders for ensuring a stable probability estimate from limited training samples. By allowing one-order attribute dependencies one-dependence estimators (ODEs) have been shown to be able to approximate the ground-truth attribute dependencies whilst keeping the effectiveness of probability estimation and therefore leading to excellent performance. In previous studies however ODEs were exploited in simple ways such as by averaging for classification. In this paper we propose a semi-naive exploitation of ODEs that fits a function of ODEs to pursue higher-order attribute dependencies. Extensive experiments show that the proposed SNODE approach can achieve better performance than many state-of-the-art Bayesian classifiers.
title: A Framework for Computing the Privacy Scores of Users in Online Social Networks
author: Kun Liu,Evimaria Terzi
abstract: A large body of work has been devoted to address corporate-scale privacy concerns related to social networks. The main focus was on how to share social networks owned by organizations without revealing the identities or sensitive relationships of the users involved. Not much attention has been given to the privacy risk of users posed by their information sharing activities. In this paper we approach the privacy concerns arising in online social networks from the individual users viewpoint,we propose a framework to compute a privacy score of a user which indicates the potential privacy risk caused by his participation in the network. Our definition of privacy score satisfies the following intuitive properties,the more sensitive the information revealed by a user the higher his privacy risk. Also the more visible the disclosed information becomes in the network the higher the privacy risk. We develop mathematical models to estimate both sensitivity and visibility of the information. We apply our methods to synthetic and real-world data and demonstrate their efficacy and practical utility.
title: Least Square Incremental Linear Discriminant Analysis
author: Li-Ping Liu,Yuan Jiang,Zhi-Hua Zhou
abstract: Linear discriminant analysis (LDA) is a well-known dimension reduction approach which projects high-dimensional data into a low-dimensional space with the best separation of different classes. In many tasks the data accumulates over time and thus incremental LDA is more desirable than batch LDA. Several incremental LDA algorithms have been developed and achieved success,however the eigen-problem involved requires a large computation cost which hampers the efficiency of these algorithms. In this paper we propose a new incremental LDA algorithm LS-ILDA based on the least square solution of LDA. When new samples are received LS-ILDA incrementally updates the least square solution of LDA. Our analysis discloses that this algorithm produces the exact least square solution of batch LDA while its computational cost is O(min(n,d)  d) for one update on dataset containing n instances in d-dimensional space. Experimental results show that comparing with state-of-the-art incremental LDA algorithms our proposed LS-ILDA achieves high accuracy with low time cost.
title: Unified Solution to Nonnegative Data Factorization Problems
author: Xiaobai Liu,Shuicheng Yan,Jun Yan,Hai Jin
abstract: In this paper we restudy the non-convex data factorization problems (regularized or not unsupervised or supervised) where the optimization is confined in the emph{nonnegative} orthant and provide a emph{unified} convergency provable solution based on multiplicative nonnegative update rules. This solution is general for optimization problems with block-wisely quadratic objective functions and thus direct update rules can be derived by skipping over the tedious specific procedure deduction process and algorithmic convergence proof. By taking this unified solution as a general template we i) re-explain several existing nonnegative data factorization algorithms ii) develop a variant of nonnegative matrix factorization formulation for handling out-of-sample data and iii) propose a new nonnegative data factorization algorithm called Correlated Co-Decomposition (CCD) to simultaneously factorize two feature spaces by exploring the inter-correlated information. Experiments on both face recognition and multi-label image annotation tasks demonstrate the wide applicability of the unified solution as well as the effectiveness of two proposed new algorithms.
title: Extended Boolean Matrix Decomposition
author: Haibing Lu,Jaideep Vaidya,Vijayalakshmi Atluri,Yuan Hong
abstract: With the vast increase in collection and storage of data the problem of data summarization is most critical for effective data management. Since much of this data is categorical in nature it can be viewed in terms of a Boolean matrix. Boolean matrix decomposition (BMD) has been used to provide concise and interpretable representations of Boolean data sets. A Boolean matrix can be expressed as a product of two Boolean matrices where the first matrix represents a set of meaningful concepts and the second describes how the observed data can be expressed as combinations of those concepts. Typically the combination is only in terms of the set union. In other words a successful Boolean matrix decomposition gives a set of concepts and shows how every column of the input data can be expressed as a union of some subset of those concepts. However this way of modeling only incompletely represents real data semantics. Essentially it ignores a critical component -- the set difference operation,a column can be expressed as the combination of union of certain concepts as well as the exclusion of other concepts. This has two significant benefits. First the total number of concepts required to describe the data may itself be reduced. Second a more succinct summarization may be found for every column. In this paper we propose the extended Boolean matrix decomposition (EBMD) problem which aims to factor Boolean matrices using both the set union and set difference operations. We study several variants of the problem show that they are NP-hard and propose efficient heuristics to solve them. Extensive experimental results demonstrate the power of EBMD.
title: Active Learning with Adaptive Heterogeneous Ensembles
author: Zhenyu Lu,Xindong Wu,Josh Bongard
abstract: One common approach to active learning is to iteratively train a single classifier by choosing data points based on its uncertainty but it is nontrivial to design uncertainty measures unbiased by the choice of classifier. Query by committee suggests that given an ensemble of diverse but accurate classifiers the most informative data points are those that cause maximal disagreement among the predictions of the ensemble members. However the method for finding ensembles appropriate to a given data set remains an open question. In this paper the random subspace method is combined with active learning to create multiple instances of different classifier types and an algorithm is introduced that adapts the ratio of different classifier types in the ensemble towards better overall accuracy. Here we show that the proposed algorithm outperforms C4.5 with uncertainty sampling Naive Bayes with uncertainty sampling bagging boosting and the random subspace method with random sampling. To the best of our knowledge our work is the first to adapt the ratio of classifiers in a heterogeneous ensemble for active learning.
title: Non-negative Laplacian Embedding
author: Dijun Luo,Chris Ding,Heng Huang,Tao Li
abstract: Laplacian embedding provides a low dimensional representation for a matrix of pairwise similarity data using the eigenvectors of the Laplacian matrix. The true power of Laplacian embedding is that it provides an approximation of the Ratio Cut clustering. However Ratio Cut clustering requires the solution to be {it nonnegative}. In this paper we propose a new approach nonnegative Laplacian embedding which approximates Ratio Cut clustering in a more direct way than traditional approaches. From the solution of our approach clustering structures can be read off directly. We also propose an efficient algorithm to optimize the objective function utilized in our approach. Empirical studies on many real world datasets show that our approach leads to more accurate Ratio Cut solution and improves clustering accuracy at the same time.
title: Scalable Algorithms for Distribution Search
author: Yasuko Matsubara,Yasushi Sakurai,Masatoshi Yoshikawa
abstract: Distribution data naturally arise in countless domains such as meteorology biology geology industry and economics. However relatively little attention has been paid to data mining for large distribution sets. Given n distributions of multiple categories and a query distribution Q we want to find similar clouds (i.e. distributions) to discover patterns rules and outlier clouds. For example consider the numerical case of sales of items where for each item sold we record the unit price and quantity,then each customer is represented as a distribution of 2-d points (one for each item he/she bought). We want to find similar users e.g. for market segmentation anomaly/fraud detection. We propose to address this problem and present D-Search which includes fast and effective algorithms for similarity search in large distribution datasets. Our main contributions are (1) approximate KL divergence which can speed up cloud-similarity computations (2) multi-step sequential scan which efficiently prunes a significant number of search candidates and leads to a direct reduction in the search cost. We also introduce an extended version of D-Search,(3) time-series distribution mining which finds similar subsequences in time-series distribution datasets. Extensive experiments on real multi-dimensional datasets show that our solution achieves up to 2,300 faster wall-clock time over the naive implementation while it does not sacrifice accuracy.
title: A Deep Non-linear Feature Mapping for Large-Margin kNN Classification
author: Renqiang Min,David A. Stanley,Zineng Yuan,Anthony Bonner,Zhaolei Zhang
abstract: KNN is one of the most popular data mining methods for classification but it often fails to work well with inappropriate choice of distance metric or due to the presence of numerous class-irrelevant features. Linear feature transformation methods have been widely applied to extract class-relevant information to improve kNN classification which is very limited in many applications. Kernels have also been used to learn powerful non-linear feature transformations but these methods fail to scale to large datasets. In this paper we present a scalable non-linear feature mapping method based on a deep neural network pretrained with Restricted Boltzmann Machines for improving kNN classification in a large-margin framework which we call DNet-kNN. DNet-kNN can be used for both classification and for supervised dimensionality reduction. The experimental results on two benchmark handwritten digit datasets and one newsgroup text dataset show that DNet-kNN has much better performance than large-margin kNN using a linear mapping and kNN based on a deep autoencoder pretrained with Restricted Boltzmann Machines.
title: Finding Time Series Motifs in Disk-Resident Data
author: Abdullah Mueen,Eamonn Keogh,Nima Bigdely-Shamlo
abstract: Time series motifs are sets of very similar subsequences of a long time series. They are of interest in their own right and are also used as inputs in several higher-level data mining algorithms including classification clustering rule-discovery and summarization. In spite of extensive research in recent years finding exact time series motifs in massive databases is an open problem. Previous efforts either found approximate motifs or considered relatively small datasets residing in main memory. In this work we describe for the first time a disk-aware algorithm to find exact time series motifs in multi-gigabyte databases which contain on the order of tens of millions of time series. We have evaluated our algorithm on datasets from diverse areas including medicine anthropology computer networking and image processing and show that we can find interesting and meaningful motifs in datasets that are many orders of magnitude larger than anything considered before.
title: Relevant Subspace Clustering,Mining the Most Interesting Non-redundant Concepts in High Dimensional Data
author: Emmanuel Mller,Ira Assent,Stephan Gnnemann,Ralph Krieger,Thomas Seidl
abstract: Subspace clustering aims at detecting clusters in any subspace projection of a high dimensional space. As the number of possible subspace projections is exponential in the number of dimensions the result is often tremendously large. Recent approaches fail to reduce results to relevant subspace clusters. Their results are typically highly redundant i.e. many clusters are detected multiple times in several projections. In this work we propose a novel model for relevant subspace clustering (RESCU). We present a global optimization which detects the most interesting non-redundant subspace clusters. We prove that computation of this model is NP-hard. For RESCU we propose an approximative solution that shows high accuracy with respect to our relevance model. Thorough experiments on synthetic and real world data show that RESCU successfully reduces the result to manageable sizes. It reliably achieves top clustering quality while competing approaches show greatly varying performance.
title: Stacked Gaussian Process Learning
author: Marion Neumann,Kristian Kersting,Zhao Xu,Daniel Schulz
abstract: Triggered by a market relevant application that involves making joint predictions of pedestrian and public transit flows in urban areas we address the question of how to utilize hidden common cause relations among variables of interest in order to improve performance in the two related regression tasks. Specifically we propose stacked Gaussian process learning a meta-learning scheme in which a base Gaussian process is enhanced by adding the posterior covariance functions of other related tasks to its covariance function in a stage-wise optimization. The idea is that the stacked posterior covariances encode the hidden common causes among variables of interest that are shared across the related regression tasks. Stacked Gaussian process learning is efficient capable of capturing shared common causes and can be implemented with any kind of standard Gaussian process regression model such as sparse approximations and relational variants. Our experimental results on real-world data from the market relevant application show that stacked Gaussian processes learning can significantly improve prediction performance of a standard Gaussian process.
title: Evaluating Statistical Tests for Within-Network Classifiers of Relational Data
author: Jennifer Neville,Brian Gallagher,Tina Eliassi-Rad
abstract: Recently a number of modeling techniques have been developed for data mining and machine learning in relational and network domains where the instances are not independent and identically distributed (i.i.d.). These methods specifically exploit the statistical dependencies among instances in order to improve classification accuracy. However there has been little focus on how these same dependencies affect our ability to draw accurate conclusions about the performance of the models. More specifically the complex link structure and attribute dependencies in network data violate the assumptions of many conventional statistical tests and make it difficult to use these tests to assess the models in an unbiased manner. In this work we examine the task of within-network classification and the question of whether two algorithms will learn models which will result in significantly different levels of performance. We show that the commonly-used form of evaluation (paired t-test on overlapping network samples) can result in an unacceptable level of Type I error. Furthermore we show that Type I error increases as (1) the correlation among instances increases and (2) the size of the evaluation set increases (i.e. the proportion of labeled nodes in the network decreases). We propose a method for network cross-validation that combined with paired t-tests produces more acceptable levels of Type I error while still providing reasonable levels of statistical power (i.e. Type II error).
title: Discovering Excitatory Networks from Discrete Event Streams with Applications to Neuronal Spike Train Analysis
author: Debprakash Patnaik,Srivatsan Laxman,Naren Ramakrishnan
abstract: Mining temporal network models from discrete event streams is an important problem with applications in computational neuroscience physical plant diagnostics and human-computer interaction modeling. We focus in this paper on temporal models representable as excitatory networks where all connections are stimulative rather than inhibitory. Through this emphasis on excitatory networks we show how they can be learned by creating bridges to frequent episode mining. Specifically we show that frequent episodes help identify nodes with high mutual information relationships and which can be summarized into a dynamic Bayesian network (DBN). To demonstrate the practical feasibility of our approach we show how excitatory networks can be inferred from both mathematical models of spiking neurons as well as real neuroscience datasets.
title: Clustering Trajectories of Moving Objects in an Uncertain World
author: Nikos Pelekis,Ioannis Kopanakis,Evangelos Kotsifakos,Elias Frentzos,Yannis Theodoridis
abstract: Mining Trajectory Databases (TD) has recently gained great interest due to the popularity of tracking devices. On the other hand the inherent presence of uncertainty in TD (e.g. due to GPS errors) has not been taken yet into account during the mining process. In this paper we study the effect of uncertainty in TD clustering and introduce a three-step approach to deal with it. First we propose an intuitionistic point vector representation of trajectories that encompasses the underlying uncertainty and introduce an effective distance metric to cope with uncertainty. Second we devise CenTra a novel algorithm which tackles the problem of discovering the Centroid Trajectory of a group of movements. Third we propose a variant of the Fuzzy C-Means (FCM) clustering algorithm which embodies CenTra at its update procedure. The experimental evaluation over real world TD demonstrates the efficiency and effectiveness of our approach.
title: Semi-Supervised Sequence Labeling with Self-Learned Features
author: Yanjun Qi,Pavel Kuksa,Ronan Collobert,Kunihiko Sadamasa,Koray Kavukcuoglu,Jason Weston
abstract: Typical information extraction (IE) systems can be seen as tasks assigning labels to words in a natural language sequence. The performance is restricted by the availability of labeled words. To tackle this issue we propose a semi-supervised approach to improve the sequence labeling procedure in IE through a class of algorithms with {em self-learned features} (SLF). A supervised classifier can be trained with annotated text sequences and used to classify each word in a large set of unannotated sentences. By averaging predicted labels over all cases in the unlabeled corpus SLF training builds class label distribution patterns for each word (or word attribute) in the dictionary and re-trains the current model iteratively adding these distributions as extra word {em features}. Basic SLF models how likely a word could be assigned to target class types. Several extensions are proposed such as learning words' class boundary distributions. SLF exhibits robust and scalable behaviour and is easy to tune. We applied this approach on four classical IE tasks,named entity recognition (German and English) part-of-speech tagging (English) and one gene name recognition corpus. Experimental results show effective improvements over the supervised baselines on all tasks. In addition when compared with the closely related self-training idea this approach shows favorable advantages.
title: Semi-Markov kMeans Clustering and Activity Recognition from Body-Worn Sensors
author: Matthew W. Robards,Peter Sunehag
abstract: Subsequence clustering aims to find patterns that appear repeatedly in time series data. We introduce a novel subsequence clustering technique that we call semi-Markov kmeans clustering. The clustering results in ideal examples of the repeating patterns and in labeled segmentations that can be used as training data for sophisticated discriminative methods like max-margin semi-Markov models. We are applying the new clustering technique to activity recognition from body-worn sensors by showing how it can enable a system to learn from data that is only annotated by an ordered list of activity types that have been undertaken. This kind of annotation unlike a detailed segmentation of the sensor data is easily provided by a non-expert user. We show that we can achieve equally good results using only an ordered list of activity types for training as when using a full detailed labeled segmentation.
title: A Sparsification Approach for Temporal Graphical Model Decomposition
author: Ning Ruan,Ruoming Jin,Victor E. Lee,Kun Huang
abstract: Temporal causal modeling can be used to recover the causal structure among a group of relevant time series variables. Several methods have been developed to explicitly construct temporal causal graphical models. However how to best understand and conceptualize these complicated causal relationships is still an open problem. In this paper we propose a decomposition approach to simplify the temporal graphical model. Our method clusters time series variables into groups such that strong interactions appear among the variables within each group and weak (or no) interactions exist for cross-group variable pairs. Specifically we formulate the clustering problem for temporal graphical models as a regression-coefficient sparsification problem and define an interesting objective function which balances the model prediction power and its cluster structure. We introduce an iterative optimization approach utilizing the Quasi-Newton method and generalized ridge regression to minimize the objective function and to produce a clustered temporal graphical model. We also present a novel optimization procedure utilizing a graph theoretical tool based on the maximum weight independent set problem to speed up the Quasi-Newton method for a large number of variables. Finally our detailed experimental study on both synthetic and real datasets demonstrates the effectiveness of our methods.
title: Resolving Identity Uncertainty with Learned Random Walks
author: Ted Sandler,Lyle H. Ungar,Koby Crammer
abstract: A pervasive problem in large relational databases is identity uncertainty which occurs when multiple entries in a database refer to the same underlying entity in the world. Relational databases exhibit rich graphical structure and are naturally modeled as graphs whose nodes represent entities and whose typed-edges represent relations between them. We propose using random walk models for resolving identity uncertainty since they have proven effective for finding points which are proximately located in a network. Because not all types of relations are equally helpful in alleviating identity uncertainty we develop a supervised approach to learning the usefulness of different database relations from a training set of database entries whose true identities are known. When tested on the task of resolving uncertainty of ambiguously named authors in bibliographical data the learned random walk models yield performance superior to support vector machines and to a related spectral clustering method.
title: Discriminative Mixed-Membership Models
author: Hanhuai Shan,Arindam Banerjee,Nikunj C Oza
abstract: Although mixed-membership models have achieved great success in unsupervised learning they have not been widely applied to classification problems. In this paper we propose a family of discriminative mixed-membership models for classification by combining unsupervised mixed-membership models with multi-class logistic regression. In particular we propose two variants respectively applicable to text classification based on latent Dirichlet allocation and usual feature vector classification based on mixed-membership naive Bayes models. The proposed models allow the number of components in the mixed membership to be different from the number of classes. We propose two variational inference based algorithms for learning the models including a fast variational inference which is substantially more efficient than mean-field variational approximation. Through extensive experiments on UCI and text classification benchmark datasets we show that the models are competitive with the state of the art and can discover components not explicitly captured by the class labels.
title: Argumentation Based Constraint Acquisition
author: Kostyantyn Shchekotykhin,Gerhard Friedrich
abstract: Efficient acquisition of constraint networks is a key factor for the applicability of constraint problem solving methods. Current techniques learn constraint networks from sets of training examples where each example is classified as either a solution or non-solution of a target network. However in addition to this classification an expert can usually provide arguments as to why examples should be rejected or accepted. Generally speaking domain specialists have partial knowledge about the theory to be acquired which can be exploited for knowledge acquisition. Based on this observation we discuss the various types of arguments an expert can formulate and develop a knowledge acquisition algorithm for processing these types of arguments which gives the expert the possibility to input arguments in addition to the learning examples. The result of this approach is a significant reduction in the number of examples which must be provided to the learner in order to learn the target constraint network.
title: Extending Semi-supervised Learning Methods for Inductive Transfer Learning
author: Yuan Shi,Zhenzhong Lan,Wei Liu,Wei Bi
abstract: Inductive transfer learning and semi-supervised learning are two different branches of machine learning. The former tries to reuse knowledge in labeled out-of-domain instances while the later attempts to exploit the usefulness of unlabeled in-domain instances. In this paper we bridge the two branches by pointing out that many semi-supervised learning methods can be extended for inductive transfer learning if the step of labeling an unlabeled instance is replaced by re-weighting a diff-distribution instance. Based on this recognition we develop a new transfer learning method namely COITL by extending the co-training method in semi-supervised learning. Experimental results reveal that COITL can achieve significantly higher generalization and robustness compared with two state-of-the-art methods in inductive transfer learning.
title: iTopicModel,Information Network-Integrated Topic Modeling
author: Yizhou Sun,Jiawei Han,Jing Gao,Yintao Yu
abstract: Document networks i.e. networks associated with text information are becoming increasingly popular due to the ubiquity of Web documents blogs and various kinds of online data. In this paper we propose a novel topic modeling framework for document networks which builds a unified generative topic model that is able to consider both text and structure information for documents. A graphical model is proposed to describe the generative model. On the top layer of this graphical model we define a novel multivariate Markov Random Field for topic distribution random variables for each document to model the dependency relationships among documents over the network structure. On the bottom layer we follow the traditional topic model to model the generation of text for each document. A joint distribution function for both the text and structure of the documents is thus provided. A solution to estimate this topic model is given by maximizing the log-likelihood of the joint probability. Some important practical issues in real applications are also discussed including how to decide the topic number and how to choose a good network structure. We apply the model on two real datasets DBLP and Cora and the experiments show that this model is more effective in comparison with the state-of-the-art topic modeling algorithms.
title: Uncoverning Groups via Heterogeneous Interaction Analysis
author: Lei Tang,Xufei Wang,Huan Liu
abstract: With the pervasive availability of Web 2.0 and social networking sites people can interact with each other easily through various social media. For instance popular sites like Del.icio.us Flickr and YouTube allow users to comment shared content (bookmark photos videos) and users can tag their own favorite content. Users can also connect to each other and subscribe to or become a fan or a follower of others. These diverse individual activities result in a multi-dimensional network among actors forming cross-dimension group structures with group members sharing certain similarities. It is challenging to effectively integrate the network information of multiple dimensions in order to discover cross-dimension group structures. In this work we propose a two-phase strategy to identify the hidden structures shared across dimensions in multi-dimensional networks. We extract structural features from each dimension of the network via modularity analysis and then integrate them all to find out a robust community structure among actors. Experiments on synthetic and real-world data validate the superiority of our strategy enabling the analysis of collective behavior underneath diverse individual activities in a large scale.
title: Significance of Episodes Based on Minimal Windows
author: Nikolaj Tatti
abstract: Discovering episodes frequent sets of events from a sequence has been an active field in pattern mining. Traditionally a level-wise approach is used to discover all frequent episodes. While this technique is computationally feasible it may result in a vast number of patterns especially when low thresholds are used. In this paper we propose a new quality measure for episodes. We say that an episode is significant if the average length of its minimal windows deviates greatly when compared to the expected length according to the independence model. We can apply this measure as a post-pruning step to test whether the discovered frequent episodes are truly interesting and consequently to reduce the number of output. As a main contribution we introduce a technique that allows us to compute the distribution of lengths of minimal windows using the independence model. Such a computation task is surpisingly complex and in order to solve it we compute the distribution iteratively starting from simple episodes and progressively moving towards the more complex ones. In our experiments we discover candidate episodes that have a sufficient amount of minimal windows and test each candidate for significance. The experimental results demonstrate that our approach finds significant episodes while ignoring uninteresting ones.
title: Convex Non-negative Matrix Factorization in the Wild
author: Christian Thurau,Kristian Kersting,Christian Bauckhage
abstract: Non-negative matrix factorization (NMF) has recently received a lot of attention in data mining information retrieval and computer vision. It factorizes a non-negative input matrix V into two non-negative matrix factors V = WH such that W describes "clusters" of the datasets. Analyzing genotypes social networks or images it can be beneficial to ensure V to contain meaningful ``cluster centroids'' i.e. to restrict W to be convex combinations of data points. But how can we run this convex NMF in the wild i.e. given millions of data points? Triggered by the simple observation that each data point is a convex combination of vertices of the data convex hull we propose to restrict W further to be vertices of the convex hull. The benefits of this convex-hull NMF approach are twofold. First the expected size of the convex hull of the candidate set typically grows much slower than the data set. Second distance preserving low-dimensional embeddings allow one to compute candidate vertices efficiently. Our extensive experimental evaluation shows that convex-hull NMF compares favorably to convex NMF for large data sets both in terms of speed and reconstruction quality. Moreover we show that our method can easily be applied to large-scale real-world data sets in our case consisting of 1.6 million images respectively 160 million votes on World of Warcraft guilds.
title: On K-Means Cluster Preservation Using Quantization Schemes
author: Deepak S. Turaga,Michail Vlachos,Olivier Verscheure
abstract: This work examines under what conditions compression methodologies can retain the outcome of clustering operations. We focus on the popular k-Means clustering algorithm and we demonstrate how a properly constructed compression scheme based on post-clustering quantization is capable of maintaining the global cluster structure. Our analytical derivations indicate that a 1-bit moment preserving quantizer per cluster is sufficient to retain the original data clusters. Merits of the proposed compression technique include,a) reduced storage requirements with clustering guarantees b) data privacy on the original values and c) shape preservation for data visualization purposes. We evaluate quantization scheme on various high-dimensional datasets including 1-dimensional and 2-dimensional time-series (shape datasets) and demonstrate the cluster preservation property. We also compare with previously proposed simplification techniques in the time-series area and show significant improvements both on the clustering and shape preservation of the compressed datasets.
title: Scalable Classification in Large Scale Spatiotemporal Domains Applied to Voltage-Sensitive Dye Imaging
author: Igor Vainer,Sarit Kraus,Gal Kaminka,Hamutal Slovin
abstract: We present an approach for learning models that obtain accurate classification of large scale data objects collected in spatiotemporal domains. The model generation is structured in three phases,pixel selection (spatial dimension reduction) spatiotemporal features extraction and feature selection. Novel techniques for the first two phases are presented with two alternatives for the middle phase. Model generation based on the combinations of techniques from each phase is explored. The introduced methodology is applied on datasets from the Voltage-Sensitive Dye Imaging (VSDI) domain where the generated classification models successfully decode neuronal population responses in the visual cortex of behaving animals. VSDI currently is the best technique enabling simultaneous high spatial (10,000 points) and temporal (10 ms or less) resolution imaging from neuronal population in the cortex. We demonstrate that not only our approach is scalable enough to handle computationally challenging data but it also contributes to the neuroimaging field of study with its decoding abilities.
title: Extracting Output Metadata from Scientific Deep Web Data Sources
author: Fan Wang,Gagan Agrawal
abstract: Increasingly many data sources appear as online databases hidden behind query forms thus forming the deep web. The popularity of this new medium for data dissemination is leading to new problems in data integration. Particularly to enable data integration from multiple deep web data sources one needs to obtain the metadata for each of the data sources. Obtaining the metadata particularly the output schema can be very challenging. This is because given an input query many deep web data sources only return a subset of the output schema attributes i.e the ones that have a non-NULL value for the corresponding input. In this paper we propose two approaches which are the sampling model approach and the mixture model approach respectively to efficiently obtain an approximately complete set of output schema attributes from a deep web data source. Our experiments show while each of the above two approaches has limitations a hybrid strategy where we combine the two approaches achieves high recall with good precision for most data sources.
title: Semi-supervised Multi-task Learning with Task Regularizations
author: Fei Wang,Xin Wang,Tao Li
abstract: Multi-task learning refers to the learning problem of performing inference by jointly considering multiple related tasks. There have already been many research efforts on supervised multi-task learning. However collecting sufficient labeled data for each task is usually time consuming and expensive. In this paper we consider the semi-supervised multitask learning (SSMTL) problem where we are given a small portion of labeled points together with a large pool of unlabeled data within each task. We assume that the different tasks can form some task clusters and the task in the same cluster share similar classifier parameters. The final learning problem is relaxed to a convex one and an efficient gradient descent strategy is proposed. Finally the experimental results on both synthetic and real world data sets are presented to show the effectiveness of our method.
title: Fast Online Training of Ramp Loss Support Vector Machines
author: Zhuang Wang,Slobodan Vucetic
abstract: A fast online algorithm OnlineSVMR for training Ramp-Loss Support Vector Machines (SVMRs) is proposed. It finds the optimal SVMR for t+1 training examples using SVMR built on t previous examples. The algorithm retains the KarushCKuhnCTucker conditions on all previously observed examples. This is achieved by an SMO-style incremental learning and decremental unlearning under the Concave-Convex Procedure framework. Further speedup of training time could be achieved by dropping the requirement of optimality. A variant called OnlineASVMR is a greedy approach that approximately optimizes the SVMR objective function and is suitable for online active learning. The proposed algorithms were comprehensively evaluated on 9 large benchmark data sets. The results demonstrate that OnlineSVMR (1) has the similar computational cost as its offline counterpart,(2) outperforms IDSVM its competing online algorithm that uses hinge-loss in terms of accuracy model sparsity and training time. The experiments on online active learning show that for a fixed number of label queries OnlineASVMR (1) achieves consistently better accuracy than QueryAll and competitive accuracy to Greedy approach,(2) outperforms the active learning version of IDSVM.
title: Mining Peculiarity Groups in Day-by-Day Behavioral Datasets
author: Yun Xiong,Yangyong Zhu
abstract: Behavior mining is one of the most important issues in data mining. The growing interest in the study of behavior mining has been credited to the availability of a large amount of individual behavioral data. Some objects containing common behavioral patterns in the dataset are dramatically different from other individual objects and show their peculiarities. It is very important for behavior analysis to mine these peculiar objects' groups as this has great potential in practice. However to the best of our knowledge it has not been explored before. In this paper we identify this interesting and practical problem of behavior mining,mining peculiarity groups and defining a measurement of the degree of peculiarity. As the first attempt to tackle the problem we present a set-value-oriented day-by-day behavioral data expression mode considering that daily behaviors with respect to an object should be recorded as a set of behaviors and devise a peculiarity group mining algorithm in view of the set-value-oriented data expression which cannot be very well handled by existing methods. Furthermore we show that our method is practical and efficient using real datasets.
title: Online System Problem Detection by Mining Patterns of Console Logs
author: Wei Xu,Ling Huang,Armando Fox,David Patterson,Michael Jordan
abstract: We describe a novel application of using data mining and statistical learning methods to automatically monitor and detect abnormal execution traces from console logs in an online setting. Different from existing solutions we use a two stage detection system. The first stage uses frequent pattern mining and distribution estimation techniques to capture the dominant patterns (both frequent sequences and time duration). The second stage use principal component analysis based anomaly detection technique to identify actual problems. Using real system data from a 203-node Hadoop [1] cluster we show that we can not only achieve highly accurate and fast problem detection but also help operators better understand execution patterns in their system.
title: Synthesizing Novel Dimension Reduction Algorithms in Matrix Trace Oriented Optimization Framework
author: Jun Yan,Ning Liu,Shuicheng Yan,Qiang Yang,Zheng Chen
abstract: Dimension Reduction (DR) algorithms are generally categorized into feature extraction and feature selection algorithms. In the past few works have been done to contrast and unify the two algorithm categories. In this work we introduce a matrix trace oriented optimization framework to provide a unifying view for both feature extraction and selection algorithms. We show that the unified view of DR algorithms allows us to discover some essential relationships among many state-of- the-art DR algorithms. Inspired by these essential insights we propose to synthesize unlimited number of novel DR algorithms by combining mapping and integrating the state- of-the-art algorithms. We present examples of newly synthesized DR algorithms with experimental results to show the effectiveness of our automatically synthesized algorithms.
title: Peculiarity Analysis for Classifications
author: Jian Yang,Ning Zhong,Yiyu Yao,Jue Wang
abstract: Peculiarity-oriented mining (POM) is a new data mining method consisting of peculiar data identification and peculiar data analysis. Peculiarity factor (PF) and local peculiarity factor (LPF) are important concepts employed to describe the peculiarity of points in the identification step. One can study the notions at both attribute and record levels. In this paper a new record LPF called distance based record LPF (D-record LPF) is proposed which is defined as the sum of distances between a point and its nearest neighbors. It is proved mathematically that D-record LPF can characterize accurately the probability density function of a continuous m-dimensional distribution. This provides a theoretical basis for some existing distance based anomaly detection techniques. More important it also provides an effective method for describing the class conditional probabilities in the Bayesian classifier. The result enables us to apply peculiarity analysis for classification problems. A novel algorithm called LPF-Bayes classifier and its kernelized implementation are presented which have some connection to the Bayesian classifier. Experimental results on several benchmark data sets demonstrate that the proposed classifiers are effective.
title: Filtering and Refinement,A Two-Stage Approach for Efficient and Effective Anomaly Detection
author: Xiao Yu,Lu An Tang,Jiawei Han
abstract: Anomaly detection is an important data mining task. Most existing methods treat anomalies as inconsistencies and spend the majority amount of time on modeling normal instances. A recently proposed sampling-based approach may substantially boost the efficiency in anomaly detection but may also lead to weaker accuracy and robustness. In this study we propose a two-stage approach to find anomalies in complex datasets with high accuracy as well as low time complexity and space cost. Instead of analyzing normal instances our algorithm first employs an efficient deterministic space partition algorithm to eliminate obvious normal instances and generates a small set of anomaly candidates with a single scan of the dataset. It then checks each candidate with density-based multiple criteria to determine the final results. This two-stage framework also detects anomalies of different notions. Our experiments show that this new approach finds anomalies successfully in different conditions and ensures a good balance of efficiency accuracy and robustness.
title: Mining Data Streams with Labeled and Unlabeled Training Examples
author: Peng Zhang,Xingquan Zhu,Li Guo
abstract: In this paper we propose a framework to build prediction models from data streams which contain both labeled and unlabeled examples. We argue that due to the increasing data collection ability but limited resources for labeling stream data collected at hand may only have a small number of labeled examples whereas a large portion of data remain unlabeled but can be beneficial for learning. Unleashing the full potential of the unlabeled instances for stream data mining is however a significant challenge consider that even fully labeled data streams may suffer from the concept drifting and inappropriate uses of the unlabeled samples may only make the problem even worse. To build prediction models we first categorize the stream data into four different categories each of which corresponds to the situation where concept drifting may or may not exist in the labeled and unlabeled data. After that we propose a relational k-means based transfer semi-supervised SVM learning framework (RK-TS3VM) which intends to leverage labeled and unlabeled samples to build prediction models. Experimental results and comparisons on both synthetic and real-world data streams demonstrate that the proposed framework is able to help build prediction models more accurate than other simple approaches can offer.
title: Maximum Margin Clustering with Multivariate Loss Function
author: Bin Zhao,James Kwok,Changshui Zhang
abstract: This paper presents a simple but powerful extension of the maximum margin clustering (MMC) algorithm that optimizes multivariate performance measure specifically defined for clustering including Normalized Mutual In- formation Rand Index and F-measure. Different from previous MMC algorithms that always employ the error rate as the loss function our formulation involves a multivariate loss function that is a non-linear combination of the individual clustering results. Computationally we propose a cutting plane algorithm to approximately solve the resulting optimization problem with a guaranteed accuracy. Experimental evaluations show clear improvements in clustering performance of our method over previous maximum margin clustering algorithms.
title: Efficient Discovery of Confounders in Large Data Sets
author: Wenjun Zhou,Hui Xiong
abstract: Given a large transaction database association analysis is concerned with efficiently finding strongly related objects. Unlike traditional associate analysis where relationships among variables are searched at a global level we examine confounding factors at a local level. Indeed many real-world phenomena are localized to specific regions and times. These relationships may not be visible when the entire data set is analyzed. Specially confounding effects that change the direction of correlation is the most significant. Along this line we propose to efficiently find confounding effects attributable to local associations. Specifically we derive an upper bound by a necessary condition of confounders which can help us prune the search space and efficiently identify confounders. Experimental results show that the proposed CONFOUND algorithm can effectively identify confounders and the computational performance is an order of magnitude faster than benchmark methods.
title: Vague One-Class Learning for Data Streams
author: Xingquan Zhu,Xindong Wu,Chengqi Zhang
abstract: In this paper we formulate a new research problem of learning from vaguely labeled one-class data streams where the main objective is to allow users to label instance groups instead of single instances as positive samples for learning. The batch-labeling however raises serious issues because labeled groups may contain non-positive samples and users may change their labeling interests at any time. To solve this problem we propose a Vague One-Class Learning (VOCL) framework which employs a double weighting approach at both instance and classifier levels to build an ensembling framework for learning. At instance level both local and global filterings are considered for instance weight adjustment. Two solutions are proposed to take instance weight values into the classifier training process. At classifier level a weight value is assigned to each classifier of the ensemble to ensure that learning can quickly adapt to users interests. Experimental results on synthetic and real-world data streams demonstrate that the proposed VOCL framework significantly outperforms other methods for vaguely labeled one-class data streams.
title: Inverse Time Dependency in Convex Regularized Learning
author: Zeyuan Allen Zhu,Weizhu Chen,Chenguang Zhu,Gang Wang,Haixun Wang,Zheng Chen
abstract: In the conventional regularized learning training time increases as the training set expands. Recent work on L2 linear SVM challenges this common sense by proposing the inverse time dependency on the training set size. In this paper we first put forward a Primal Gradient Solver (PGS) to effectively solve the convex regularized learning problem. This solver is based on the stochastic gradient descent method and the Fenchel conjugate adjustment employing the well-known online strongly convex optimization algorithm with logarithmic regret. We then theoretically prove the inverse dependency property of our PGS embracing the previous work of the L2 linear SVM as a special case and enable the l_p-norm optimization to run within a bounded sphere which qualifies more convex loss functions in PGS. We further illustrate this solver in three examples,SVM logistic regression and regularized least square. Experimental results substantiate the property of the inverse dependency on training data size.
title: P-packSVM,Parallel Primal grAdient desCent Kernel SVM
author: Zeyuan Allen Zhu,Weizhu Chen,Gang Wang,Chenguang Zhu,Zheng Chen
abstract: It is an extreme challenge to produce a nonlinear SVM classifier on very large scale data. In this paper we describe a novel P-packSVM algorithm that can solve the Support Vector Machine (SVM) optimization problem with an arbitrary kernel. This algorithm embraces the best known stochastic gradient descent method to optimize the primal objective and has 1/?? dependency in complexity to obtain a solution of optimization error ??. The algorithm can be highly parallelized with a special packing strategy and experiences sub-linear speed-up with hundreds of processors. We demonstrate that P-packSVM achieves accuracy sufficiently close to that of SVM-light and overwhelms the state-of-the-art parallel SVM trainer PSVM in both accuracy and efficiency. As an illustration our algorithm trains CCAT dataset with 800k samples in 13 minutes and 95% accuracy while PSVM needs 5 hours but only has 92% accuracy. We at last demonstrate the capability of P-packSVM on 8 million training samples.
title: An L-infinity Norm Visual Classifier
author: Anushka Anand,Leland Wilkinson,Dang Nhon Tuan
abstract: We introduce a mathematical framework based on the L-infinity norm distance metric to describe human interactions in a visual data mining environment. We use the framework to build a classifier that involves an algebra on hyper-rectangles. Our classifier called VisClassifier generates set-wise rules from simple gestures in an exploratory visual GUI. Logging these rules allows us to apply our analysis to a new sample or batch of data so that we can assess the predictive power of our visual-processing motivated classifier. The accuracy of this classifier on widely-used benchmark datasets rivals the accuracy of competitive classifiers.
title: Outlier Detection Using Inductive Logic Programming
author: Fabrizio Angiulli,Fabio Fassetti
abstract: We present a novel definition of outlier in the context of inductive logic programming. Given a set of positive and negative examples the definition aims at singling out the examples showing anomalous behavior. We note that the task here pursued is different from noise removal and in fact the anomalous observations we discover are different in nature from noisy ones. We discuss pecularities of the novel approach present an algorithm for detecting outliers discuss some examples of knowledge mined and compare it with alternative approaches.
title: Joint Emotion-Topic Modeling for Social Affective Text Mining
author: Shenghua Bao,Shengliang Xu,Li Zhang,Rong Yan,Zhong Su,Dingyi Han,Yong Yu
abstract: This paper is concerned with the problem of social affective text mining which aims to discover the connections between social emotions and affective terms based on user-generated emotion labels. We propose a joint emotion-topic model by augmenting latent Dirichlet allocation with an additional layer for emotion modeling. It first generates a set of latent topics from emotions followed by generating affective terms from each topic. Experimental results on an online news collection show that the proposed model can effectively identify meaningful latent topics for each emotion. Evaluation on emotion prediction further verifies the effectiveness of the proposed model.
title: Algorithms for Large Sparse Network Alignment Problems
author: Mohsen Bayati,Margot Gerritsen,David F. Gleich,Amin Saberi,Ying Wang
abstract: We propose a new distributed algorithm for sparse variants of the network alignment problem which occurs in a variety of data mining areas including systems biology database matching and computer vision. Our algorithm uses a belief propagation heuristic and provides near optimal solutions for this NP-hard combinatorial optimization problem. We show that our algorithm is faster and outperforms or ties existing algorithms on synthetic problems a problem in bioinformatics and a problem in ontology matching. We also provide a unified framework for studying and comparing all network alignment solvers.
title: Dirichlet Mixture Allocation for Multiclass Document Collections Modeling
author: Wei Bian,Dacheng Tao
abstract: Topic model Latent Dirichlet Allocation (LDA) is an effective tool for statistical analysis of large collections of documents. In LDA each document is modeled as a mixture of topics and the topic proportions are generated from the unimodal Dirichlet distribution prior. When a collection of documents are drawn from multiple classes this unimodal prior is insufficient for data fitting. To solve this problem we exploit the multimodal Dirichlet mixture prior and propose the Dirichlet mixture allocation (DMA). We report experiments on the popular TDT2 Corpus demonstrating that DMA models a collection of documents more precisely than LDA when the documents are obtained from multiple classes.
title: SLIDER,Mining Correlated Motifs in Protein-Protein Interaction Networks
author: Peter Boyen,Frank Neven,Dyck Dries Van,Dijk Aalt-Jan van,Ham Roeland C.H.J van
abstract: Correlated motif mining (CMM) is the problem to find overrepresented pairs of patterns called motif pairs in interacting protein sequences. Algorithmic solutions for CMM thereby provide a computational method for predicting binding sites for protein interaction. In this paper we adopt a motif-driven approach where the support of candidate motif pairs is evaluated in the network. We experimentally establish the superiority of the Chi-square-based support measure over other support measures. Furthermore we obtain that CMM is an NP-hard problem for a large class of support measures (including Chi-square) and reformulate the search for correlated motifs as a combinatorial optimization problem. We then present the method SLIDER which uses local search with a neighborhood function based on sliding motifs and employs the Chi-square-based support measure. We show that SLIDER outperforms existing motif-driven CMM methods and scales to large protein-protein interaction networks.
title: Effective Anomaly Detection in Sensor Networks Data Streams
author: Saha Budhaditya,Duc-Son Pham,Mihai Lazarescu,Svetha Venkatesh
abstract: This paper addresses a major challenge in data mining applications where the full information about the underlying processes such as sensor networks or large online database cannot be practically obtained due to physical limitations such as low bandwidth or memory storage or computing power. Motivated by the recent theory on direct information sampling called compressed sensing (CS) we propose a framework for detecting anomalies from these large-scale data mining applications where the full information is not practically possible to obtain. Exploiting the fact that the intrinsic dimension of the data in these applications are typically small relative to the raw dimension and the fact that compressed sensing is capable of capturing most information with few measurements our work show that spectral methods that used for volume anomaly detection can be directly applied to the CS data with guarantee on performance. Our theoretical contributions are supported by extensive experimental results on large datasets which show satisfactory performance.
title: Hierarchical Bayesian Models for Collaborative Tagging Systems
author: Saha Budhaditya,Duc-Son Pham,Mihai Lazarescu,Svetha Venkatesh
abstract: Collaborative tagging systems with user generated content have become a fundamental element of websites such as Delicious Flickr or CiteULike. By sharing common knowledge massively linked semantic data sets are generated that provide new challenges for data mining. In this paper we reduce the data complexity in these systems by finding meaningful topics that serve to group similar users and serve to recommend tags or resources to users. We propose a well-founded probabilistic approach that can model every aspect of a collaborative tagging system. By integrating both user information and tag information into the well-known Latent Dirichlet Allocation framework the developed models can be used to solve a number of important information extraction and retrieval tasks.
title: Efficient Algorithm for Computing Link-Based Similarity in Real World Networks
author: Yuanzhe Cai,Gao Cong,Xu Jia,Hongyan Liu,Jun He,Jiaheng Lu,Xiaoyong Du
abstract: Similarity calculation has many applications such as information retrieval and collaborative filtering among many others. It has been shown that link-based similarity measure such as SimRank is very effective in characterizing the object similarities in networks such as the Web by exploiting the object-to-object relationship. Unfortunately it is prohibitively expensive to compute the link-based similarity in a relatively large graph. In this paper based on the observation that link-based similarity scores of real world graphs follow the power-law distribution we propose a new approximate algorithm namely Power-SimRank with guaranteed error bound to efficiently compute link-based similarity measure. We also prove the convergence of the proposed algorithm. Extensive experiments conducted on real world datasets and synthetic datasets show that the proposed algorithm outperforms SimRank by four-five times in terms of efficiency while the error generated by the approximation is small.
title: Fine-Grain Perturbation for Privacy Preserving Data Publishing
author: Rhonda Chaytor,Ke Wang,Patricia Brantingham
abstract: Recent work [12] shows that conventional privacy preserving publishing techniques based on anonymity-groups are susceptible to corruption attacks. In a corruption attack if the sensitive information of any anonymity-group member is uncovered then the remaining group members are at risk. In this study we abandon anonymity-groups and hide sensitive information through perturbation on the sensitive attribute. With each record being perturbed independently corruption attacks cannot be effectively carried out. Previous anti-corruption work did not minimize information loss. This paper proposes to address this issue by allowing fine-grain privacy specification. We demonstrate the power of our approach through experiments on real medical and synthetic datasets.
title: Accelerated Gradient Method for Multi-task Sparse Learning Problem
author: Xi Chen,Weike Pan,James T. Kwok,Jaime G Carbonell
abstract: Many real world learning problems can be recast as multi-task learning problems which utilize correlations among different tasks to obtain better generalization performance than learning each task individually. The feature selection problem in multi-task setting has many applications in fields of computer vision text classification and bio-informatics. Generally it can be realized by solving a L-1-infinity regularized optimization problem. And the solution automatically yields the joint sparsity among different tasks. However due to the nonsmooth nature of the L-1-infinity norm there lacks an efficient training algorithm for solving such problem with general convex loss functions. In this paper we propose an accelerated gradient method based on an ``optimal'' first order black-box method named after Nesterov and provide the convergence rate for smooth convex loss functions. For nonsmooth convex loss functions such as hinge loss our method still has fast convergence rate empirically. Moreover by exploiting the structure of the L-1-infinity ball we solve the black-box oracle in Nesterov's method by a simple sorting scheme. Our method is suitable for large-scale multi-task learning problem since it only utilizes the first order information and is very easy to implement. Experimental results show that our method significantly outperforms the most state-of-the-art methods in both convergence speed and learning accuracy.
title: CoFKM,A Centralized Method for Multiple-View Clustering
author: Guillaume Cleuziou,Matthieu Exbrayat,Lionel Martin,Jacques-Henri Sublemontier
abstract: This paper deals with clustering for multi-view data i.e. objects described by several sets of variables or proximity matrices. Many important domains or applications such as Information Retrieval biology chemistry and marketing are concerned by this problematic. The aim of this data mining research field is to search for clustering patterns that perform a consensus between the patterns from different views. This requires to merge information from each view by performing a fusion process that identifies the agreement between the views and solves the conflicts. Various fusion strategies can be applied occurring either before after or during the clustering process. We draw our inspiration from the existing algorithms based on a centralized strategy. We propose a fuzzy clustering approach that generalizes the three fusion strategies and outperforms the main existing multi-view clustering algorithm both on synthetic and real datasets.
title: Active Selection of Sensor Sites in Remote Sensing Applications
author: Debasish Das,Zoran Obradovic,Slobodan Vucetic
abstract: In a data-mining approach a model for estimation of Aerosol Optical Depth (AOD) from satellite observations is learned using collocated satellite and ground-based observations. For accurate learning of such a spatio-temporal model it is important to collect ground-based data from a large number of sites. The objective of this project is to determine appropriate locations for the next set of ground-based data collection sites to maximize accuracy of AOD estimation. Ideally a new site should capture the most significant unseen aerosol patterns and should be the least correlated with the previously observed patterns. We propose achieving this aim by selecting the locations on which the existing prediction model is the most uncertain. Several criteria were considered for site selection including uncertainty spatial diversity similarity in temporal pattern and their combination. Extensive experiments on globally distributed data over 90 AERONET sites from the years 2005 and 2006 provide strong evidence that sites selected using the proposed algorithms improve the overall AOD prediction accuracy at a faster rate than those selected randomly or based on spatial diversity among sites.
title: Large Scale Relation Acquisition Using Class Dependent Patterns
author: Saeger Stijn De,Kentaro Torisawa,Jun'ichi Kazama,Kow Kuroda,Masaki Murata
abstract: This paper proposes a minimally supervised method for acquiring high-level semantic relations such as causality and prevention from the Web. Our method learns linguistic patterns that express causality such as x gave rise to y and uses them to extract causal noun pairs like (global warming malaria epidemic) from sentences like global warming gave rise to a new malaria epidemic. The novelty of our method lies in the use of semantic word classes acquired by large scale clustering for learning class dependent patterns. We demonstrate the effectiveness of this class based approach on three large-scale relation mining tasks from 50 million Japanese Web pages. In two of these tasks we obtained more than 30,000 relation instances with over 80% precision outperforming a state-of-the-art system by a large margin.
title: Finding Maximal Fully-Correlated Itemsets in Large Databases
author: Lian Duan,William Nick Street
abstract: Finding the most interesting correlations among items is essential for problems in many commercial medical and scientific domains. Much previous research focuses on finding correlated pairs instead of correlated itemsets in which all items are correlated with each other. When designing gift sets store shelf arrangements or website product categories we are more interested in correlated itemsets than correlated pairs. We solve this problem by finding maximal fully-correlated itemsets (MFCIs) in which all subsets are closely related to all other subsets. Putting the items in an MFCI together can promote sales within this itemset. Though some exsiting methods find high-correlation itemsets they suffer from both efficiency and effectiveness problems in large datasets. In this paper we explore high-dimensional correlation in two ways. First we expand the set of desirable properties for correlation measures and study the advantages and disadvantages of various measures. Second we propose an MFCI framework to decouple the correlation measure from the need for efficient search. By wrapping the best measure in our MFCI framework we take advantage of likelihood ratios superiority in evaluating itemsets make use of the properties of MFCI to eliminate itemsets with irrelevant items and still achieve good computational performance.
title: Bayesian Overlapping Subspace Clustering
author: Qiang Fu,Arindam Banerjee
abstract: Given a data matrix the problem of finding dense/uniform sub-blocks in the matrix is becoming important in several applications. The problem is inherently combinatorial since the uniform sub-blocks may involve arbitrary subsets of rows and columns and may even be overlapping. While there are a few existing methods based on co-clustering or subspace clustering they typically rely on local search heuristics and in general do not have a systematic model for such data. We present a Bayesian Overlapping Subspace Clustering (BOSC) model which is a hierarchical generative model for matrices with potentially overlapping uniform sub-block structures. The BOSC model can also handle matrices with missing entries. We propose an EM-style algorithm based on approximate inference using Gibbs sampling and parameter estimation using coordinate descent for the BOSC model. Through experiments on both simulated and real datasets we demonstrate that the proposed algorithm outperforms the state-of-the-art.
title: Unsupervised Relation Extraction by Massive Clustering
author: Edgar Gonzlez,Jordi Turmo
abstract: The goal of Information Extraction is to automatically generate structured pieces of information from the relevant information contained in text documents. Machine Learning techniques have been applied to reduce the cost of Information Extraction system adaptation. However elements of human supervision strongly bias the learning process. Unsupervised learning approaches can avoid these biases. In this paper we propose an unsupervised approach to learning for Relation Detection based on the use of massive clustering ensembles. The results obtained on the ACE Relation Mention Detection task outperform in terms of F1 score by 5 points the state of the art of unsupervised techniques for this evaluation framework in addition to being simpler and more flexible.
title: Regression Learning Vector Quantization
author: Mihajlo Grbovic,Slobodan Vucetic
abstract: Learning Vector Quantization (LVQ) is a popular class of nearest prototype classifiers for multiclass classification. Learning algorithms from this family are widely used because of their intuitively clear learning process and ease of implementation. In this paper we propose an extension of the LVQ algorithm to regression. Just like the LVQ algorithm the proposed modification uses a supervised learning procedure to learn the best prototype positions but unlike LVQ algorithm for classification it also learns the best prototype target values. This results in the effective partition of the feature space similar to the one the K-means algorithm would make. Experimental results on benchmark datasets showed that the proposed Regression LVQ algorithm performs better than the nearest prototype competitors that choose prototypes randomly or through K-means clustering classification LVQ on quantized target values and similarly to the memory-based Parzen Window and Nearest Neighbor algorithms.
title: Projective Clustering Ensembles
author: Francesco Gullo,Carlotta Domeniconi,Andrea Tagarelli
abstract: Recent advances in data clustering concern clustering ensembles and projective clustering methods each addressing different issues in clustering problems. In this paper we consider for the first time the projective clustering ensemble (PCE) problem whose main goal is to derive a proper projective consensus partition from an ensemble of projective clustering solutions. We formalize PCE as an optimization problem which does not rely on any particular clustering ensemble algorithm and which has the ability to handle hard as well as soft data clustering and different feature weightings. We provide two formulations for PCE namely a two-objective and a single-objective problem in which the object-based and feature-based representations of the ensemble solutions are taken into account differently. Experiments have demonstrated that the proposed methods for PCE show clear improvements in terms of accuracy of the output consensus partition.
title: Knowledge Discovery from Citation Networks
author: Zhen Guo,Zhongfei Zhang,Shenghuo Zhu,Yun Chi,Yihong Gong
abstract: Knowledge discovery from scientific articles has received increasing attentions recently since huge repositories are made available by the development of the Internet and digital databases. In a corpus of scientific articles such as a digital library documents are connected by citations and one document plays two different roles in the corpus,emph{document itself} and emph{a citation of other documents}. In the existing topic models little effort is made to differentiate these two roles. We believe that the topic distributions of these two roles are different and related in a certain way. In this paper we propose a emph{Bernoulli Process Topic}~(BPT) model which models the corpus at two levels,emph{document level} and emph{citation level}. In the BPT model each document has two different representations in the latent topic space associated with its roles. Moreover the multi-level hierarchical structure of the citation network is captured by a generative process involving a Bernoulli process. The distribution parameters of the BPT model are estimated by a variational approximation approach. In addition to conducting the experimental evaluations on the document modeling task we also apply the BPT model to a well known scientific corpus to discover the latent topics. The comparisons against state-of-the-art methods demonstrate a very promising performance.
title: An Effective Approach to Inverse Frequent Set Mining
author: Antonella Guzzo,Domenico Sacc,Edoardo Serra
abstract: The inverse frequent set mining problem is the problem of computing a database on which a given collection of itemsets must emerge to be frequent. Earlier studies focused on investigating computational and approximability properties of this problem. In this paper we face it under the pragmatic perspective of defining heuristic solution approaches that are effective and scalable in real scenarios. In particular a general formulation of the problem is considered where minimum and maximum support constraints can be defined on each itemset and where no bound is given beforehand on the size of the resulting output database. Within this setting an algorithm is proposed that always satisfies the maximum support constraints but which treats minimum support constraints as soft ones that are enforced as long as possible. A thorough experimentation evidences that minimum support constraints are hardly violated in practice and that such negligible degradation in accuracy (which is unavoidable due to the theoretical intractability of the problem) is compensated by very good scaling performances.
title: Parallel PathFinder Algorithms for Mining Structures from Graphs
author: Samson Hauguel,ChengXiang Zhai,Jiawei Han
abstract: PathFinder networks are increasingly used in Data Mining for different purposes like network visualization or knowledge extraction. This novel way of representing graphical data has been proven to give better results than other link reduction algorithms like minimum spanning networks However this increase in quality comes with a high computation cost typically of the order of n^3 or higher where n is the number of nodes in the graph. While this problem has previously been tackled by using mathematical properties to speed up the algorithm in this paper we propose two new algorithms to speed up PathFinder computation based on parallelization techniques to take advantage of the increasingly available multi-core hardware platform. Experiments show that both new algorithms are more efficient than the state of the art algorithms,one of them can achieve speed-ups of up to x127 with an average of x23 on recent hardware (2007).
title: A Bootstrap Approach to Eigenvalue Correction
author: Anne Hendrikse,Luuk Spreeuwers,Raymond Veldhuis
abstract: Eigenvalue analysis is an important aspect in many data modeling methods. Unfortunately the eigenvalues of the sample covariance matrix (sample eigenvalues) are biased estimates of the eigenvalues of the covariance matrix of the data generating process (population eigenvalues). We present a new method based on bootstrapping to reduce the bias in the sample eigenvalues,the eigenvalue estimates are updated in several iterations where in each iteration synthetic data is generated to determine how to update the population eigenvalue estimates. Comparison of the bootstrap eigenvalue correction with a state of the art correction method by Karoui shows that depending on the type of population eigenvalue distribution sometimes the Karoui method performs better and sometimes our bootstrap method.
title: Modeling Syntactic Structures of Topics with a Nested HMM-LDA
author: Jing Jiang
abstract: Latent Dirichlet Allocation (LDA) is a commonly used topic modeling method for text analysis and mining. Standard LDA treats documents as bags of words ignoring the syntactic structures of sentences. In this paper we propose a hybrid model that embeds hidden Markov models (HMMs) within LDA topics to jointly model both the topics and the syntactic structures within each topic. Our model is general and subsumes standard LDA and HMM as special cases. Compared with standard LDA and HMM our model can simultaneously discover both topic-specific content words and background functional words shared among topics. Our model can also automatically separate content words that play different roles within a topic. Using perplexity as evaluation metric our model returns lower perplexity for unseen test documents compared with standard LDA which shows its better generalization power than LDA.
title: Redistricting Using Heuristic-Based Polygonal Clustering
author: Deepti Joshi,Leen-Kiat Soh,Ashok Samal
abstract: Redistricting is the process of dividing a geographic area into districts or zones. This process has been considered in the past as a problem that is computationally too complex for an automated system to be developed that can produce unbiased plans. In this paper we present a novel method for redistricting a geographic area using a heuristic-based approach for polygonal spatial clustering. While clustering geospatial polygons several complex issues need to be addressed C such as,removing order dependency clustering all polygons assuming no outliers and strategically utilizing domain knowledge to guide the clustering process. In order to address these special needs we have developed the Constrained Polygonal Spatial Clustering (CPSC) algorithm that holistically integrates do-main knowledge in the form of cluster-level and instance-level constraints and uses heuristic functions to grow clusters. In order to illustrate the usefulness of our algorithm we have applied it to the problem of formation of unbiased congressional districts. Furthermore we compare and contrast our algorithm with two other approaches proposed in the literature for redistricting namely C graph partitioning and simulated annealing.
title: A Walk from 2-Norm SVM to 1-Norm SVM
author: Jussi Kujala,Timo Aho,Tapio Elomaa
abstract: This paper studies how useful the standard 2-norm regularized SVM is in approximating the 1-norm SVM problem. To this end we examine a general method that is based on iteratively re-weighting the features and solving a 2-norm optimization problem. The convergence rate of this method is unknown. Previous work indicates that it might require an excessive number of iterations. We study how well we can do with just a small number of iterations. In theory the convergence rate is fast except for coordinates of the current solution that are close to zero. Our empirical experiments confirm this. In many problems with irrelevant features already one iteration is often enough to produce accuracy as good as or better than that of the 1-norm SVM. Hence it seems that in these problems we do not need to converge to the 1-norm SVM solution near zero values. The benefit of this approach is that we can build something similar to the 1-norm regularized solver based on any 2-norm regularized solver. This is quick to implement and the solution inherits the good qualities of the solver such as scalability and stability.
title: Semi-supervised Density-Based Clustering
author: Levi Lelis,J?rg Sander
abstract: Most of the effort in the semi-supervised clustering literature was devoted to variations of the K-means algorithm. In this paper we show how background knowledge can be used to bias a partitional density-based clustering algorithm. Our work describes how labeled objects can be used to help the algorithm detecting suitable density parameters for the algorithm to extract density-based clusters in specific parts of the feature space. Considering the set of constraints estabilished by the labeled dataset we show that our algorithm called SSDBSCAN automatically finds density parameters for each natural cluster in a dataset. Four of the most interesting characteristics of SSDBSCAN are that (1) it only requires a single robust input parameter (2) it does not need any user intervention (3) it automaticaly finds the noise objects according to the density of the natural clusters and (4) it is able to find the natural cluster structure even when the density among clusters vary widely. The algorithm presented in this paper is evaluated with artificial and real-world datasets demonstrating better results when compared to other unsupervised and semi-supervised density-based approaches.
title: VIF Regression,A Fast Regression Algorithm for Large Data
author: Dongyu Lin,Dean P Foster
abstract: We propose a fast regression algorithm that can substantially reduce the computational complexity of searching yet retain good accuracy. It also guarantees to discover correlated features that are collectively predictive and avoid model over-fitting. Its capability of controlling mFDR (marginal False Discovery Rate) statistically enables the one-pass search of the fast algorithm and guarantees the accuracy of the sparse model chosen by the algorithm without cross validation. Numerical results show that our algorithm is much faster than any other algorithm and is competitively as accurate as the best but slower algorithms.
title: Sparse Norm-Regularized Reconstructive Coefficients Learning
author: Bin Liu,Shuo Chen,Mingjie Qian,Changshui Zhang
abstract: Inspired by the fact that the final decision rule is mainly affected by a small subset of the training samples i.e. Support Vector Machine(SVM) shows that the decision function relies on the few samples that are on or over the margin. We propose a new framework that explicitly strengthen this intuitive fact by adding an $l_1$-norm regularizer. We give different formulations for our framework in different scenarios and the experiments show that our framework can not only lead to high sparse solutions but also better performance than traditional methods.
title: A Contrast Pattern Based Clustering Quality Index for Categorical Data
author: Qingbao Liu,Guozhu Dong
abstract: Since clustering is unsupervised and highly explorative clustering validation (i.e. assessing the quality of clustering solutions) has been an important and long standing research problem. Existing validity measures have significant shortcomings. This paper proposes a novel Contrast Pattern based Clustering Quality index (CPCQ) for categorical data by utilizing the quality and diversity of the contrast patterns (CPs) which contrast the clusters in clusterings. High quality CPs can characterize clusters and discriminate them against each other. Experiments show that the CPCQ index (1) can recognize that expert-determined classes are the best clusters for many datasets from the UCI repository,(2) does not give inappropriate preference to larger number of clusters,(3) does not require a user to provide a distance function.
title: Multi-document Summarization by Information Distance
author: Chong Long,Minlie Huang,Xiaoyan Zhu,Ming Li
abstract: Fast changing knowledge on the Internet can be acquired more efficiently with the help of automatic document summarization and updating techniques. This paper described a novel approach for multi-document update summarization. The best summary is defined to be the one which has the minimum information distance to the entire document set. The best update summary has the minimum conditional information distance to a document cluster given that a prior document cluster has already been read. Experiments on the DUC 2007 dataset and the TAC 2008 dataset have proved that our method closely correlates with the human summaries and outperforms other programs such as LexRank in many categories under the ROUGE evaluation criterion.
title: On the (In)Security and (Im)Practicality of Outsourcing Precise Association Rule Mining
author: Ian Molloy,Ninghui Li,Tiancheng Li
abstract: The recent interest in outsourcing IT services onto the cloud raises two main concerns,security and cost. One task that could be outsourced is data mining. In VLDB 2007 Wong et al. propose an approach for outsourcing association rule mining. Their approach maps a set of real items into a set of pseudo items then maps each transaction non-deterministically. This paper analyzes both the security and costs associated with outsourcing association rule mining. We show how to break the encoding scheme from Wong et al. without using context specific information and reduce the security to a one-to-one mapping. We present a stricter notion of security than used by Wong et al. and then consider the practicality of outsourcing association rule mining. Our results indicate that outsourcing association rule mining may not be practical if the data owner is concerned with data confidentiality.
title: Promoting Total Efficiency in Text Clustering via Iterative and Interactive Metric Learning
author: Michinari Momma,Satoshi Morinaga,Daisuke Komura
abstract: In this paper we propose a framework to make the text clustering process as a whole efficient. In a real text clustering task an analyst usually has some expectation on the results in mind. However a single run of a clustering algorithm on the preprocessed data would not satisfy the expectation. Then the analyst faces labor-intensive trials for improving the results that involve repetitive feature refinement and parameter tuning. We develop the Iterative and Interactive Metric Learning System (IIMLS) for addressing the challenge. Specifically IIMLS allows analysts to input feedback on a current clustering result. Given the feedback IIMLS optimizes metric in the feature space so that the clustering algorithm applied with the refined metric would reflect the feedback. As a byproduct learned metric may be used for a similar dataset. Illustrative examples on a real-world dataset show IIMLS can dramatically improve efficiency of a text clustering task. The learned knowledge or the metric is visualized for gaining insights of the optimized feature metric.
title: A New Clustering Algorithm Based on Regions of Influence with Self-Detection of the Best Number of Clusters
author: Fabrice Muhlenbach,Stphane Lallich
abstract: Clustering methods usually require to know the best number of clusters or another parameter e.g. a threshold which is not ever easy to provide. This paper proposes a new graph-based clustering method called GBC which detects automatically the best number of clusters without requiring any other parameter. In this method based on regions of influence a graph is constructed and the edges of the graph having the higher values are cut according to a hierarchical divisive procedure. An index is calculated from the size average of the cut edges which self-detects the more appropriate number of clusters. The results of GBC for 3 quality indices (Dunn Silhouette and Davies-Bouldin) are compared with those of K-Means Ward's hierarchical clustering method and DBSCAN on 8 benchmarks. The experiments show the good performance of GBC in the case of well separated clusters even if the data are unbalanced non-convex or with presence of outliers whatever the shape of the clusters.
title: Automatically Extracting Dialog Models from Conversation Transcripts
author: Sumit Negi,Sachindra Joshi,Anup K. Chalamalla,L. Venkata Subramaniam
abstract: There is a growing need for task-oriented natural language dialog systems that can interact with a user to accomplish a given objective. Recent work on building task-oriented dialog systems have emphasized the need for acquiring task-specific knowledge from un-annotated conversational data. In our work we acquire task-specific knowledge by defining textit{sub-task} as the key unit of a task-oriented conversation. We propose an unsupervised apriori like algorithm that extracts the sub-tasks and their valid orderings from un-annotated human-human conversations. Modeling dialogues as a combination of sub-tasks and their valid orderings easily captures the variability in conversations. It also provides us the ability to map our dialogue model to AIML constructs and therefore use off-the-shelf AIML interpreters to build task-oriented chat-bots. We conduct experiments on real world data sets to establish the effectiveness of the sub-task extraction process. We codify the extracted sub-tasks in an AIML knowledge base and build a chatbot using this knowledge base. We also show the usefulness of the chatbot in automatically handling customer requests by performing a user evaluation study.
title: To Trust or Not to Trust? Predicting Online Trusts Using Trust Antecedent Framework
author: Viet-An Nguyen,Ee-Peng Lim,Jing Jiang,Aixin Sun
abstract: This paper analyzes the trustor and trustee factors that lead to inter-personal trust using a well studied Trust Antecedent framework in management science cite{mayer}. To apply these factors to trust ranking problem in online rating systems we derive features that correspond to each factor and develop different trust ranking models. The advantage of this approach is that features relevant to trust can be systematically derived so as to achieve good prediction accuracy. Through a series of experiments on real data from Epinions we show that even a simple model using the derived features yields good accuracy and outperforms MoleTrust a trust propagation based model. SVM classifiers using these features also show improvements.
title: Analysis of Subsequence Time-Series Clustering Based on Moving Average
author: Miho Ohsaki,Masakazu Nakase,Shigeru Katagiri
abstract: Subsequence time-series clustering (STSC) which consists of subsequence cutout with a sliding window and k-means clustering had been commonly used in time-series data mining. However a problem was pointed out that STSC always generates moderate sinusoidal patterns independently of the input. To address this problem we theoretically explain and empirically confirm the similarity between STSC and moving average. The present analysis is consistent with and simpler than one of the most important analyses of STSC. We also question the pattern extraction in the time domain and discuss another solution.
title: Permutation Tests for Studying Classifier Performance
author: Markus Ojala,Gemma C Garriga
abstract: We explore the framework of permutation-based p-values for assessing the behavior of the classification error. In this paper we study two simple permutation tests. The first test estimates the null distribution by permuting the labels in the data,this has been used extensively in classification problems in computational biology. The second test produces permutations of the features within classes inspired by restricted randomization techniques traditionally used in statistics. We study the properties of these tests and present an extensive empirical evaluation on real and synthetic data. Our analysis shows that studying the classification error via permutation tests is effective,in particular the restricted permutation test clearly reveals whether the classifier exploits the interdependency between the features in the data.
title: Interaction-Based Clustering of Multivariate Time Series
author: Claudia Plant,Afra M. Wohlschl?ger,Andrew Zherdin
abstract: In this paper we present a novel approach to clustering multivariate time series. In contrast to previous approaches we base our cluster notion on the interactions between the univariate time series within a data object. Our objective is to assign objects with a similar intrinsic interaction pattern to a common cluster. To formalize this idea we define a cluster by a set of mathematical models describing the cluster-specific interaction pattern. In addition we propose interaction K-means (IKM) an efficient algorithm for partitioning clustering of multivariate time series. The cluster-specific interaction patterns detected by IKM provide valuable information for interpretation of the cluster content. An extensive experimental evaluation on synthetic and real world data demonstrates the effectiveness and efficiency of our approach.
title: PUB,A Class Description Technique Based on Partial Coverage of Subspace
author: Ardian Kristanto Poernomo,Vivekanand Gopalkrishnan
abstract: A good description of a class should be accurate and interpretable. Previous works describe classes either by analyzing the correlation of each attribute with the class or by producing rules as in building a classifier. These solutions suffer from issues in accuracy and interpretability. A description naturally consists of sentences where each sentence consists of a set of terms. Normally a sentence is defined as a disjunction or conjunction of several terms each of which specifies a constraint (range/set of values) on an attribute. From the data analysis point of view a sentence specifies a subspace in the database. In this paper we create a richer yet interpretable form of a sentence i.e. a sentence describes an object if any $k$ attributes of that object satisfy the specified constraints. To that end we design textsc{Pub} an algorithm that produces descriptions with our form of sentences. While constructing a sentence (within the description) textsc{Pub} finds the optimal range/set of values for each attribute in linear time. We also empirically show that textsc{Pub} is efficient and able to produce more accurate concise and interpretable descriptions than current approaches on various real datasets.
title: Online and Batch Learning of Generalized Cosine Similarities
author: Ali Mustafa Qamar,Eric Gaussier
abstract: In this paper we define an online algorithm to learn the generalized cosine similarity measures for kNN classification and hence a similarity matrix A corresponding to a bilinear form. In contrary to the standard cosine measure the normalization is itself dependent on the similarity matrix which makes it impossible to use directly the algorithms developed for learning Mahanalobis distances based on positive semi-definite (PSD) matrices. We follow the approach where we first find an appropriate matrix and then project it onto the cone of PSD matrices which we have adapted to the particular form of generalized cosine similarities and more particularly to the fact that such measures are normalized. The resulting online algorithm as well as its batch version is fast and has got better accuracy as compared with state-of-the-art methods on standard data sets.
title: Discovering Organizational Structure in Dynamic Social Network
author: Jiangtao Qiu,Zhangxi Lin,Changjie Tang,Shaojie Qiao
abstract: Applying the concept of organizational structure to social network analysis may well represent the power of members and the scope of their power in a social network. In this paper we propose a data structure called Community Tree to represent the organizational structure in the social network. We combine the PageRank algorithm and random walks on graph to derive the community tree from the social network. In the real world a social network is constantly changing. Hence the organizational structure in the social network is also constantly changing. In order to present the organizational structure in a dynamic social network we propose a tree learning algorithm to derive an evolving community tree. The evolving community tree enables a smooth transition between the two community trees and well represents the evolution of organizational structure in the dynamic social network. Experiments conducted on real data show our methods are effective at discovering the organizational structure and representing the evolution of organizational structure in a dynamic social network.
title: Kernel Conditional Quantile Estimation via Reduction Revisited
author: Novi Quadrianto,Kristian Kersting,Mark D. Reid,Tibrio S. Caetano,Wray L Buntine
abstract: Quantile regression refers to the process of estimating the quantiles of a conditional distribution and has many important applications within econometrics and data mining among other domains. In this paper we show how to estimate these conditional quantile functions within a Bayes risk minimization framework using a Gaussian process prior. The resulting non-parametric probabilistic model is easy to implement and allows non-crossing quantile functions to be enforced. Moreover it can directly be used in combination with tools and extensions of standard Gaussian Processes such as principled hyperparameter estimation sparsification and quantile regression with input-dependent noise rates. No existing approach enjoys all of these desirable properties. Experiments on benchmark datasets show that our method is competitive with state-of-the-art approaches."
title: Naive Bayes Classification of Uncertain Data
author: Jiangtao Ren,Sau Dan Lee,Xianlu Chen,Ben Kao,Reynold Cheng,David Cheung
abstract: Traditional machine learning algorithms assume that data are exact or precise. However this assumption may not hold in some situations because of data uncertainty arising from measurement errors data staleness and repeated measurements etc. With uncertainty the value of each data item is represented by a probability distribution function (pdf). In this paper we propose a novel naive Bayes classification algorithm for uncertain data with a pdf. Our key solution is to extend the class conditional probability estimation in the Bayes model to handle pdfs. Extensive experiments on UCI datasets show that the accuracy of naive Bayes model can be improved by taking into account the uncertainty information.
title: Constraint-Based Pattern Mining in Dynamic Graphs
author: Cline Robardet
abstract: Dynamic graphs are used to represent relationships between entities that evolve over time. Meaningful patterns in such structured data must capture strong interactions and their evolution over time. In social networks such patterns can be seen as dynamic community structures i.e. sets of individuals who strongly and repeatedly interact. In this paper we propose a constraint-based mining approach to uncover evolving patterns. We propose to mine dense and isolated subgraphs defined by two user-parameterized constraints. The temporal evolution of such patterns is captured by associating a temporal event type to each identified subgraph. We consider five basic temporal events,The formation dissolution growth diminution and stability of subgraphs from one time stamp to the next. We propose an algorithm that finds such subgraphs in a time series of graphs processed incrementally. The extraction is feasible due to efficient patterns and data pruning strategies. We demonstrate the applicability of our method on several real-world dynamic graphs and extract meaningful evolving communities.
title: Global Slope Change Synopses for Measurement Maps
author: Frank Rosenthal,Ulrike Fischer,Peter B. Volk,Wolfgang Lehner
abstract: Quality control using scalar quality measures is standard practice in manufacturing. However there are also quality measures that are determined at a large number of positions on a product since the spatial distribution is important. We denote such a mapping of local coordinates on the product to values of a measure as a measurement map. In this paper we examine how measurement maps can be clustered according to a novel notion of similaritymapscape similaritythat considers the overall course of the measure on the map. We present a class of synopses called global slope change that uses the profile of the measure along several lines from a reference point to different points on the borders to represent a measurement map. We conduct an evaluation of global slope change using a real-world data set from manufacturing and demonstrate its superiority over other synopses.
title: A Fully Automated Method for Discovering Community Structures in High Dimensional Data
author: Jianhua Ruan
abstract: Identifying modules or natural communities in large complex networks is fundamental in many fields including social sciences biological sciences and engineering. Recently several methods have been developed to automatically identify communities from complex networks by optimizing the modularity function. The advantage of this type of approaches is that the algorithm does not require any parameter to be tuned. However the modularity-based methods for community discovery assume that the network structure is given explicitly and is correct. In addition these methods work best if the network is unweighted and/or sparse. In reality networks are often not directly defined or may be given as an affinity matrix. In the first case each node of the network is defined as a point in a high dimensional space and different networks can be obtained with different network construction methods resulting in different community structures. In the second case an affinity matrix may define a dense weighted graph for which modularity-based methods do not perform well. In this work we propose a very simple algorithm to automatically identify community structures from these two types of data. Our approach utilizes a k-nearest-neighbor network construction method to capture the topology embedded in high dimensional data and applies a modularity-based algorithm to identify the optimal community structure. A key to our approach is that the network construction is incorporated with the community identification process and is totally parameter-free. Furthermore our method can suggest appropriate pre-processing / normalization of the data to improve the results of community identification. We tested our methods on several synthetic and real data sets and evaluated its performance by internal or external accuracy indices. Compared with several existing approaches our method is not only fully automatic but also has the best accuracy overall.
title: Hierarchical Probabilistic Segmentation of Discrete Events
author: Guy Shani,Christopher Meek,Asela Gunawardana
abstract: Segmentation the task of splitting a long sequence of discrete symbols into chunks can provide important information about the nature of the sequence that is understandable to humans. Algorithms for segmenting mostly belong to the supervised learning family where a labeled corpus is available to the algorithm in the learning phase. We are interested however in the unsupervised scenario where the algorithm never sees examples of successful segmentation but still needs to discover meaningful segments. In this paper we present an unsupervised learning algorithm for segmenting sequences of symbols or categorical events. Our algorithm Hierarchical Multigram hierarchically builds a lexicon of segments and computes a maximum likelihood segmentation given the current lexicon. Thus our algorithm is most appropriate to hierarchical sequences where smaller segments are grouped into larger segments. Our probabilistic approach also allows us to suggest conditional entropy as a measurement of the quality of a segmentation in the absence of labeled data. We compare our algorithm to two previous approaches from the unsupervised segmentation literature showing it to provide superior segmentation over a number of benchmarks. We also compare our algorithm to previous approaches over a segmentation of the unlabeled interactions of a web service and its client.
title: Topic Modeling for Sequences of Temporal Activities
author: Zhiyong Shen,Ping Luo,Yuhong Xiong,Jun Sun,Yidong Shen
abstract: Temporally-ordered activity sequences are popular in many real-world domains. This paper presents an LDA-style topic model for sequences of temporal activities that captures three features of such sequences,1) the counts of unique activities 2) the Markov transition dependence and 3) the absolute or relative timestamp on each activity. In modeling the first two features we propose the concept of global transition probability and distinguish it with local transition probability used in previous work. In modeling the third feature we employ a continuous time distribution to depict the time range of latent topics. The combination of the global transition probability and the temporal information helps to refine the mixture distribution over topics for temporal sequence analysis. We present results on the data of system call traces showing better next activity prediction and sequence clustering.
title: Combining Super-Structuring and Abstraction on Sequence Classification
author: Adrian Silvescu,Cornelia Caragea,Vasant Honavar
abstract: We present an approach to adapting the data representation used by a learner on sequence classification tasks. Our approach that exploits the complementary strengths of super-structuring (constructing complex features by combining existing features) and abstraction (grouping of similar features to generate more abstract features) yields smaller and at the same time accurate models. Super-structuring provides a way to increase the predictive accuracy of the learned models by enriching the data representation (and hence increases the complexity of the learned models) whereas abstraction helps reduce the number of model parameters by simplifying the data representation. The results of our experiments on two data sets drawn from macromolecular sequence classification applications show that adapting data representation by combining super-structuring and abstraction makes it possible to construct predictive models that use significantly smaller number of features (by one to three orders of magnitude) than those that are obtained using super-structuring alone without sacrificing predictive accuracy. Our experiments also show that simplifying data representation using abstraction yields better performing models than those obtained using feature selection.
title: A Global-Model Naive Bayes Approach to the Hierarchical Prediction of Protein Functions
author: Jr. Carlos N. Silla,Alex A Freitas
abstract: In this paper we propose a new global--model approach for hierarchical classification where a single global classification model is built by considering all the classes in the hierarchy -- rather than building a number of local classification models as it is more usual in hierarchical classification. The method is an extension of the flat classification algorithm naive Bayes. We present the extension made to the original algorithm as well as its evaluation on eight protein function hierarchical classification datasets. The achieved results are positive and show that the proposed global model is better than using a local model approach.
title: Spatio-temporal Energy Based Gait Recognition
author: Shamsher Singh,K.K Biswas
abstract: Recently there has been lot of interest in using the gait energy image (GEI) of human walk sequence for individual recognition. Researchers have reported very good recognition rates using both unsupervised and supervised methods for normal walk sequences. However the performance degrades when there is a variant like change in clothing or carrying a bag. This paper shows that the performance for the variant situations can be improved by constructing the GEI with sway alignment instead of upper body alignment and dynamically selecting just the required number of rows from the bottom of the silhouette as inputs for an unsupervised feature selection approach. The improvement in recognition rates are established with performance testing on a large gait dataset.
title: Feature Selection in the Tensor Product Feature Space
author: Aaron Smalter,Jun Huan,Gerald Lushington
abstract: Classifying objects that are sampled jointly from two or more domains has many applications. The tensor product feature space is useful for modeling interactions between feature sets in different domains but feature selection in the tensor product feature space is challenging. Conventional feature selection methods ignore the structure of the feature space and may not provide the optimal results. In this paper we propose methods for selecting features in the original feature spaces of different domains. We obtained sparsity through two approaches one using integer quadratic programming and another using L1-norm regularization. Experimental studies on biological data sets validate our approach.
title: Topic Distributions over Links on Web
author: Jie Tang,Jing Zhang,Jeffrey Xu Yu,Zi Yang,Keke Cai,Rui Ma,Li Zhang,Zhong Su
abstract: It is well known that Web users create links with different intentions. However a key question which is not well studied is how to categorize the links and how to quantify the strength of the influence of a web page on another if there is a link between the two linked web pages. In this paper we focus on the problem of link semantics analysis and propose a novel supervised learning approach to build a model based on a training link-labeled and link-weighted graph where a link-label represents the category of a link and a link-weight represents the influence of one web page on the other in a link. Based on the model built we categorize links and quantify the influence of web pages on the others in a large graph in the same application domain. We discuss our proposed approach namely Pairwise Restricted Boltzmann Machines (PRBMs) and conduct extensive experimental studies to demonstrate the effectiveness of our approach using large real datasets.
title: Clustering with Multiple Graphs
author: Wei Tang,Zhengdong Lu,Inderjit S Dhillon
abstract: In graph-based learning models entities are often represented as vertices in an undirected graph with weighted edges describing the relationships between entities. In many real-world applications however entities are often associated with relations of different types and/or from different sources which can be well captured by multiple undirected graphs over the same set of vertices. How to exploit such multiple sources of information to make better inferences on entities remains an interesting open problem. In this paper we focus on the problem of clustering the vertices based on multiple graphs in both unsupervised and semi-supervised settings. As one of our contributions we propose Linked Matrix Factorization (LMF) as a novel way of fusing information from multiple graph sources. In LMF each graph is approximated by matrix factorization with a graph-specific factor and a factor common to all graphs where the common factor provides features for all vertices. Experiments on SIAM journal data show that (1) we can improve the clustering accuracy through fusing multiple sources of information with several models and (2) LMF yields superior or competitive results compared to other graph-based clustering methods.
title: Two Heads Better Than One,Metric+Active Learning and its Applications for IT Service Classification
author: Fei Wang,Jimeng Sun,Tao Li,Nikos Anerousis
abstract: Large IT service providers track service requests and their execution through problem/change tickets. It is important to classify the tickets based on the problem/change description in order to understand service quality and to optimize service processes. However two challenges exist in solving this classification problem,1) ticket descriptions from different classes are of highly diverse characteristics which invalidates most standard distance metrics,2) it is very expensive to obtain high-quality labeled data. To address these challenges we develop two seemingly independent methods 1) Discriminative Neighborhood Metric Learning (DNML) and 2) Active Learning with Median Selection (ALMS) both of which are however based on the same core technique,iterated representative selection. A case study on real IT service classification application is presented to demonstrate the effectiveness and efficiency of our proposed methods.
title: Maximum Margin Clustering on Data Manifolds
author: Fei Wang,Xin Wang,Tao Li
abstract: Clustering is one of the most fundamental and important problems in computer vision and pattern recognition communities. Maximum Margin Clustering(MMC) is a recently proposed clustering technique which has shown promising experimental results. The main theme behind MMC is to extend the standard maximum margin principle in Support Vector Machine (SVM) to the unsupervised scenario. This paper will consider the problem of maximum margin clustering on data manifolds. Specifically we propose an approach called Manifold Regularized Maximum Margin clustering (MRMMC) which combines both the maximum margin data discrimination and data manifold information in a unified clustering objective and propose an efficient algorithm to solve it. Finally the experimental results on several real world data sets are presented to show the effectiveness of our method.
title: Discovering Contexts and Contextual Outliers Using Random Walks in Graphs
author: Xiang Wang,Ian Davidson
abstract: The identifying of contextual outliers allows the discovery of anomalous behavior that other forms of outlier detection cannot find. What may appear to be normal behavior with respect to the entire data set can be shown to be anomalous by subsetting the data according to specific spatial or temporal context. However in many real-world applications we may not have sufficient a priori contextual information to discover these contextual outliers. This paper addresses the problem by proposing a probabilistic approach based on random walks which can simultaneously explore meaningful contexts and score contextual outliers therein. Our approach has several advantages including producing outlier scores which can be interpreted as stationary expectations and their calculation in closed form in polynomial time. In addition we show that point outlier detection using the stationary distribution is a special case of our approach. It allows us to find both global and contextual outliers simultaneously and to create a meaningful ranked list consisting of both types of outliers. This is a major departure from existing work where an algorithm typically identifies one type of outlier. The effectiveness of our method is justified by empirical results on real data sets with comparison to related work.
title: Effective Criterion Functions for Efficient Agglomerative Clustering on Very Large Networks
author: Yang Wang,Mingyuan An
abstract: As the agglomerative clustering algorithm is widely used in data mining image processing bioinformatics and pattern recognition. it has attracted great interests from both academical and industrial communities. However existing studies neglect the decisive factor of the efficiency of the agglomerative clustering algorithm for large complex networks and usually use criterion functions which lead to inefficiency. In this paper we propose three effective criterion functions for improving performance of agglomerative clustering algorithm. We note that clustering efficiency is determined by two factors,a) the number of neighbors of two merged clusters in each merge step,b) the number of neighbors shared by the two clusters. Based on these observations we propose a framework for designing criterion functions in order to efficiently find clusters in very large networks. We devise three criterion functions that can effectively control the number of neighbors of clusters and they can efficiently produce high-quality clusters. We have implemented our method and compared with existing studies on real networks and our method outperforms state-of-the-art approaches significantly on large networks.
title: Binomial Matrix Factorization for Discrete Collaborative Filtering
author: Jinlong Wu
abstract: Matrix factorization (MF) models have proved efficient and well scalable for collaborative filtering (CF) problems. Many researchers also present the probabilistic interpretation of MF. They usually assume that the factor vectors of users and items are from normal distributions and so are the ratings when the user and item factors are given. Then they can derive the exact MF algorithm by finding a MAP estimate of the model parameters. In this paper we suggest a new probabilistic perspective on MF for discrete CF problems. We assume that all ratings are from binomial distributions with different preference parameters instead of the original normal distributions. The new interpretation is more reasonable for discrete CF problems since they only allow several legal discrete rating values. We also present two effective algorithms to learn the new model and make predictions. They are applied to the Netflix Prize data set and acquire considerably better accuracy than those of MF.
title: Bi-relational Network Analysis Using a Fast Random Walk with Restart
author: Jing Xia,Doina Caragea,William H Hsu
abstract: Identification of nodes relevant to a given node in a relational network is a basic problem in network analysis with great practical importance. Most existing network analysis algorithms utilize one single relation to define relevancy among nodes. However in real world applications multiple relationships exist between nodes in a network. Therefore network analysis algorithms that can make use of more than one relation to identify the relevance set for a node are needed. In this paper we show how the Random Walk with Restart (RWR) approach can be used to study relevancy in a bi-relational network from the bibliographic domain and show that making use of two relations results in better results as compared to approaches that use a single relation. As relational networks can be very large we also propose a fast implementation for RWR by adapting an existing Iterative Aggregation and Disaggregation (IAD) approach. The IAD-based RWR exploits the block-wise structure of real world networks. Experimental results show significant increase in running time for the IAD-based RWR compared to the traditional power method based RWR.
title: A New MCA-Based Divisive Hierarchical Algorithm for Clustering Categorical Data
author: Tengke Xiong,Shengrui Wang,Andr Mayers,Ernest Monga
abstract: Clustering categorical data faces two challenges one is lacking of inherent similarity measure and the other is that the clusters are prone to being embedded in different subspace. In this paper we propose the first divisive hierarchical clustering algorithm for categorical data. The algorithm which is based on Multiple Correspondence Analysis (MCA) is systematic efficient and effective. In our algorithm MCA plays an important role in analyzing the data globally. The proposed algorithm has five merits. First our algorithm yields a dendrogram representing nested groupings of patterns and similarity levels at different granularities. Second it is parameter-free fully automatic and most importantly requires no assumption regarding the number of clusters. Third it is independent of the order in which the data are processed. Forth it is scalable to large data sets,and finally using the novel data representation and Chi-square distance measures makes our algorithm capable of seamlessly discovering the clusters embedded in the subspaces. Experiments on both synthetic and real data demonstrate the superior performance of our algorithm.
title: Non-sparse Multiple Kernel Learning for Fisher Discriminant Analysis
author: Fei Yan,Josef Kittler,Krystian Mikolajczyk,Atif Tahir
abstract: We consider the problem of learning a linear combination of pre-specified kernel matrices in the Fisher discriminant analysis setting. Existing methods for such a task impose an $ell_1$ norm regularisation on the kernel weights which produces sparse solution but may lead to loss of information. In this paper we propose to use $ell_2$ norm regularisation instead. The resulting learning problem is formulated as a semi-infinite program and can be solved efficiently. Through experiments on both synthetic data and a very challenging object recognition benchmark the relative advantages of the proposed method and its $ell_1$ counterpart are demonstrated and insights are gained as to how the choice of regularisation norm should be made.
title: Multirelational Topic Models
author: Jia Zeng,William K. Cheung,Chun-hung Li,Jiming Liu
abstract: In this paper we propose the multirelational topic model (MRTM) for multiple types of link modeling such as citation and coauthor links in document networks. In the citation network the MRTM models the citation link between each pair of documents as a binary variable conditioned on their topic distributions. In the coauthor network the MRTM models the coauthor link between each pair of authors as a binary variable conditioned on their expertise distributions. The topic discovery is collectively regularized by multiple relations in both citation and coauthor networks. This model can summarize topics from the document network predict citation links between documents and coauthor links between authors. Efficient inference and learning algorithms are derived based on Gibbs sampling. Experiments demonstrate that the MRTM significantly outperforms other state-of-the-art single-relational link modeling methods for large scientific document networks.
title: Learning Local Components to Understand Large Bayesian Networks
author: Yifeng Zeng,Yanping Xiang,Jorge Cordero H.,Yujian Lin
abstract: Bayesian networks are known for providing an intuitive and compact representation of probabilistic information and allowing the creation of models over a large and complex domain. Bayesian learning and reasoning are nontrivial for a large Bayesian network. In parallel it is a tough job for users (domain experts) to extract accurate information from a large Bayesian network due to dimensional difficulty. We define a formulation of local components and propose a clustering algorithm to learn such local components given complete data. The algorithm groups together most inter-relevant attributes in a domain. We evaluate its performance on three benchmark Bayesian networks and provide results in support. We further show that the learned components may represent local knowledge more precisely in comparison to the full Bayesian networks when working with a small amount of data.
title: RING,An Integrated Method for Frequent Representative Subgraph Mining
author: Shijie Zhang,Jiong Yang,Shirong Li
abstract: We propose a novel representative based subgraph mining model. A series of standards and methods are proposed to select invariants. Patterns are mapped into invariant vectors in a multidimensional space. To find qualified patterns only a subset of frequent patterns is generated as representatives such that every frequent pattern is close to one of the representative patterns while representative patterns are distant from each other. We devise the RING algorithm integrating the representative selection into the pattern mining process. Meanwhile we use R-trees to assist this mining process. Last but not least a large number of real and synthetic datasets are employed for the empirical study which show the benefits of the representative model and the efficiency of the RING algorithm.
title: A Cost-Effective LSH Filter for Fast Pairwise Mining
author: Gang Zhao,Yun Xiong,Longbing Cao,Dan Luo,Xuchun Su,Yangyong Zhu
abstract: The pairwise mining problem is to discover pairwise objects having measures greater than the user-specified minimum threshold from a collection of objects. It is essential in a large variety of database and data-mining applications. Of late there has been increasing interest in applying a Locality-Sensitive Hashing (LSH) scheme for pairwise mining. LSH-type methods have shown themselves to be simply implementable and capable of achieving significant performance gain in running time over most exact methods. However the present LSH-type methods still suffer from some bottlenecks such as the curse of threshold. In this paper we proposed a novel LSHbased method namely Cost-effective LSH filter (Ce-LSH for short) for pairwise mining. Compared with previous LSH-type methods it uses a lower fixed number of LSH functions and is thus more cost-effective. Substantial experiments evidence that our method gives significant improvement in running time over existing LSH-type methods and some recently reported method based on upper-bound. Experimental results also indicate that it scales well even for a relatively low minimum threshold and for a fairly small miss ratio.
title: A New Kernel-Based Classification Algorithm
author: Xiaofei Zhou,Wenhan Jiang,Yingjie Tian,Peng Zhang,Guangli Nie,Yong Shi
abstract: A new kernel-based learning algorithm called kernel affine subspace nearest point (KASNP) approach is proposed in this paper. Inspired by the geometrical explanation of Support Vector Machines (SVMs) and its nearest point problem in convex hulls we extend the convex hull of each class to its corresponding affine subspace in high dimensional space induced by kernel. In two class affine subspaces KASNP finds the nearest points and then constructs a separating hyperplane which bisects the line segment joining them. The nearest point problem of KASNP is only an unconstrained optimal problem whose solution can be directly computed. Compared with SVM KASNP avoids solving convex quadratic programming. Experiments on two-spiral dataset two UCI credit datasets and face recognition datasets show that our proposed KASNP is effective for data classification.
title: Regression-based latent factor models
author: Deepak Agarwal,Bee-Chung Chen
abstract: We propose a novel latent factor model to accurately predict response for large scale dyadic data in the presence of features. Our approach is based on a model that predicts response as a multiplicative function of row and column latent factors that are estimated through separate regressions on known row and column features. In fact, our model provides a single unified framework to address both cold and warm start scenarios that are commonplace in practical applications like recommender systems, online advertising, web search, etc. We provide scalable and accurate model fitting methods based on Iterated Conditional Mode and Monte Carlo EM algorithms. We show our model induces a stochastic process on the dyadic space with kernel (covariance) given by a polynomial function of features. Methods that generalize our procedure to estimate factors in an online fashion for dynamic applications are also considered. Our method is illustrated on benchmark datasets and a novel content recommendation application that arises in the context of Yahoo! Front Page. We report significant improvements over several commonly used methods on all datasets
title: Frequent pattern mining with uncertain data
author: Charu C. Aggarwal,Yan Li,Jianyong Wang,Jing Wang
abstract: This paper studies the problem of frequent pattern mining with uncertain data. We will show how broad classes of algorithms can be extended to the uncertain data setting. In particular, we will study candidate generate-and-test algorithms, hyper-structure algorithms and pattern growth based algorithms. One of our insightful observations is that the experimental behavior of different classes of algorithms is very different in the uncertain case as compared to the deterministic case. In particular, the hyper-structure and the candidate generate-and-test algorithms perform much better than tree-based algorithms. This counter-intuitive behavior is an important observation from the perspective of algorithm design of the uncertain variation of the problem. We will test the approach on a number of real and synthetic data sets, and show the effectiveness of two of our approaches over competitive techniques
title: Structured correspondence topic models for mining captioned figures in biological literature
author: Amr Ahmed,Eric P. Xing,William W. Cohen,Robert F. Murphy
abstract: A major source of information (often the most crucial and informative part) in scholarly articles from scientific journals, proceedings and books are the figures that directly provide images and other graphical illustrations of key experimental results and other scientific contents. In biological articles, a typical figure often comprises multiple panels, accompanied by either scoped or global captioned text. Moreover, the text in the caption contains important semantic entities such as protein names, gene ontology, tissues labels, etc., relevant to the images in the figure. Due to the avalanche of biological literature in recent years, and increasing popularity of various bio-imaging techniques, automatic retrieval and summarization of biological information from literature figures has emerged as a major unsolved challenge in computational knowledge extraction and management in the life science. We present a new structured probabilistic topic model built on a realistic figure generation schemeto model the structurally annotated biological figures, and we derive an efficient inference algorithm based on collapsed Gibbs sampling for information retrieval and visualization. The resulting program constitutes one of the key IR engines in our SLIF system that has recently entered the final round (4 out 70 competing systems) of the Elsevier Grand Challenge on Knowledge Enhancement in the Life Science. Here we present various evaluations on a number of data mining tasks to illustrate our method
title: Name-ethnicity classification from open sources
author: Anurag Ambekar,Charles Ward,Jahangir Mohammed,Swapna Male,Steven Skiena
abstract: The problem of ethnicity identification from names has a variety of important applications, including biomedical research, demographic studies, and marketing. Here we report on the development of an ethnicity classifier where all training data is extracted from public, non-confidential (and hence somewhat unreliable) sources. Our classifier uses hidden Markov models (HMMs) and decision trees to classify names into 13 cultural/ethnic groups with individual group accuracy comparable accuracy to earlier binary (e.g., Spanish/non-Spanish) classifiers. We have applied this classifier to over 20 million names from a large-scale news corpus, identifying interesting temporal and spatial trends on the representation of particular cultural/ethnic groups
title: Detection of unique temporal segments by information theoretic meta-clustering
author: Shin Ando,Einoshin Suzuki
abstract: The central challenge in temporal data analysis is to obtain knowledge about its underlying dynamics. In this paper, we address the observation of noisy, stochastic processes and attempt to detect temporal segments that are related to inconsistencies and irregularities in its dynamics. Many conventional anomaly detection approaches detect anomalies based on the distance between patterns, and often provide only limited intuition about the generative process of the anomalies. Meanwhile, model-based approaches have difficulty in identifying a small, clustered set of anomalies We propose Information-theoretic Meta-clustering (ITMC), a formalization of model-based clustering principled by the theory of lossy data compression. ITMC identifies a 'unique' cluster whose distribution diverges significantly from the entire dataset. Furthermore, ITMC employs a regularization term derived from the preference for high compression rate, which is critical to the precision of detection For empirical evaluation, we apply ITMC to two temporal anomaly detection tasks. Datasets are taken from generative processes involving heterogeneous and inconsistent dynamics. A comparison to baseline methods shows that the proposed algorithm detects segments from irregular states with significantly high precision and recall
title: Collusion-resistant anonymous data collection method
author: Mafruz Zaman Ashrafi,See Kiong Ng
abstract: The availability and the accuracy of the data dictate the success of a data mining application. Increasingly, there is a need to resort to on-line data collection to address the problem of data availability. However, participants in on-line data collection applications are naturally distrustful of the data collector as well as their peer respondents, resulting in inaccurate data collected as the respondents refuse to provide truthful data in fear of collusion attacks. The current anonymity-preserving solutions for on-line data collection are unable to adequately resist such attacks in a scalable fashion. In this paper, we present an efficient anonymous data collection protocol for a malicious environment such as the Internet. The protocol employs cryptographic and random shuffling techniques to preserve participants' anonymity. The proposed method is collusion-resistant and guarantees that an attacker will be unable to breach an honest participant's anonymity unless she controls all N-1 participants. In addition, our method is efficient and achieved 15-42% communication overhead reduction in comparison to the prior state-of-the-art methods
title: A viewpoint-based approach for interaction graph analysis
author: Sitaram Asur,Srinivasan Parthasarathy
abstract: Recent innovations have resulted in a plethora of social applications on the Web, such as blogs, social networks, and community photo and video sharing applications. Such applications can typically be represented as evolving interaction graphs with nodes denoting entities and edges representing their interactions. The study of entities and communities and how they evolve in such large dynamic graphs is both important and challenging While much of the past work in this area has focused on static analysis, more recently researchers have investigated dynamic analysis. In this paper, in a departure from recent efforts, we consider the problem of analyzing patterns and critical events that affect the dynamic graph from the viewpoint of a single node, or a selected subset of nodes. Defining and extracting a relevant viewpoint neighborhood efficiently, while also quantifying the key relationships among nodes involved are the key challenges we address. We also examine the evolution of viewpointneighborhoods for different entities over time to identify key structural and behavioral transformations that occur
title: Optimizing web traffic via the media scheduling problem
author: Lars Backstrom,Jon Kleinberg,Ravi Kumar
abstract: Website traffic varies through time in consistent and predictable ways, with highest traffic in the middle of the day. When providing media content to visitors, it is important to present repeat visitors with new content so that they keep coming back. In this paper we present an algorithm to balance the need to keep a website fresh with new content with the desire to present the best content to the most visitors at times of peak traffic. We formulate this as the media scheduling problem, where we attempt to maximize total clicks, given the overall traffic pattern and the time varying clickthrough rates of available media content. We present an efficient algorithm to perform this scheduling under certain conditions and apply this algorithm to real data obtained from server logs, showing evidence of significant improvements in traffic from our algorithmic schedules. Finally, we analyze the click data, presenting models for why and how the clickthrough rate for new content declines as it ages
title: Improving clustering stability with combinatorial MRFs
author: Ron Bekkerman,Martin Scholz,Krishnamurthy Viswanathan
abstract: As clustering methods are often sensitive to parameter tuning, obtaining stability in clustering results is an important task. In this work, we aim at improving clustering stability by attempting to diminish the influence of algorithmic inconsistencies and enhance the signal that comes from the data. We propose a mechanism that takes m clusterings as input and outputs m clusterings of comparable quality, which are in higher agreement with each other. We call our method the Clustering Agreement Process (CAP). To preserve the clustering quality, CAP uses the same optimization procedure as used in clustering. In particular, we study the stability problem of randomized clustering methods (which usually produce different results at each run). We focus on methods that are based on inference in a combinatorial Markov Random Field (or Comraf, for short) of a simple topology. We instantiate CAP as inference within a more complex, bipartite Comraf. We test the resultingsystem on four datasets, three of which are medium-sized text collections, while the fourth is a large-scale user/movie dataset. First, in all the four cases, our system significantly improves the clustering stability measured in terms of the macro-averaged Jaccard index. Second, in all the four cases our system managed to significantly improve clustering quality as well, achieving the state-of-the-art results. Third, our system significantly improves stability of consensus clustering built on top of the randomized clustering solutions
title: Temporal mining for interactive workflow data analysis
author: Michele Berlingerio,Fabio Pinelli,Mirco Nanni,Fosca Giannotti
abstract: In the past few years there has been an increasing interest in the analysis of process logs. Several proposed techniques, such as workflow mining, are aimed at automatically deriving the underlying workflow models. However, current approaches only pay little attention on an important piece of information contained in process logs: the timestamps, which are used to define a sequential ordering of the performed tasks. In this work we try to overcome these limitations by explicitly including time in the extracted knowledge, thus making the temporal information a first-class citizen of the analysis process. This makes it possible to discern between apparently identical process executions that are performed with different transition times between consecutive tasks This paper proposes a framework for the user-interactive exploration of a condensed representation of groups of executions of a given process. The framework is based on the use of an existing mining paradigm: Temporally-Annotated Sequences (TAS). These are aimed at extracting sequential patterns where each transition between two events is annotated with a typical transition time that emerges from input data. With the extracted TAS, which represent sets of possible frequent executions with their typical transition times, a few factorizing operators are built. These operators condense such executions according to possible parallel or possible mutual exclusive executions. Lastly, such condensed representation is rendered to the user via the exploration graph, namely the Temporally-Annotated Graph (TAG) The user, the domain expert, is allowed to explore the different and alternative factorizations corresponding to different interpretations of the actual executions. According to the user choices, the system discards or retains certain hypotheses on actual executions and shows the consequent scenarios resulting from the coresponding re-aggregation of the actual data
title: Probabilistic frequent itemset mining in uncertain databases
author: Thomas Bernecker,Hans-Peter Kriegel,Matthias Renz,Florian Verhein,Andreas Zuefle
abstract: Probabilistic frequent itemset mining in uncertain transaction databases semantically and computationally differs from traditional techniques applied to standard "certain" transaction databases. The consideration of existential uncertainty of item(sets), indicating the probability that an item(set) occurs in a transaction, makes traditional techniques inapplicable. In this paper, we introduce new probabilistic formulations of frequent itemsets based on possible world semantics. In this probabilistic context, an itemset X is called frequent if the probability that X occurs in at least minSup transactions is above a given threshold . To the best of our knowledge, this is the first approach addressing this problem under possible worlds semantics. In consideration of the probabilistic formulations, we present a framework which is able to solve the Probabilistic Frequent Itemset Mining (PFIM) problem efficiently. An extensive experimental evaluation investigates the impact of ourproposed techniques and shows that our approach is orders of magnitude faster than straight-forward approaches
title: The offset tree for learning with partial labels
author: Alina Beygelzimer,John Langford
abstract: We present an algorithm, called the Offset Tree, for learning to make decisions in situations where the payoff of only one choice is observed, rather than all choices. The algorithm reduces this setting to binary classification, allowing one to reuse any existing, fully supervised binary classification algorithm in this partial information setting. We show that the Offset Tree is an optimal reduction to binary classification. In particular, it has regret at most (k-1) times the regret of the binary classifier it uses (where k is the number of choices), and no reduction to binary classification can do better. This reduction is also computationally optimal, both at training and test time, requiring just O(log2 k) work to train on an example or make a prediction Experiments with the Offset Tree show that it generally performs better than several alternative approaches
title: New ensemble methods for evolving data streams
author: Albert Bifet,Geoff Holmes,Bernhard Pfahringer,Richard Kirkby,Ricard Gavalda
abstract: Advanced analysis of data streams is quickly becoming a key area of data mining research as the number of applications demanding such processing increases. Online mining when such data streams evolve over time, that is when concepts drift or change completely, is becoming one of the core issues. When tackling non-stationary concepts, ensembles of classifiers have several advantages over single classifier methods: they are easy to scale and parallelize, they can adapt to change quickly by pruning under-performing parts of the ensemble, and they therefore usually also generate more accurate concept descriptions. This paper proposes a new experimental data stream framework for studying concept drift, and two new variants of Bagging: ADWIN Bagging and Adaptive-Size Hoeffding Tree (ASHT) Bagging. Using the new experimental framework, an evaluation study on synthetic and real-world datasets comprising up to ten million examples shows that the new ensemble methods perform very well compared to severalknown methods
title: CoCo: coding cost for parameter-free outlier detection
author: Christian Bohm,Katrin Haegler,Nikola S. Muller,Claudia Plant
abstract: How can we automatically spot all outstanding observations in a data set? This question arises in a large variety of applications, e.g. in economy, biology and medicine. Existing approaches to outlier detection suffer from one or more of the following drawbacks: The results of many methods strongly depend on suitable parameter settings being very difficult to estimate without background knowledge on the data, e.g. the minimum cluster size or the number of desired outliers. Many methods implicitly assume Gaussian or uniformly distributed data, and/or their result is difficult to interpret. To cope with these problems, we propose CoCo, a technique for parameter-free outlier detection. The basic idea of our technique relates outlier detection to data compression: Outliers are objects which can not be effectively compressed given the data set. To avoid the assumption of a certain data distribution, CoCo relies on a very general data model combining the Exponential Power Distribution with Independent Components. We define an intuitive outlier factor based on the principle of the Minimum Description Length together with an novel algorithm for outlier detection. An extensive experimental evaluation on synthetic and real world data demonstrates the benefits of our technique. Availability: The source code of CoCo and the data sets used in the experiments are available at: http://www.dbs.ifi.lmu.de/Forschung/KDD/Boehm/CoCo
title: Efficient anomaly monitoring over moving object trajectory streams
author: Yingyi Bu,Lei Chen,Ada Wai-Chee Fu,Dawei Liu
abstract: Lately there exist increasing demands for online abnormality monitoring over trajectory streams, which are obtained from moving object tracking devices. This problem is challenging due to the requirement of high speed data processing within limited space cost. In this paper, we present a novel framework for monitoring anomalies over continuous trajectory streams. First, we illustrate the importance of distance-based anomaly monitoring over moving object trajectories. Then, we utilize the local continuity characteristics of trajectories to build local clusters upon trajectory streams and monitor anomalies via efficient pruning strategies. Finally, we propose a piecewise metric index structure to reschedule the joining order of local clusters to further reduce the time cost. Our extensive experiments demonstrate the effectiveness and efficiency of our methods
title: Connections between the lines: augmenting social networks with text
author: Jonathan Chang,Jordan Boyd-Graber,David M. Blei
abstract: Network data is ubiquitous, encoding collections of relationships between entities such as people, places, genes, or corporations. While many resources for networks of interesting entities are emerging, most of these can only annotate connections in a limited fashion. Although relationships between entities are rich, it is impractical to manually devise complete characterizations of these relationships for every pair of entities on large, real-world corpora In this paper we present a novel probabilistic topic model to analyze text corpora and infer descriptions of its entities and of relationships between those entities. We develop variational methods for performing approximate inference on our model and demonstrate that our model can be practically deployed on large corpora such as Wikipedia. We show qualitatively and quantitatively that our model can construct and annotate graphs of relationships and make useful predictions
title: Extracting discriminative concepts for domain adaptation in text mining
author: Bo Chen,Wai Lam,Ivor Tsang,Tak-Lam Wong
abstract: One common predictive modeling challenge occurs in text mining problems is that the training data and the operational (testing) data are drawn from different underlying distributions. This poses a great difficulty for many statistical learning methods. However, when the distribution in the source domain and the target domain are not identical but related, there may exist a shared concept space to preserve the relation. Consequently a good feature representation can encode this concept space and minimize the distribution gap. To formalize this intuition, we propose a domain adaptation method that parameterizes this concept space by linear transformation under which we explicitly minimize the distribution difference between the source domain with sufficient labeled data and target domains with only unlabeled data, while at the same time minimizing the empirical loss on the labeled data in the source domain. Another characteristic of our method is its capability for considering multiple classes and their interactions simultaneously. We have conducted extensive experiments on two common text mining problems, namely, information extraction and document classification to demonstrate the effectiveness of our proposed method
title: Constrained optimization for validation-guided conditional random field learning
author: Minmin Chen,Yixin Chen,Michael R. Brent,Aaron E. Tenney
abstract: Conditional random fields(CRFs) are a class of undirected graphical models which have been widely used for classifying and labeling sequence data. The training of CRFs is typically formulated as an unconstrained optimization problem that maximizes the conditional likelihood. However, maximum likelihood training is prone to overfitting. To address this issue, we propose a novel constrained nonlinear optimization formulation in which the prediction accuracy of cross-validation sets are included as constraints. Instead of requiring multiple passes of training, the constrained formulation allows the cross-validation be handled in one pass of constrained optimization The new formulation is discontinuous, and classical Lagrangian based constraint handling methods are not applicable. A new constrained optimization algorithm based on the recently proposed extended saddle point theory is developed to learn the constrained CRF model. Experimental results on gene and stock-price prediction tasks show that the constrained formulation is able to significantly improve the generalization ability of CRF training
title: Efficient influence maximization in social networks
author: Wei Chen,Yajun Wang,Siyu Yang
abstract: Influence maximization is the problem of finding a small subset of nodes (seed nodes) in a social network that could maximize the spread of influence. In this paper, we study the efficient influence maximization from two complementary directions. One is to improve the original greedy algorithm of [5] and its improvement [7] to further reduce its running time, and the second is to propose new degree discount heuristics that improves influence spread. We evaluate our algorithms by experiments on two large academic collaboration graphs obtained from the online archival database arXiv.org. Our experimental results show that (a) our improved greedy algorithm achieves better running time comparing with the improvement of [7] with matching influence spread, (b) our degree discount heuristics achieve much better influence spread than classic degree and centrality-based heuristics, and when tuned for a specific influence cascade model, it achieves almost matching influence thread with the greedy algorithm, and more importantly (c) the degree discount heuristics run only in milliseconds while even the improved greedy algorithms run in hours in our experiment graphs with a few tens of thousands of nodes Based on our results, we believe that fine-tuned heuristics may provide truly scalable solutions to the influence maximization problem with satisfying influence spread and blazingly fast running time. Therefore, contrary to what implied by the conclusion of [5] that traditional heuristics are outperformed by the greedy approximation algorithm, our results shed new lights on the research of heuristic algorithms
title: Large-scale behavioral targeting
author: Ye Chen,Dmitry Pavlov,John F. Canny
abstract: Behavioral targeting (BT) leverages historical user behavior to select the ads most relevant to users to display. The state-of-the-art of BT derives a linear Poisson regression model from fine-grained user behavioral data and predicts click-through rate (CTR) from user history. We designed and implemented a highly scalable and efficient solution to BT using Hadoop MapReduce framework. With our parallel algorithm and the resulting system, we can build above 450 BT-category models from the entire Yahoo's user base within one day, the scale that one can not even imagine with prior systems. Moreover, our approach has yielded 20% CTR lift over the existing production system by leveraging the well-grounded probabilistic model fitted from a much larger training dataset Specifically, our major contributions include: (1) A MapReduce statistical learning algorithm and implementation that achieve optimal data parallelism, task parallelism, and load balance in spite of the typically skewed distribution of domain data. (2) An in-place feature vector generation algorithm with linear time complexity O(n) regardless of the granularity of sliding target window. (3) An in-memory caching scheme that significantly reduces the number of disk IOs to make large-scale learning practical. (4) Highly efficient data structures and sparse representations of models and data to enable fast model updates. We believe that our work makes significant contributions to solving large-scale machine learning problems of industrial relevance in general. Finally, we report comprehensive experimental results, using industrial proprietary codebase and datasets
title: On compressing social networks
author: Flavio Chierichetti,Ravi Kumar,Silvio Lattanzi,Michael Mitzenmacher,Alessandro Panconesi,Prabhakar Raghavan
abstract: Motivated by structural properties of the Web graph that support efficient data structures for in memory adjacency queries, we study the extent to which a large network can be compressed. Boldi and Vigna (WWW 2004), showed that Web graphs can be compressed down to three bits of storage per edge; we study the compressibility of social networks where again adjacency queries are a fundamental primitive. To this end, we propose simple combinatorial formulations that encapsulate efficient compressibility of graphs. We show that some of the problems are NP-hard yet admit effective heuristics, some of which can exploit properties of social networks such as link reciprocity. Our extensive experiments show that social networks and the Web graph exhibit vastly different compressibility characteristics
title: Regret-based online ranking for a growing digital library
author: Erick Delage
abstract: The most common environment in which ranking is used takes a very specific form. Users sequentially generate queries in a digital library. For each query, ranking is applied to order a set of relevant items from which the user selects his favorite. This is the case when ranking search results for pages on the World Wide Web or for merchandize on an e-commerce site. In this work, we present a new online ranking algorithm, called NoRegret KLRank. Our algorithm is designed to use "clickthrough" information as it is provided by the users to improve future ranking decisions. More importantly, we show that its long term average performance will converge to the best rate achievable by any competing fixed ranking policy selected with the benefit of hindsight. We show how to ensure that this property continues to hold as new items are added to the set thus requiring a richer class of ranking policies. Finally, our empirical results show that, while in some context NoRegret KLRank might be considered conservative, a greedy variant of this algorithm actually outperforms many popular ranking algorithms
title: A generalized Co-HITS algorithm and its application to bipartite graphs
author: Hongbo Deng,Michael R. Lyu,Irwin King
abstract: Recently many data types arising from data mining and Web search applications can be modeled as bipartite graphs. Examples include queries and URLs in query logs, and authors and papers in scientific literature. However, one of the issues is that previous algorithms only consider the content and link information from one side of the bipartite graph. There is a lack of constraints to make sure the final relevance of the score propagation on the graph, as there are many noisy edges within the bipartite graph. In this paper, we propose a novel and general Co-HITS algorithm to incorporate the bipartite graph with the content information from both sides as well as the constraints of relevance. Moreover, we investigate the algorithm based on two frameworks, including the iterative and the regularization frameworks, and illustrate the generalized Co-HITS algorithm from different views. For the iterative framework, it contains HITS and personalized PageRank as special cases. In the regularization framework, we successfully build a connection with HITS, and develop a new cost function to consider the direct relationship between two entity sets, which leads to a significant improvement over the baseline method. To illustrate our methodology, we apply the Co-HITS algorithm, with many different settings, to the application of query suggestion by mining the AOL query log data. Experimental results demonstrate that CoRegu-0.5 (i.e., a model of the regularization framework) achieves the best performance with consistent and promising improvements
title: Mining for the most certain predictions from dyadic data
author: Meghana Deodhar,Joydeep Ghosh
abstract: In several applications involving regression or classification, along with making predictions it is important to assess how accurate or reliable individual predictions are. This is particularly important in cases where due to finite resources or domain requirements, one wants to make decisions based only on the most reliable rather than on the entire set of predictions. This paper introduces novel and effective ways of ranking predictions by their accuracy for problems involving large-scale, heterogeneous data with a dyadic structure, i.e., where the independent variables can be naturally decomposed into three groups associated with two sets of elements and their combination. These approaches are based on modeling the data by a collection of localized models learnt while simultaneously partitioning (co-clustering) the data. For regression this leads to the concept of "certainty lift". We also develop a robust predictive modeling technique that identifies and models only the most coherent regions of the data to give high predictive accuracy on the selected subset of response values. Extensive experimentation on real life datasets highlights the utility of our proposed approaches
title: Efficiently learning the accuracy of labeling sources for selective sampling
author: Pinar Donmez,Jaime G. Carbonell,Jeff Schneider
abstract: Many scalable data mining tasks rely on active learning to provide the most useful accurately labeled instances. However, what if there are multiple labeling sources ('oracles' or 'experts') with different but unknown reliabilities? With the recent advent of inexpensive and scalable online annotation tools, such as Amazon's Mechanical Turk, the labeling process has become more vulnerable to noise - and without prior knowledge of the accuracy of each individual labeler. This paper addresses exactly such a challenge: how to jointly learn the accuracy of labeling sources and obtain the most informative labels for the active learning task at hand minimizing total labeling effort. More specifically, we present IEThresh (Interval Estimate Threshold) as a strategy to intelligently select the expert(s) with the highest estimated labeling accuracy. IEThresh estimates a confidence interval for the reliability of each expert and filters out the one(s) whose estimated upper-bound confidence interval is below a threshold - which jointly optimizes expected accuracy (mean) and need to better estimate the expert's accuracy (variance). Our framework is flexible enough to work with a wide range of different noise levels and outperforms baselines such as asking all available experts and random expert selection. In particular, IEThresh achieves a given level of accuracy with less than half the queries issued by all-experts labeling and less than a third the queries required by random expert selection on datasets such as the UCI mushroom one. The results show that our method naturally balances exploration and exploitation as it gains knowledge of which experts to rely upon, and selects them with increasing frequency
title: Large human communication networks: patterns and a utility-driven generator
author: Nan Du,Christos Faloutsos,Bai Wang,Leman Akoglu
abstract: Given a real, and weighted person-to-person network which changes over time, what can we say about the cliques that it contains? Do the incidents of communication, or weights on the edges of a clique follow any pattern? Real, and in-person social networks have many more triangles than chance would dictate. As it turns out, there are many more cliques than one would expect, in surprising patterns In this paper, we study massive real-world social networks formed by direct contacts among people through various personal communication services, such as Phone-Call, SMS, IM etc. The contributions are the following: (a) we discover surprising patterns with the cliques, (b) we report power-laws of the weights on the edges of cliques, (c) our real networks follow these patterns such that we can trust them to spot outliers and finally, (d) we propose the first utility-driven graph generator for weighted time-evolving networks, which match the observed patterns. Our study focused on three large datasets, each of which is a different type of communication service, with over one million records, and spans several months of activity
title: Learning with a non-exhaustive training dataset: a case study: detection of bacteria cultures using optical-scattering technology
author: M. Murat Dundar,E. Daniel Hirleman,Arun K. Bhunia,J. Paul Robinson,Bartek Rajwa
abstract: For a training dataset with a nonexhaustive list of classes, i.e. some classes are not yet known and hence are not represented, the resulting learning problem is ill-defined. In this case a sample from a missing class is incorrectly classified to one of the existing classes. For some applications the cost of misclassifying a sample could be negligible. However, the significance of this problem can better be acknowledged when the potentially undesirable consequences of incorrectly classifying a food pathogen as a nonpathogen are considered. Our research is directed towards the real-time detection of food pathogens using optical-scattering technology. Bacterial colonies consisting of the progeny of a single parent cell scatter light at 635 nm to produce unique forward-scatter signatures. These spectral signatures contain descriptive characteristics of bacterial colonies, which can be used to identify bacteria cultures in real time. One bottleneck that remains to be addressed is the nonexhaustivenature of the training library. It is very difficult if not impractical to collect samples from all possible bacteria colonies and construct a digital library with an exhaustive set of scatter signatures. This study deals with the real-time detection of samples from a missing class and the associated problem of learning with a nonexhaustive training dataset. Our proposed method assumes a common prior for the set of all classes, known and missing. The parameters of the prior are estimated from the samples of the known classes. This prior is then used to generate a large number of samples to simulate the space of missing classes. Finally a Bayesian maximum likelihood classifier is implemented using samples from real as well as simulated classes. Experiments performed with samples collected for 28 bacteria subclasses favor the proposed approach over the state of the art
title: Turning down the noise in the blogosphere
author: Khalid El-Arini,Gaurav Veda,Dafna Shahaf,Carlos Guestrin
abstract: In recent years, the blogosphere has experienced a substantial increase in the number of posts published daily, forcing users to cope with information overload. The task of guiding users through this flood of information has thus become critical. To address this issue, we present a principled approach for picking a set of posts that best covers the important stories in the blogosphere We define a simple and elegant notion of coverage and formalize it as a submodular optimization problem, for which we can efficiently compute a near-optimal solution. In addition, since people have varied interests, the ideal coverage algorithm should incorporate user preferences in order to tailor the selected posts to individual tastes. We define the problem of learning a personalized coverage function by providing an appropriate user-interaction model and formalizing an online learning framework for this task. We then provide a no-regret algorithm which can quickly learn a user's preferences fromlimited feedback We evaluate our coverage and personalization algorithms extensively over real blog data. Results from a user study show that our simple coverage algorithm does as well as most popular blog aggregation sites, including Google Blog Search, Yahoo! Buzz, and Digg. Furthermore, we demonstrate empirically that our algorithm can successfully adapt to user preferences. We believe that our technique, especially with personalization, can dramatically reduce information overload
title: Feature shaping for linear SVM classifiers
author: George Forman,Martin Scholz,Shyamsundar Rajaram
abstract: Linear classifiers have been shown to be effective for many discrimination tasks. Irrespective of the learning algorithm itself, the final classifier has a weight to multiply by each feature. This suggests that ideally each input feature should be linearly correlated with the target variable (or anti-correlated), whereas raw features may be highly non-linear. In this paper, we attempt to re-shape each input feature so that it is appropriate to use with a linear weight and to scale the different features in proportion to their predictive value. We demonstrate that this pre-processing is beneficial for linear SVM classifiers on a large benchmark of text classification tasks as well as UCI datasets
title: A multi-relational approach to spatial classification
author: Richard Frank,Martin Ester,Arno Knobbe
abstract: Spatial classification is the task of learning models to predict class labels based on the features of entities as well as the spatial relationships to other entities and their features. Spatial data can be represented as multi-relational data, however it presents novel challenges not present in multi-relational problems. One such problem is that spatial relationships are embedded in space, unknown a priori, and it is part of the algorithm's task to determine which relationships are important and what properties to consider. In order to determine when two entities are spatially related in an adaptive and non-parametric way, we propose a Voronoi-based neighbourhood definition upon which spatial literals can be built. Properties of these neighbourhoods also need to be described and used for classification purposes. Non-spatial aggregation literals already exist within the multi-relational framework, but are not sufficient for comprehensive spatial classification. A formal set of additions to themulti-relational data mining framework is proposed, to be able to represent spatial aggregations as well as spatial features and literals. These additions allow for capturing more complex interactions and spatial occurrences such as spatial trends. In order to more efficiently perform the rule learning and exploit powerful multi-processor machines, a scalable parallelized method capable of reducing the runtime by several factors is presented. The method is compared against existing methods by experimental evaluation on a real world crime dataset which demonstrate the importance of the neighbourhood definition and the advantages of parallelization
title: Scalable pseudo-likelihood estimation in hybrid random fields
author: Antonino Freno,Edmondo Trentin,Marco Gori
abstract: Learning probabilistic graphical models from high-dimensional datasets is a computationally challenging task. In many interesting applications, the domain dimensionality is such as to prevent state-of-the-art statistical learning techniques from delivering accurate models in reasonable time. This paper presents a hybrid random field model for pseudo-likelihood estimation in high-dimensional domains. A theoretical analysis proves that the class of pseudo-likelihood distributions representable by hybrid random fields strictly includes the class of joint probability distributions representable by Bayesian networks. In order to learn hybrid random fields from data, we develop the Markov Blanket Merging algorithm. Theoretical and experimental evidence shows that Markov Blanket Merging scales up very well to high-dimensional datasets. As compared to other widely used statistical learning techniques, Markov Blanket Merging delivers accurate results in a number of link prediction tasks, while achievingalso significant improvements in terms of computational efficiency Our software implementation of the models investigated in this paper is publicly available at http://www.dii.unisi.it/~freno/. The same website also hosts the datasets used in this work that are not available elsewhere in the same preprocessing used for our experiments
title: Issues in evaluation of stream learning algorithms
author: Joao Gama,Raquel Sebastiao,Pedro Pereira Rodrigues
abstract: Learning from data streams is a research area of increasing importance. Nowadays, several stream learning algorithms have been developed. Most of them learn decision models that continuously evolve over time, run in resource-aware environments, detect and react to changes in the environment generating data. One important issue, not yet conveniently addressed, is the design of experimental work to evaluate and compare decision models that evolve over time. There are no golden standards for assessing performance in non-stationary environments. This paper proposes a general framework for assessing predictive stream learning algorithms. We defend the use of Predictive Sequential methods for error estimate - the prequential error. The prequential error allows us to monitor the evolution of the performance of models that evolve over time. Nevertheless, it is known to be a pessimistic estimator in comparison to holdout estimates. To obtain more reliable estimators we need some forgetting mechanism. Two viable alternatives are: sliding windows and fading factors. We observe that the prequential error converges to an holdout estimator when estimated over a sliding window or using fading factors. We present illustrative examples of the use of prequential error estimators, using fading factors, for the tasks of: i) assessing performance of a learning algorithm; ii) comparing learning algorithms; iii) hypothesis testing using McNemar test; and iv) change detection using Page-Hinkley test. In these tasks, the prequential error estimated using fading factors provide reliable estimators. In comparison to sliding windows, fading factors are faster and memory-less, a requirement for streaming applications. This paper is a contribution to a discussion in the good-practices on performance assessment when learning dynamic models that evolve over time
title: Heterogeneous source consensus learning via decision propagation and negotiation
author: Jing Gao,Wei Fan,Yizhou Sun,Jiawei Han
abstract: Nowadays, enormous amounts of data are continuously generated not only in massive scale, but also from different, sometimes conflicting, views. Therefore, it is important to consolidate different concepts for intelligent decision making. For example, to predict the research areas of some people, the best results are usually achieved by combining and consolidating predictions obtained from the publication network, co-authorship network and the textual content of their publications. Multiple supervised and unsupervised hypotheses can be drawn from these information sources, and negotiating their differences and consolidating decisions usually yields a much more accurate model due to the diversity and heterogeneity of these models. In this paper, we address the problem of "consensus learning" among competing hypotheses, which either rely on outside knowledge (supervised learning) or internal structure (unsupervised clustering). We argue that consensus learning is an NP-hard problem and thus propose to solve it by an efficient heuristic method. We construct a belief graph to first propagate predictions from supervised models to the unsupervised, and then negotiate and reach consensus among them. Their final decision is further consolidated by calculating each model's weight based on its degree of consistency with other models. Experiments are conducted on 20 Newsgroups data, Cora research papers, DBLP author-conference network, and Yahoo! Movies datasets, and the results show that the proposed method improves the classification accuracy and the clustering quality measure (NMI) over the best base model by up to 10%. Furthermore, it runs in time proportional to the number of instances, which is very efficient for large scale data sets
title: Multi-focal learning and its application to customer service support
author: Yong Ge,Hui Xiong,Wenjun Zhou,Ramendra Sahoo,Xiaofeng Gao,Weili Wu
abstract: In this study, we formalize a multi-focal learning problem, where training data are partitioned into several different focal groups and the prediction model will be learned within each focal group. The multi-focal learning problem is motivated by numerous real-world learning applications. For instance, for the same type of problems encountered in a customer service center, the problem descriptions from different customers can be quite different. The experienced customers usually give more precise and focused descriptions about the problem. In contrast, the inexperienced customers usually provide more diverse descriptions. In this case, the examples from the same class in the training data can be naturally in different focal groups. As a result, it is necessary to identify those natural focal groups and exploit them for learning at different focuses. The key developmental challenge is how to identify those focal groups in the training data. As a case study, we exploit multi-focal learning for profiling problems in customer service centers. The results show that multifocal learning can significantly boost the learning accuracies of existing learning algorithms, such as Support Vector Machines (SVMs), for classifying customer problems
title: Co-clustering on manifolds
author: Quanquan Gu,Jie Zhou
abstract: Co-clustering is based on the duality between data points (e.g. documents) and features (e.g. words), i.e. data points can be grouped based on their distribution on features, while features can be grouped based on their distribution on the data points. In the past decade, several co-clustering algorithms have been proposed and shown to be superior to traditional one-side clustering. However, existing co-clustering algorithms fail to consider the geometric structure in the data, which is essential for clustering data on manifold. To address this problem, in this paper, we propose a Dual Regularized Co-Clustering (DRCC) method based on semi-nonnegative matrix tri-factorization. We deem that not only the data points, but also the features are sampled from some manifolds, namely data manifold and feature manifold respectively. As a result, we construct two graphs, i.e. data graph and feature graph, to explore the geometric structure of data manifold and feature manifold. Then our co-clustering method is formulated as semi-nonnegative matrix tri-factorization with two graph regularizers, requiring that the cluster labels of data points are smooth with respect to the data manifold, while the cluster labels of features are smooth with respect to the feature manifold. We will show that DRCC can be solved via alternating minimization, and its convergence is theoretically guaranteed. Experiments of clustering on many benchmark data sets demonstrate that the proposed method outperforms many state of the art clustering methods
title: Analyzing patterns of user content generation in online social networks
author: Lei Guo,Enhua Tan,Songqing Chen,Xiaodong Zhang,Yihong (Eric) Zhao
abstract: Various online social networks (OSNs) have been developed rapidly on the Internet. Researchers have analyzed different properties of such OSNs, mainly focusing on the formation and evolution of the networks as well as the information propagation over the networks. In knowledge-sharing OSNs, such as blogs and question answering systems, issues on how users participate in the network and how users "generate/contribute" knowledge are vital to the sustained and healthy growth of the networks. However, related discussions have not been reported in the research literature In this work, we empirically study workloads from three popular knowledge-sharing OSNs, including a blog system, a social bookmark sharing network, and a question answering social network to examine these properties. Our analysis consistently shows that (1) users' posting behavior in these networks exhibits strong daily and weekly patterns, but the user active time in these OSNs does not follow exponential distributions; (2)the user posting behavior in these OSNs follows stretched exponential distributions instead of power-law distributions, indicating the influence of a small number of core users cannot dominate the network; (3) the distributions of user contributions on high-quality and effort-consuming contents in these OSNs have smaller stretch factors for the stretched exponential distribution. Our study provides insights into user activity patterns and lays out an analytical foundation for further understanding various properties of these OSNs
title: Tell me something I don't know: randomization strategies for iterative data mining
author: Sami Hanhijarvi,Markus Ojala,Niko Vuokko,Kai Puolamaki,Nikolaj Tatti,Heikki Mannila
abstract: There is a wide variety of data mining methods available, and it is generally useful in exploratory data analysis to use many different methods for the same dataset. This, however, leads to the problem of whether the results found by one method are a reflection of the phenomenon shown by the results of another method, or whether the results depict in some sense unrelated properties of the data. For example, using clustering can give indication of a clear cluster structure, and computing correlations between variables can show that there are many significant correlations in the data. However, it can be the case that the correlations are actually determined by the cluster structure In this paper, we consider the problem of randomizing data so that previously discovered patterns or models are taken into account. The randomization methods can be used in iterative data mining. At each step in the data mining process, the randomization produces random samples from the set of data matrices satisfying the already discovered patterns or models. That is, given a data set and some statistics (e.g., cluster centers or co-occurrence counts) of the data, the randomization methods sample data sets having similar values of the given statistics as the original data set. We use Metropolis sampling based on local swaps to achieve this. We describe experiments on real data that demonstrate the usefulness of our approach. Our results indicate that in many cases, the results of, e.g., clustering actually imply the results of, say, frequent pattern discovery
title: Exploiting Wikipedia as external knowledge for document clustering
author: Xiaohua Hu,Xiaodan Zhang,Caimei Lu,E. K. Park,Xiaohua Zhou
abstract: In traditional text clustering methods, documents are represented as "bags of words" without considering the semantic information of each document. For instance, if two documents use different collections of core words to represent the same topic, they may be falsely assigned to different clusters due to the lack of shared core words, although the core words they use are probably synonyms or semantically associated in other forms. The most common way to solve this problem is to enrich document representation with the background knowledge in an ontology. There are two major issues for this approach: (1) the coverage of the ontology is limited, even for WordNet or Mesh, (2) using ontology terms as replacement or additional features may cause information loss, or introduce noise. In this paper, we present a novel text clustering method to address these two issues by enriching document representation with Wikipedia concept and category information. We develop two approaches, exact match and relatedness-match, to map text documents to Wikipedia concepts, and further to Wikipedia categories. Then the text documents are clustered based on a similarity metric which combines document content information, concept information as well as category information. The experimental results using the proposed clustering framework on three datasets (20-newsgroup, TDT2, and LA Times) show that clustering performance improves significantly by enriching document representation with Wikipedia concepts and categories
title: TrustWalker: a random walk model for combining trust-based and item-based recommendation
author: Mohsen Jamali,Martin Ester
abstract: Collaborative filtering is the most popular approach to build recommender systems and has been successfully employed in many applications. However, it cannot make recommendations for so-called cold start users that have rated only a very small number of items. In addition, these methods do not know how confident they are in their recommendations. Trust-based recommendation methods assume the additional knowledge of a trust network among users and can better deal with cold start users, since users only need to be simply connected to the trust network. On the other hand, the sparsity of the user item ratings forces the trust-based approach to consider ratings of indirect neighbors that are only weakly trusted, which may decrease its precision. In order to find a good trade-off, we propose a random walk model combining the trust-based and the collaborative filtering approach for recommendation. The random walk model allows us to define and to measure the confidence of a recommendation. We performed an evaluation on the Epinions dataset and compared our model with existing trust-based and collaborative filtering methods
title: Drosophila gene expression pattern annotation using sparse features and term-term interactions
author: Shuiwang Ji,Lei Yuan,Ying-Xin Li,Zhi-Hua Zhou,Sudhir Kumar,Jieping Ye
abstract: The Drosophila gene expression pattern images document the spatial and temporal dynamics of gene expression and they are valuable tools for explicating the gene functions, interaction, and networks during Drosophila embryogenesis. To provide text-based pattern searching, the images in the Berkeley Drosophila Genome Project (BDGP) study are annotated with ontology terms manually by human curators. We present a systematic approach for automating this task, because the number of images needing text descriptions is now rapidly increasing. We consider both improved feature representation and novel learning formulation to boost the annotation performance. For feature representation, we adapt the bag-of-words scheme commonly used in visual recognition problems so that the image group information in the BDGP study is retained. Moreover, images from multiple views can be integrated naturally in this representation. To reduce the quantization error caused by the bag-of-words representation, we propose an improved feature representation scheme based on the sparse learning technique. In the design of learning formulation, we propose a local regularization framework that can incorporate the correlations among terms explicitly. We further show that the resulting optimization problem admits an analytical solution. Experimental results show that the representation based on sparse learning outperforms the bag-of-words representation significantly. Results also show that incorporation of the term-term correlations improves the annotation performance consistently
title: Cartesian contour: a concise representation for a collection of frequent sets
author: Ruoming Jin,Yang Xiang,Lin Liu
abstract: In this paper, we consider a novel scheme referred to as Cartesian contour to concisely represent the collection of frequent itemsets. Different from the existing works, this scheme provides a complete view of these itemsets by covering the entire collection of them. More interestingly, it takes a first step in deriving a generative view of the frequent pattern formulation, i.e., how a small number of patterns interact with each other and produce the complexity of frequent itemsets. We perform a theoretical investigation of the concise representation problem and link it to the biclique set cover problem and prove its NP-hardness. We develop a novel approach utilizing the technique developed in frequent itemset mining, set cover, and max k-cover to approximate the minimal biclique set cover problem. In addition, we consider several heuristic techniques to speedup the construction of Cartesian contour. The detailed experimental study demonstrates the effectiveness and efficiency of our approach
title: Genre-based decomposition of email class noise
author: Aleksander Kolcz,Gordon V. Cormack
abstract: Corruption of data by class-label noise is an important practical concern impacting many classification problems. Studies of data cleaning techniques often assume a uniform label noise model, however, which is seldom realized in practice. Relatively little is understood, as to how the natural label noise distribution can be measured or simulated. Using email spam-filtering data, we demonstrate that class noise can have substantial content specific bias. We also demonstrate that noise detection techniques based on classifier confidence tend to identify instances that human assessors are likely to label in error. We show that genre modeling can be very informative in identifying potential areas of mislabeling. Moreover, we are able to show that genre decomposition can also be used to substantially improve spam filtering accuracy, with our results outperforming the best published figures for the trec05-p1 and ceas-2008 benchmark collections
title: Characteristic relational patterns
author: Arne Koopman,Arno Siebes
abstract: Research in relational data mining has two major directions: finding global models of a relational database and the discovery of local relational patterns within a database. While relational patterns show how attribute values co-occur in detail, their huge numbers hamper their usage in data analysis. Global models, on the other hand, only provide a summary of how different tables and their attributes relate to each other, lacking detail of what is going on at the local level In this paper we introduce a new approach that combines the positive properties of both directions: it provides a detailed description of the complete database using a small set of patterns. More in particular, we utilise a rich pattern language and show how a database can be encoded by such patterns. Then, based on the MDLprinciple, the novel RDB-KRIMP algorithm selects the set of patterns that allows for the most succinct encoding of the database. This set, the code table, is a compact description of the databasein terms of local relational patterns. We show that this resulting set is very small, both in terms of database size and in number of its local relational patterns: a reduction of up to 4 orders of magnitude is attained
title: Collaborative filtering with temporal dynamics
author: Yehuda Koren
abstract: Customer preferences for products are drifting over time. Product perception and popularity are constantly changing as new selection emerges. Similarly, customer inclinations are evolving, leading them to ever redefine their taste. Thus, modeling temporal dynamics should be a key when designing recommender systems or general customer preference models. However, this raises unique challenges. Within the eco-system intersecting multiple products and customers, many different characteristics are shifting simultaneously, while many of them influence each other and often those shifts are delicate and associated with a few data instances. This distinguishes the problem from concept drift explorations, where mostly a single concept is tracked. Classical time-window or instance-decay approaches cannot work, as they lose too much signal when discarding data instances. A more sensitive approach is required, which can make better distinctions between transient effects and long term patterns. The paradigmwe offer is creating a model tracking the time changing behavior throughout the life span of the data. This allows us to exploit the relevant components of all data instances, while discarding only what is modeled as being irrelevant. Accordingly, we revamp two leading collaborative filtering recommendation approaches. Evaluation is made on a large movie rating dataset by Netflix. Results are encouraging and better than those previously reported on this dataset
title: Collective annotation of Wikipedia entities in web text
author: Sayali Kulkarni,Amit Singh,Ganesh Ramakrishnan,Soumen Chakrabarti
abstract: To take the first step beyond keyword-based search toward entity-based search, suitable token spans ("spots") on documents must be identified as references to real-world entities from an entity catalog. Several systems have been proposed to link spots on Web pages to entities in Wikipedia. They are largely based on local compatibility between the text around the spot and textual metadata associated with the entity. Two recent systems exploit inter-label dependencies, but in limited ways. We propose a general collective disambiguation approach. Our premise is that coherent documents refer to entities from one or a few related topics or domains. We give formulations for the trade-off between local spot-to-entity compatibility and measures of global coherence between entities. Optimizing the overall entity assignment is NP-hard. We investigate practical solutions based on local hill-climbing, rounding integer linear programs, and pre-clustering entities followed by local optimization within clusters. In experiments involving over a hundred manually-annotated Web pages and tens of thousands of spots, our approaches significantly outperform recently-proposed algorithms
title: Finding a team of experts in social networks
author: Theodoros Lappas,Kun Liu,Evimaria Terzi
abstract: Given a task T, a pool of individuals X with different skills, and a social network G that captures the compatibility among these individuals, we study the problem of finding X, a subset of X, to perform the task. We call this the TEAM FORMATION problem. We require that members of X' not only meet the skill requirements of the task, but can also work effectively together as a team. We measure effectiveness using the communication cost incurred by the subgraph in G that only involves X'. We study two variants of the problem for two different communication-cost functions, and show that both variants are NP-hard. We explore their connections with existing combinatorial problems and give novel algorithms for their solution. To the best of our knowledge, this is the first work to consider the TEAM FORMATION problem in the presence of a social network of individuals. Experiments on the DBLP dataset showthat our framework works well in practice and gives useful and intuitive results
title: On burstiness-aware search for document sequences
author: Theodoros Lappas,Benjamin Arai,Manolis Platakis,Dimitrios Kotsakos,Dimitrios Gunopulos
abstract: As the number and size of large timestamped collections (e.g. sequences of digitized newspapers, periodicals, blogs) increase, the problem of efficiently indexing and searching such data becomes more important. Term burstiness has been extensively researched as a mechanism to address event detection in the context of such collections. In this paper, we explore how burstiness information can be further utilized to enhance the search process. We present a novel approach to model the burstiness of a term, using discrepancy theory concepts. This allows us to build a parameter-free, linear-time approach to identify the time intervals of maximum burstiness for a given term. Finally, we describe the first burstiness-driven search framework and thoroughly evaluate our approach in the context of different scenarios
title: Improving data mining utility with projective sampling
author: Mark Last
abstract: Overall performance of the data mining process depends not just on the value of the induced knowledge but also on various costs of the process itself such as the cost of acquiring and pre-processing training examples, the CPU cost of model induction, and the cost of committed errors. Recently, several progressive sampling strategies for maximizing the overall data mining utility have been proposed. All these strategies are based on repeated acquisitions of additional training examples until a utility decrease is observed. In this paper, we present an alternative, projective sampling strategy, which fits functions to a partial learning curve and a partial run-time curve obtained from a small subset of potentially available data and then uses these projected functions to analytically estimate the optimal training set size. The proposed approach is evaluated on a variety of benchmark datasets using the RapidMiner environment for machine learning and data mining processes. The results show that thelearning and run-time curves projected from only several data points can lead to a cheaper data mining process than the common progressive sampling methods
title: Meme-tracking and the dynamics of the news cycle
author: Jure Leskovec,Lars Backstrom,Jon Kleinberg
abstract: Tracking new topics, ideas, and "memes" across the Web has been an issue of considerable interest. Recent work has developed methods for tracking topic shifts over long time scales, as well as abrupt spikes in the appearance of particular named entities. However, these approaches are less well suited to the identification of content that spreads widely and then fades over time scales on the order of days - the time scale at which we perceive news and events We develop a framework for tracking short, distinctive phrases that travel relatively intact through on-line text; developing scalable algorithms for clustering textual variants of such phrases, we identify a broad class of memes that exhibit wide spread and rich variation on a daily basis. As our principal domain of study, we show how such a meme-tracking approach can provide a coherent representation of the news cycle - the daily rhythms in the news media that have long been the subject of qualitative interpretation but have neverbeen captured accurately enough to permit actual quantitative analysis. We tracked 1.6 million mainstream media sites and blogs over a period of three months with the total of 90 million articles and we find a set of novel and persistent temporal patterns in the news cycle. In particular, we observe a typical lag of 2.5 hours between the peaks of attention to a phrase in the news media and in blogs respectively, with divergent behavior around the overall peak and a "heartbeat"-like pattern in the handoff between news and blogs. We also develop and analyze a mathematical model for the kinds of temporal variation that the system exhibits
title: DynaMMo: mining and summarization of coevolving sequences with missing values
author: Lei Li,James McCann,Nancy S. Pollard,Christos Faloutsos
abstract: Given multiple time sequences with missing values, we propose DynaMMo which summarizes, compresses, and finds latent variables. The idea is to discover hidden variables and learn their dynamics, making our algorithm able to function even when there are missing values We performed experiments on both real and synthetic datasets spanning several megabytes, including motion capture sequences and chlorine levels in drinking water. We show that our proposed DynaMMo method (a) can successfully learn the latent variables and their evolution; (b) can provide high compression for little loss of reconstruction accuracy; (c) can extract compact but powerful features for segmentation, interpretation, and forecasting; (d) has complexity linear on the duration of sequences
title: On the tradeoff between privacy and utility in data publishing
author: Tiancheng Li,Ninghui Li
abstract: In data publishing, anonymization techniques such as generalization and bucketization have been designed to provide privacy protection. In the meanwhile, they reduce the utility of the data. It is important to consider the tradeoff between privacy and utility. In a paper that appeared in KDD 2008, Brickell and Shmatikov proposed an evaluation methodology by comparing privacy gain with utility gain resulted from anonymizing the data, and concluded that "even modest privacy gains require almost complete destruction of the data-mining utility". This conclusion seems to undermine existing work on data anonymization. In this paper, we analyze the fundamental characteristics of privacy and utility, and show that it is inappropriate to directly compare privacy with utility. We then observe that the privacy-utility tradeoff in data publishing is similar to the risk-return tradeoff in financial investment, and propose an integrated framework for considering privacy-utility tradeoff, borrowing concepts from the Modern Portfolio Theory for financial investment. Finally, we evaluate our methodology on the Adult dataset from the UCI machine learning repository. Our results clarify several common misconceptions about data utility and provide data publishers useful guidelines on choosing the right tradeoff between privacy and utility
title: MetaFac: community discovery via relational hypergraph factorization
author: Yu-Ru Lin,Jimeng Sun,Paul Castro,Ravi Konuru,Hari Sundaram,Aisling Kelliher
abstract: This paper aims at discovering community structure in rich media social networks, through analysis of time-varying, multi-relational data. Community structure represents the latent social context of user actions. It has important applications in information tasks such as search and recommendation. Social media has several unique challenges. (a) In social media, the context of user actions is constantly changing and co-evolving; hence the social context contains time-evolving multi-dimensional relations. (b) The social context is determined by the available system features and is unique in each social media website. In this paper we propose MetaFac (MetaGraph Factorization), a framework that extracts community structures from various social contexts and interactions. Our work has three key contributions: (1) metagraph, a novel relational hypergraph representation for modeling multi-relational and multi-dimensional social data; (2) an efficient factorization method for community extractionon a given metagraph; (3) an on-line method to handle time-varying relations through incremental metagraph factorization. Extensive experiments on real-world social data collected from the Digg social media website suggest that our technique is scalable and is able to extract meaningful communities based on the social media contexts. We illustrate the usefulness of our framework through prediction tasks. We outperform baseline methods (including aspect model and tensor analysis) by an order of magnitude
title: BBM: bayesian browsing model from petabyte-scale data
author: Chao Liu,Fan Guo,Christos Faloutsos
abstract: Given a quarter of petabyte click log data, how can we estimate the relevance of each URL for a given query? In this paper, we propose the Bayesian Browsing Model (BBM), a new modeling technique with following advantages: (a) it does exact inference; (b) it is single-pass and parallelizable; (c) it is effective We present two sets of experiments to test model effectiveness and efficiency. On the first set of over 50 million search instances of 1.1 million distinct queries, BBM out-performs the state-of-the-art competitor by 29.2% in log-likelihood while being 57 times faster. On the second click-log set, spanning a quarter of petabyte data, we showcase the scalability of BBM: we implemented it on a commercial MapReduce cluster, and it took only 3 hours to compute the relevance for 1.15 billion distinct query-URL pairs
title: Large-scale sparse logistic regression
author: Jun Liu,Jianhui Chen,Jieping Ye
abstract: Logistic Regression is a well-known classification method that has been used widely in many applications of data mining, machine learning, computer vision, and bioinformatics. Sparse logistic regression embeds feature selection in the classification framework using the l1-norm regularization, and is attractive in many applications involving high-dimensional data. In this paper, we propose Lassplore for solving large-scale sparse logistic regression. Specifically, we formulate the problem as the l1-ball constrained smooth convex optimization, and propose to solve the problem using the Nesterov's method, an optimal first-order black-box method for smooth convex optimization. One of the critical issues in the use of the Nesterov's method is the estimation of the step size at each of the optimization iterations. Previous approaches either applies the constant step size which assumes that the Lipschitz gradient is known in advance, or requires a sequence of decreasing step size which leads to slow convergence in practice. In this paper, we propose an adaptive line search scheme which allows to tune the step size adaptively and meanwhile guarantees the optimal convergence rate. Empirical comparisons with several state-of-the-art algorithms demonstrate the efficiency of the proposed Lassplore algorithm for large-scale problems
title: Classification of software behaviors for failure detection: a discriminative pattern mining approach
author: David Lo,Hong Cheng,Jiawei Han,Siau-Cheng Khoo,Chengnian Sun
abstract: Software is a ubiquitous component of our daily life. We often depend on the correct working of software systems. Due to the difficulty and complexity of software systems, bugs and anomalies are prevalent. Bugs have caused billions of dollars loss, in addition to privacy and security threats. In this work, we address software reliability issues by proposing a novel method to classify software behaviors based on past history or runs. With the technique, it is possible to generalize past known errors and mistakes to capture failures and anomalies. Our technique first mines a set of discriminative features capturing repetitive series of events from program execution traces. It then performs feature selection to select the best features for classification. These features are then used to train a classifier to detect failures. Experiments and case studies on traces of several benchmark software systems and a real-life concurrency bug from MySQL server show the utility of the technique in capturing failures and anomalies. On average, our pattern-based classification technique outperforms the baseline approach by 24.68% in accuracy\
title: Consensus group stable feature selection
author: Steven Loscalzo,Lei Yu,Chris Ding
abstract: Stability is an important yet under-addressed issue in feature selection from high-dimensional and small sample data. In this paper, we show that stability of feature selection has a strong dependency on sample size. We propose a novel framework for stable feature selection which first identifies consensus feature groups from subsampling of training samples, and then performs feature selection by treating each consensus feature group as a single entity. Experiments on both synthetic and real-world data sets show that an algorithm developed under this framework is effective at alleviating the problem of small sample size and leads to more stable feature selection results and comparable or better generalization performance than state-of-the-art feature selection algorithms. Synthetic data sets and algorithm source code are available at http://www.cs.binghamton.edu/~lyu/KDD09/
title: Grouped graphical Granger modeling methods for temporal causal modeling
author: Aurelie C. Lozano,Naoki Abe,Yan Liu,Saharon Rosset
abstract: We develop and evaluate an approach to causal modeling based on time series data, collectively referred to as "grouped graphical Granger modeling methods." Graphical Granger modeling uses graphical modeling techniques on time series data and invokes the notion of "Granger causality" to make assertions on causality among a potentially large number of time series variables through inference on time-lagged effects. The present paper proposes a novel enhancement to the graphical Granger methodology by developing and applying families of regression methods that are sensitive to group information among variables, to leverage the group structure present in the lagged temporal variables according to the time series they belong to. Additionally, we propose a new family of algorithms we call group boosting, as an improved component of grouped graphical Granger modeling over the existing regression methods with grouped variable selection in the literature (e.g group Lasso). The introduction of group boosting methods is primarily motivated by the need to deal with non-linearity in the data. We perform empirical evaluation to confirm the advantage of the grouped graphical Granger methods over the standard (non-grouped) methods, as well as that specific to the methods based on group boosting. This advantage is also demonstrated for the real world application of gene regulatory network discovery from time-course microarray data
title: Spatial-temporal causal modeling for climate change attribution
author: Aurelie C. Lozano,Hongfei Li,Alexandru Niculescu-Mizil,Yan Liu,Claudia Perlich,Jonathan Hosking,Naoki Abe
abstract: Attribution of climate change to causal factors has been based predominantly on simulations using physical climate models, which have inherent limitations in describing such a complex and chaotic system. We propose an alternative, data centric, approach that relies on actual measurements of climate observations and human and natural forcing factors. Specifically, we develop a novel method to infer causality from spatial-temporal data, as well as a procedure to incorporate extreme value modeling into our method in order to address the attribution of extreme climate events, such as heatwaves. Our experimental results on a real world dataset indicate that changes in temperature are not solely accounted for by solar radiance, but attributed more significantly to CO2 and other greenhouse gases. Combined with extreme value modeling, we also show that there has been a significant increase in the intensity of extreme temperatures, and that such changes in extreme temperature are also attributable to greenhouse gases. These preliminary results suggest that our approach can offer a useful alternative to the simulation-based approach to climate modeling and attribution, and provide valuable insights from a fresh perspective
title: Using graph-based metrics with empirical risk minimization to speed up active learning on networked data
author: Sofus A. Macskassy
abstract: Active and semi-supervised learning are important techniques when labeled data are scarce. Recently a method was suggested for combining active learning with a semi-supervised learning algorithm that uses Gaussian fields and harmonic functions. This classifier is relational in nature: it relies on having the data presented as a partially labeled graph (also known as a within-network learning problem). This work showed yet again that empirical risk minimization (ERM) was the best method to find the next instance to label and provided an efficient way to compute ERM with the semi-supervised classifier. The computational problem with ERM is that it relies on computing the risk for all possible instances. If we could limit the candidates that should be investigated, then we can speed up active learning considerably. In the case where the data is graphical in nature, we can leverage the graph structure to rapidly identify instances that are likely to be good candidates for labeling. This paper describes a novel hybrid approach of using of community finding and social network analytic centrality measures to identify good candidates for labeling and then using ERM to find the best instance in this candidate set. We show on real-world data that we can limit the ERM computations to a fraction of instances with comparable performance
title: Characterizing individual communication patterns
author: R. Dean Malmgren,Jake M. Hofman,Luis A.N. Amaral,Duncan J. Watts
abstract: The increasing availability of electronic communication data, such as that arising from e-mail exchange, presents social and information scientists with new possibilities for characterizing individual behavior and, by extension, identifying latent structure in human populations. Here, we propose a model of individual e-mail communication that is sufficiently rich to capture meaningful variability across individuals, while remaining simple enough to be interpretable. We show that the model, a cascading non-homogeneous Poisson process, can be formulated as a double-chain hidden Markov model, allowing us to use an efficient inference algorithm to estimate the model parameters from observed data. We then apply this model to two e-mail data sets consisting of 404 and 6,164 users, respectively, that were collected from two universities in different countries and years. We find that the resulting best-estimate parameter distributions for both data sets are surprisingly similar, indicating that at least some features of communication dynamics generalize beyond specific contexts. We also find that variability of individual behavior over time is significantly less than variability across the population, suggesting that individuals can be classified into persistent "types". We conclude that communication patterns may prove useful as an additional class of attribute data, complementing demographic and network data, for user classification and outlier detection-a point that we illustrate with an interpretable clustering of users based on their inferred model parameters
title: Large-scale graph mining using backbone refinement classes
author: Andreas Maunz,Christoph Helma,Stefan Kramer
abstract: We present a new approach to large-scale graph mining based on so-called backbone refinement classes. The method efficiently mines tree-shaped subgraph descriptors under minimum frequency and significance constraints, using classes of fragments to reduce feature set size and running times. The classes are defined in terms of fragments sharing a common backbone. The method is able to optimize structural inter-feature entropy as opposed to occurrences, which is characteristic for open or closed fragment mining. In the experiments, the proposed method reduces feature set sizes by >90 % and >30 % compared to complete tree mining and open tree mining, respectively. Evaluation using crossvalidation runs shows that their classification accuracy is similar to the complete set of trees but significantly better than that of open trees. Compared to open or closed fragment mining, a large part of the search space can be pruned due to an improved statistical constraint (dynamic upper bound adjustment), which is also confirmed in the experiments in lower running times compared to ordinary (static) upper bound pruning. Further analysis using large-scale datasets yields insight into important properties of the proposed descriptors, such as the dataset coverage and the class size represented by each descriptor. A final cross-validation run confirms that the novel descriptors render large training sets feasible which previously might have been intractable
title: Differentially private recommender systems: building privacy into the net
author: Frank McSherry,Ilya Mironov
abstract: We consider the problem of producing recommendations from collective user behavior while simultaneously providing guarantees of privacy for these users. Specifically, we consider the Netflix Prize data set, and its leading algorithms, adapted to the framework of differential privacy Unlike prior privacy work concerned with cryptographically securing the computation of recommendations, differential privacy constrains a computation in a way that precludes any inference about the underlying records from its output. Such algorithms necessarily introduce uncertainty--i.e., noise--to computations, trading accuracy for privacy We find that several of the leading approaches in the Netflix Prize competition can be adapted to provide differential privacy, without significantly degrading their accuracy. To adapt these algorithms, we explicitly factor them into two parts, an aggregation/learning phase that can be performed with differential privacy guarantees, and an individual recommendation phase that uses the learned correlations and an individual's data to provide personalized recommendations. The adaptations are non-trivial, and involve both careful analysis of the per-record sensitivity of the algorithms to calibrate noise, as well as new post-processing steps to mitigate the impact of this noise We measure the empirical trade-off between accuracy and privacy in these adaptations, and find that we can provide non-trivial formal privacy guarantees while still outperforming the Cinematch baseline Netflix provides
title: WhereNext: a location predictor on trajectory pattern mining
author: Anna Monreale,Fabio Pinelli,Roberto Trasarti,Fosca Giannotti
abstract: The pervasiveness of mobile devices and location based services is leading to an increasing volume of mobility data.This side eect provides the opportunity for innovative methods that analyse the behaviors of movements. In this paper we propose WhereNext, which is a method aimed at predicting with a certain level of accuracy the next location of a moving object. The prediction uses previously extracted movement patterns named Trajectory Patterns, which are a concise representation of behaviors of moving objects as sequences of regions frequently visited with a typical travel time. A decision tree, named T-pattern Tree, is built and evaluated with a formal training and test process. The tree is learned from the Trajectory Patterns that hold a certain area and it may be used as a predictor of the next location of a new trajectory finding the best matching path in the tree. Three dierent best matching methods to classify a new moving object are proposed and their impacton the quality of prediction is studied extensively. Using Trajectory Patterns as predictive rules has the following implications: (I) the learning depends on the movement of all available objects in a certain area instead of on the individual history of an object; (II) the prediction tree intrinsically contains the spatio-temporal properties that have emerged from the data and this allows us to define matching methods that striclty depend on the properties of such movements. In addition, we propose a set of other measures, that evaluate a priori the predictive power of a set of Trajectory Patterns. This measures were tuned on a real life case study. Finally, an exhaustive set of experiments and results on the real dataset are presented
title: Correlated itemset mining in ROC space: a constraint programming approach
author: Siegfried Nijssen,Tias Guns,Luc De Raedt
abstract: Correlated or discriminative pattern mining is concerned with finding the highest scoring patterns w.r.t. a correlation measure (such as information gain). By reinterpreting correlation measures in ROC space and formulating correlated itemset mining as a constraint programming problem, we obtain new theoretical insights with practical benefits. More specifically, we contribute 1) an improved bound for correlated itemset miners, 2) a novel iterative pruning algorithm to exploit the bound, and 3) an adaptation of this algorithm to mine all itemsets on the convex hull in ROC space. The algorithm does not depend on a minimal frequency threshold and is shown to outperform several alternative approaches by orders of magnitude, both in runtime and in memory requirements
title: TANGENT: a novel, 'Surprise me', recommendation algorithm
author: Kensuke Onuma,Hanghang Tong,Christos Faloutsos
abstract: Most of recommender systems try to find items that are most relevant to the older choices of a given user. Here we focus on the "surprise me" query: A user may be bored with his/her usual genre of items (e.g., books, movies, hobbies), and may want a recommendation that is related, but off the beaten path, possibly leading to a new genre of books/movies/hobbies How would we define, as well as automate, this seemingly selfcontradicting request? We introduce TANGENT, a novel recommendation algorithm to solve this problem. The main idea behind TANGENT is to envision the problem as node selection on a graph, giving high scores to nodes that are well connected to the older choices, and at the same time well connected to unrelated choices. The method is carefully designed to be (a) parameter-free (b) effective and (c) fast. We illustrate the benefits of TANGENT with experiments on both synthetic and real data sets. We show that TANGENT makes reasonable, yet surprising, horizon-broadening recommendations. Moreover, it is fast and scalable, since it can easily use existing fast algorithms on graph node proximity
title: Mind the gaps: weighting the unknown in large-scale one-class collaborative filtering
author: Rong Pan,Martin Scholz
abstract: One-Class Collaborative Filtering (OCCF) is a task that naturally emerges in recommender system settings. Typical characteristics include: Only positive examples can be observed, classes are highly imbalanced, and the vast majority of data points are missing. The idea of introducing weights for missing parts of a matrix has recently been shown to help in OCCF. While existing weighting approaches mitigate the first two problems above, a sparsity preserving solution that would allow to efficiently utilize data sets with e.g., hundred thousands of users and items has not yet been reported. In this paper, we study three different collaborative filtering frameworks: Low-rank matrix approximation, probabilistic latent semantic analysis, and maximum-margin matrix factorization. We propose two novel algorithms for large-scale OCCF that allow to weight the unknowns. Our experimental results demonstrate their effectiveness and efficiency on different problems, including the Netflix Prize data
title: An association analysis approach to biclustering
author: Gaurav Pandey,Gowtham Atluri,Michael Steinbach,Chad L. Myers,Vipin Kumar
abstract: The discovery of biclusters, which denote groups of items that show coherent values across a subset of all the transactions in a data set, is an important type of analysis performed on real-valued data sets in various domains, such as biology. Several algorithms have been proposed to find different types of biclusters in such data sets. However, these algorithms are unable to search the space of all possible biclusters exhaustively. Pattern mining algorithms in association analysis also essentially produce biclusters as their result, since the patterns consist of items that are supported by a subset of all the transactions. However, a major limitation of the numerous techniques developed in association analysis is that they are only able to analyze data sets with binary and/or categorical variables, and their application to real-valued data sets often involves some lossy transformation such as discretization or binarization of the attributes. In this paper, we propose a novel association analysis framework for exhaustively and efficiently mining "range support" patterns from such a data set. On one hand, this framework reduces the loss of information incurred by the binarization- and discretization-based approaches, and on the other, it enables the exhaustive discovery of coherent biclusters. We compared the performance of our framework with two standard biclustering algorithms through the evaluation of the similarity of the cellular functions of the genes constituting the patterns/biclusters derived by these algorithms from microarray data. These experiments show that the real-valued patterns discovered by our framework are better enriched by small biologically interesting functional classes. Also, through specific examples, we demonstrate the ability of the RAP framework to discover functionally enriched patterns that are not found by the commonly used biclustering algorithm ISA. The source code and data sets used in this paper, as well as the supplementary material, are available at http://www.cs.umn.edu/vk/gaurav/rap
title: CP-summary: a concise representation for browsing frequent itemsets
author: Ardian Kristanto Poernomo,Vivekanand Gopalkrishnan
abstract: This paper tackles the problem of summarizing frequent itemsets. We observe that previous notions of summaries cannot be directly used for analyzing frequent itemsets. In order to be used for analysis, one requirement is that the analysts should be able to browse all frequent itemsets by only having the summary For this purpose, we propose to build the summary based upon a novel formulation, conditional profile (or c-profile). Several features of our proposed summary are: (1) each profile in the summary can be analyzed independently, (2) it provides error guarantee (-adequate), and (3) it produces no false positives or false negativesHaving the formulation, the next challenge is to produce the most concise summary which satisfies the requirement. In this paper, we also designed an algorithm which is both effective and efficient for this task. The quality of our approach is justified by extensive experiments The implementations for the algorithms areavailable from www.cais.ntu.edu.sg/~vivek/pubs/cprofile09
title: Towards efficient mining of proportional fault-tolerant frequent itemsets
author: Ardian Kristanto Poernomo,Vivekanand Gopalkrishnan
abstract: Fault-tolerant frequent itemsets (FTFI) are variants of frequent itemsets for representing and discovering generalized knowledge. However, despite growing interest in this field, no previous approach mines proportional FTFIs with their exact support (FT-support) This problem is difficult because of two concerns: (a) non anti-monotonic property of FT-support when relaxation is proportional, and (b) difficulty in computing FT-support. Previous efforts on this problem either simplify the general problem by adding constraints, or provide approximate solutions without any error guarantees In this paper, we address these concerns in the general FTFI mining problem. We limit the search space by providing provably correct anti monotone bounds for FT-support and develop practically efficient means of achieving them. Besides, we also provide an efficient and exact FT-support counting procedure Extensive experiments using real datasets validate that our solution is reasonably efficient for completely mining FTFIs. Implementations for the algorithms are available from www.cais.ntu.edu.sg/~vivek/pubs/ftfim09
title: Audience selection for on-line brand advertising: privacy-friendly social network targeting
author: Foster Provost,Brian Dalessandro,Rod Hook,Xiaohan Zhang,Alan Murray
abstract: This paper describes and evaluates privacy-friendly methods for extracting quasi-social networks from browser behavior on user-generated content sites, for the purpose of finding good audiences for brand advertising (as opposed to click maximizing, for example). Targeting social-network neighbors resonates well with advertisers, and on-line browsing behavior data counterintuitively can allow the identification of good audiences anonymously. Besides being one of the first papers to our knowledge on data mining for on-line brand advertising, this paper makes several important contributions. We introduce a framework for evaluating brand audiences, in analogy to predictive-modeling holdout evaluation. We introduce methods for extracting quasi-social networks from data on visitations to social networking pages, without collecting any information on the identities of the browsers or the content of the social-network pages. We introduce measures of brand proximity in the network, and show that audiences with high brand proximity indeed show substantially higher brand affinity. Finally, we provide evidence that the quasi-social network embeds a true social network, which along with results from social theory offers one explanation for the increase in brand affinity of the selected audiences
title: A principled and flexible framework for finding alternative clusterings
author: ZiJie Qi,Ian Davidson
abstract: The aim of data mining is to find novel and actionable insights in data. However, most algorithms typically just find a single (possibly non-novel/actionable) interpretation of the data even though alternatives could exist. The problem of finding an alternative to a given original clustering has received little attention in the literature. Current techniques (including our previous work) are unfocused/unrefined in that they broadly attempt to find an alternative clustering but do not specify which properties of the original clustering should or should not be retained. In this work, we explore a principled and flexible framework in order to find alternative clusterings of the data. The approach is principled since it poses a constrained optimization problem, so its exact behavior is understood. It is flexible since the user can formally specify positive and negative feedback based on the existing clustering, which ranges from which clusters to keep (or not) to making a trade-off between alternativeness and clustering quality
title: Learning optimal ranking with tensor factorization for tag recommendation
author: Steffen Rendle,Leandro Balby Marinho,Alexandros Nanopoulos,Lars Schmidt-Thieme
abstract: Tag recommendation is the task of predicting a personalized list of tags for a user given an item. This is important for many websites with tagging capabilities like last.fm or delicious. In this paper, we propose a method for tag recommendation based on tensor factorization (TF). In contrast to other TF methods like higher order singular value decomposition (HOSVD), our method RTF ('ranking with tensor factorization') directly optimizes the factorization model for the best personalized ranking. RTF handles missing values and learns from pairwise ranking constraints. Our optimization criterion for TF is motivated by a detailed analysis of the problem and of interpretation schemes for the observed data in tagging systems. In all, RTF directly optimizes for the actual problem using a correct interpretation of the data. We provide a gradient descent algorithm to solve our optimization problem. We also provide an improved learning and prediction method with runtime complexity analysis for RTF. Theprediction runtime of RTF is independent of the number of observations and only depends on the factorization dimensions. Besides the theoretical analysis, we empirically show that our method outperforms other state-of-the-art tag recommendation methods like FolkRank, PageRank and HOSVD both in quality and prediction runtime
title: Scalable graph clustering using stochastic flows: applications to community discovery
author: Venu Satuluri,Srinivasan Parthasarathy
abstract: Algorithms based on simulating stochastic flows are a simple and natural solution for the problem of clustering graphs, but their widespread use has been hampered by their lack of scalability and fragmentation of output. In this article we present a multi-level algorithm for graph clustering using flows that delivers significant improvements in both quality and speed. The graph is first successively coarsened to a manageable size, and a small number of iterations of flow simulation is performed on the coarse graph. The graph is then successively refined, with flows from the previous graph used as initializations for brief flow simulations on each of the intermediate graphs. When we reach the final refined graph, the algorithm is run to convergence and the high-flow regions are clustered together, with regions without any flow forming the natural boundaries of the clusters. Extensive experimental results on several real and synthetic datasets demonstrate the effectiveness of our approach when compared to state-of-the-art algorithms
title: Measuring the effects of preprocessing decisions and network forces in dynamic network analysis
author: Jerry Scripps,Pang-Ning Tan,Abdol-Hossein Esfahanian
abstract: Social networks have become a major focus of research in recent years, initially directed towards static networks but increasingly, towards dynamic ones. In this paper, we investigate how different pre-processing decisions and different network forces such as selection and influence affect the modeling of dynamic networks. We also present empirical justification for some of the modeling assumptions made in dynamic network analysis (e.g., first-order Markovian assumption) and develop metrics to measure the alignment between links and attributes under different strategies of using the historical network data. We also demonstrate the effect of attribute drift, that is, the importance of individual attributes in forming links change over time
title: Mining discrete patterns via binary matrix factorization
author: Bao-Hong Shen,Shuiwang Ji,Jieping Ye
abstract: Mining discrete patterns in binary data is important for subsampling, compression, and clustering. We consider rank-one binary matrix approximations that identify the dominant patterns of the data, while preserving its discrete property. A best approximation on such data has a minimum set of inconsistent entries, i.e., mismatches between the given binary data and the approximate matrix. Due to the hardness of the problem, previous accounts of such problems employ heuristics and the resulting approximation may be far away from the optimal one. In this paper, we show that the rank-one binary matrix approximation can be reformulated as a 0-1 integer linear program (ILP). However, the ILP formulation is computationally expensive even for small-size matrices. We propose a linear program (LP) relaxation, which is shown to achieve a guaranteed approximation error bound. We further extend the proposed formulations using the regularization technique, which is commonly employed to address overfitting. The LP formulation is restricted to medium-size matrices, due to the large number of variables involved for large matrices. Interestingly, we show that the proposed approximate formulation can be transformed into an instance of the minimum s-t cut problem, which can be solved efficiently by finding maximum flows. Our empirical study shows the efficiency of the proposed algorithm based on the maximum flow. Results also confirm the established theoretical bounds
title: Anomalous window discovery through scan statistics for linear intersecting paths (SSLIP)
author: Lei Shi,Vandana P. Janeja
abstract: Anomalous windows are the contiguous groupings of data points. In this paper, we propose an approach for discovering anomalous windows using Scan Statistics for Linear Intersecting Paths (SSLIP). A linear path refers to a path represented by a line with a single dimensional spatial coordinate marking an observation point. Our approach for discovering anomalous windows along linear paths comprises of the following distinct steps: (a) Cross Path Discovery: where we identify a subset of intersecting paths to be considered, (b) Anomalous Window Discovery: where we outline three order invariant algorithms, namely SSLIP, Brute Force-SSLIP and Central Brute Force-SSLIP, for the traversal of the cross paths to identify varying size directional windows along the paths. For identifying an anomalous window we compute an unusualness metric, in the form of a likelihood ratio to indicate the degree of unusualness of this window with respect to the rest of the data. We identify the window with the highest likelihood ratio as our anomalous window, and (c) Monte Carlo Simulations: to ascertain whether this window is truly anomalous and not just a random occurrence we perform hypothesis testing by computing a p-value using Monte Carlo Simulations. We present extensive experimental results in real world accident datasets for various highways with known issues(code and data available from [27], [21]). Our results show that our approach indeed is effective in identifying anomalous traffic accident windows along multiple intersecting highways
title: User grouping behavior in online forums
author: Xiaolin Shi,Jun Zhu,Rui Cai,Lei Zhang
abstract: Online forums represent one type of social media that is particularly rich for studying human behavior in information seeking and diffusing. The way users join communities is a reflection of the changing and expanding of their interests toward information. In this paper, we study the patterns of user participation behavior, and the feature factors that influence such behavior on different forum datasets. We find that, despite the relative randomness and lesser commitment of structural relationships in online forums, users' community joining behaviors display some strong regularities. One particularly interesting observation is that the very weak relationships between users defined by online replies have similar diffusion curves as those of real friendships or co-authorships. We build social selection models, Bipartite Markov Random Field (BiMRF), to quantitatively evaluate the prediction performance of those feature factors and their relationships. Using these models, we show that some featurescarry supplementary information, and the effectiveness of different features vary in different types of forums. Moreover, the results of BiMRF with two-star configurations suggest that the feature of user similarity defined by frequency of communication or number of common friends is inadequate to predict grouping behavior, but adding node-level features can improve the fit of the model
title: Causality quantification and its applications: structuring and modeling of multivariate time series
author: Takashi Shibuya,Tatsuya Harada,Yasuo Kuniyoshi
abstract: Time series prediction is an important issue in a wide range of areas. There are various real world processes whose states vary continuously, and those processes may have influences on each other. If the past information of one process X improves the predictability of another process Y, X is said to have a causal influence on Y. In order to make good predictions, it is necessary to identify the appropriate causal relationships. In addition, the processes to be modeled may include symbolic data as well as numerical data. Therefore, it is important to deal with symbolic and numerical time series seamlessly when attempting to detect causality In this paper, we propose a new method for quantifying the strength of the causal influence from one time series to another. The proposed method can represent the strength of causality as the number of bits, whether each of two time series is symbolic or numerical. The proposed method can quantify causality even from a small number of samples. In addition, we propose structuring and modeling methods for multivariate time series using causal relationships of two time series. Our structuring and modeling methods can also deal with data sets which include both types of time series. Experimental results demonstrate that our methods can perform well even if the number of samples is small
title: Ranking-based clustering of heterogeneous information networks with star network schema
author: Yizhou Sun,Yintao Yu,Jiawei Han
abstract: A heterogeneous information network is an information network composed of multiple types of objects. Clustering on such a network may lead to better understanding of both hidden structures of the network and the individual role played by every object in each cluster. However, although clustering on homogeneous networks has been studied over decades, clustering on heterogeneous networks has not been addressed until recently A recent study proposed a new algorithm, RankClus, for clustering on bi-typed heterogeneous networks. However, a real-world network may consist of more than two types, and the interactions among multi-typed objects play a key role at disclosing the rich semantics that a network carries. In this paper, we study clustering of multi-typed heterogeneous networks with a star network schema and propose a novel algorithm, NetClus, that utilizes links across multityped objects to generate high-quality net-clusters. An iterative enhancement method is developed that leads to effective ranking-based clustering in such heterogeneous networks. Our experiments on DBLP data show that NetClus generates more accurate clustering results than the baseline topic model algorithm PLSA and the recently proposed algorithm, RankClus. Further, NetClus generates informative clusters, presenting good ranking and cluster membership information for each attribute object in each net-cluster
title: Social influence analysis in large-scale networks
author: Jie Tang,Jimeng Sun,Chi Wang,Zi Yang
abstract: In large social networks, nodes (users, entities) are influenced by others for various reasons. For example, the colleagues have strong influence on one's work, while the friends have strong influence on one's daily life. How to differentiate the social influences from different angles(topics)? How to quantify the strength of those social influences? How to estimate the model on real large networks To address these fundamental questions, we propose Topical Affinity Propagation (TAP) to model the topic-level social influence on large networks. In particular, TAP can take results of any topic modeling and the existing network structure to perform topic-level influence propagation. With the help of the influence analysis, we present several important applications on real data sets such as 1) what are the representative nodes on a given topic? 2) how to identify the social influences of neighboring nodes on a particular node To scale to real large networks, TAP is designed with efficient distributed learning algorithms that is implemented and tested under the Map-Reduce framework. We further present the common characteristics of distributed learning algorithms for Map-Reduce. Finally, we demonstrate the effectiveness and efficiency of TAP on real large data sets
title: Relational learning via latent social dimensions
author: Lei Tang,Huan Liu
abstract: Social media such as blogs, Facebook, Flickr, etc., presents data in a network format rather than classical IID distribution. To address the interdependency among data instances, relational learning has been proposed, and collective inference based on network connectivity is adopted for prediction. However, connections in social media are often multi-dimensional. An actor can connect to another actor for different reasons, e.g., alumni, colleagues, living in the same city, sharing similar interests, etc. Collective inference normally does not differentiate these connections. In this work, we propose to extract latent social dimensions based on network information, and then utilize them as features for discriminative learning. These social dimensions describe diverse affiliations of actors hidden in the network, and the discriminative learning can automatically determine which affiliations are better aligned with the class labels. Such a scheme is preferred when multiple diverse relations are associated with the same network. We conduct extensive experiments on social media data (one from a real-world blog site and the other from a popular content sharing site). Our model outperforms representative relational learning methods based on collective inference, especially when few labeled data are available. The sensitivity of this model and its connection to existing methods are also examined
title: Constant-factor approximation algorithms for identifying dynamic communities
author: Chayant Tantipathananandh,Tanya Berger-Wolf
abstract: We propose two approximation algorithms for identifying communities in dynamic social networks. Communities are intuitively characterized as "unusually densely knit" subsets of a social network. This notion becomes more problematic if the social interactions change over time. Aggregating social networks over time can radically misrepresent the existing and changing community structure. Recently, we have proposed an optimization-based framework for modeling dynamic community structure. Also, we have proposed an algorithm for finding such structure based on maximum weight bipartite matching. In this paper, we analyze its performance guarantee for a special case where all actors can be observed at all times. In such instances, we show that the algorithm is a small constant factor approximation of the optimum. We use a similar idea to design an approximation algorithm for the general case where some individuals are possibly unobserved at times, and to show that the approximation factor increases twofold but remains a constant regardless of the input size. This is the first algorithm for inferring communities in dynamic networks with a provable approximation guarantee. We demonstrate the general algorithm on real data sets. The results confirm the efficiency and effectiveness of the algorithm in identifying dynamic communities
title: DOULION: counting triangles in massive graphs with a coin
author: Charalampos E. Tsourakakis,U. Kang,Gary L. Miller,Christos Faloutsos
abstract: Counting the number of triangles in a graph is a beautiful algorithmic problem which has gained importance over the last years due to its significant role in complex network analysis. Metrics frequently computed such as the clustering coefficient and the transitivity ratio involve the execution of a triangle counting algorithm. Furthermore, several interesting graph mining applications rely on computing the number of triangles in the graph of interest In this paper, we focus on the problem of counting triangles in a graph. We propose a practical method, out of which all triangle counting algorithms can potentially benefit. Using a straightforward triangle counting algorithm as a black box, we performed 166 experiments on real-world networks and on synthetic datasets as well, where we show that our method works with high accuracy, typically more than 99% and gives significant speedups, resulting in even  130 times faster performance
title: Category detection using hierarchical mean shift
author: Pavan Vatturi,Weng-Keen Wong
abstract: Many applications in surveillance, monitoring, scientific discovery, and data cleaning require the identification of anomalies. Although many methods have been developed to identify statistically significant anomalies, a more difficult task is to identify anomalies that are both interesting and statistically significant. Category detection is an emerging area of machine learning that can help address this issue using a "human-in-the-loop" approach. In this interactive setting, the algorithm asks the user to label a query data point under an existing category or declare the query data point to belong to a previously undiscovered category. The goal of category detection is to bring to the user's attention a representative data point from each category in the data in as few queries as possible. In a data set with imbalanced categories, the main challenge is in identifying the rare categories or anomalies; hence, the task is often referred to as rare category detection. We present a new approach to rare category detection based on hierarchical mean shift. In our approach, a hierarchy is created by repeatedly applying mean shift with an increasing bandwidth on the data. This hierarchy allows us to identify anomalies in the data set at different scales, which are then posed as queries to the user. The main advantage of this methodology over existing approaches is that it does not require any knowledge of the dataset properties such as the total number of categories or the prior probabilities of the categories. Results on real-world data sets show that our hierarchical mean shift approach performs consistently better than previous techniques
title: Learning, indexing, and diagnosing network faults
author: Ting Wang,Mudhakar Srivatsa,Dakshi Agrawal,Ling Liu
abstract: Modern communication networks generate massive volume of operational event data, e.g., alarm, alert, and metrics, which can be used by a network management system (NMS) to diagnose potential faults. In this work, we introduce a new class of indexable fault signatures that encode temporal evolution of events generated by a network fault as well as topological relationships among the nodes where these events occur. We present an efficient learning algorithm to extract such fault signatures from noisy historical event data, and with the help of novel space-time indexing structures, we show how to perform efficient, online signature matching. We provide results from extensive experimental studies to explore the efficacy of our approach and point out potential applications of such signatures for many different types of networks including social and information networks
title: Mining broad latent query aspects from search sessions
author: Xuanhui Wang,Deepayan Chakrabarti,Kunal Punera
abstract: Search queries are typically very short, which means they are often underspecified or have senses that the user did not think of. A broad latent query aspect is a set of keywords that succinctly represents one particular sense, or one particular information need, that can aid users in reformulating such queries. We extract such broad latent aspects from query reformulations found in historical search session logs. We propose a framework under which the problem of extracting such broad latent aspects reduces to that of optimizing a formal objective function under constraints on the total number of aspects the system can store, and the number of aspects that can be shown in response to any given query. We present algorithms to find a good set of aspects, and also to pick the best k aspects matching any query. Empirical results on real-world search engine logs show significant gains over a strong baseline that uses single-keyword reformulations: a gain of 14% and 23% in terms of human-judged accuracy and click-through data respectively, and around 20% in terms of consistency among aspects predicted for "similar" queries. This demonstrates both the importance of broad query aspects, and the efficacy of our algorithms for extracting them
title: Adapting the right measures for K-means clustering
author: Junjie Wu,Hui Xiong,Jian Chen
abstract: Clustering validation is a long standing challenge in the clustering literature. While many validation measures have been developed for evaluating the performance of clustering algorithms, these measures often provide inconsistent information about the clustering performance and the best suitable measures to use in practice remain unknown. This paper thus fills this crucial void by giving an organized study of 16 external validation measures for K-means clustering. Specifically, we first introduce the importance of measure normalization in the evaluation of the clustering performance on data with imbalanced class distributions. We also provide normalization solutions for several measures. In addition, we summarize the major properties of these external measures. These properties can serve as the guidance for the selection of validation measures in different application scenarios. Finally, we reveal the interrelationships among these external measures. By mathematical transformation, we show that some validation measures are equivalent. Also, some measures have consistent validation performances. Most importantly, we provide a guide line to select the most suitable validation measures for K-means clustering
title: A LRT framework for fast spatial anomaly detection
author: Mingxi Wu,Xiuyao Song,Chris Jermaine,Sanjay Ranka,John Gums
abstract: Given a spatial data set placed on an n x n grid, our goal is to find the rectangular regions within which subsets of the data set exhibit anomalous behavior. We develop algorithms that, given any user-supplied arbitrary likelihood function, conduct a likelihood ratio hypothesis test (LRT) over each rectangular region in the grid, rank all of the rectangles based on the computed LRT statistics, and return the top few most interesting rectangles. To speed this process, we develop methods to prune rectangles without computing their associated LRT statistics
title: Quantification and semi-supervised classification methods for handling changes in class distribution
author: Jack Chongjie Xue,Gary M. Weiss
abstract: In realistic settings the prevalence of a class may change after a classifier is induced and this will degrade the performance of the classifier. Further complicating this scenario is the fact that labeled data is often scarce and expensive. In this paper we address the problem where the class distribution changes and only unlabeled examples are available from the new distribution. We design and evaluate a number of methods for coping with this problem and compare the performance of these methods. Our quantification-based methods estimate the class distribution of the unlabeled data from the changed distribution and adjust the original classifier accordingly, while our semi-supervised methods build a new classifier using the examples from the new (unlabeled) distribution which are supplemented with predicted class values. We also introduce a hybrid method that utilizes both quantification and semi-supervised learning. All methods are evaluated using accuracy and F-measure on a set of benchmarkdata sets. Our results demonstrate that our methods yield substantial improvements in accuracy and F-measure
title: Fast approximate spectral clustering
author: Donghui Yan,Ling Huang,Michael I. Jordan
abstract: Spectral clustering refers to a flexible class of clustering procedures that can produce high-quality clusterings on small data sets but which has limited applicability to large-scale problems due to its computational complexity of O(n3) in general, with n the number of data points. We extend the range of spectral clustering by developing a general framework for fast approximate spectral clustering in which a distortion-minimizing local transformation is first applied to the data. This framework is based on a theoretical analysis that provides a statistical characterization of the effect of local distortion on the mis-clustering rate. We develop two concrete instances of our general framework, one based on local k-means clustering (KASP) and one based on random projection trees (RASP). Extensive experiments show that these algorithms can achieve significant speedups with little degradation in clustering accuracy. Specifically, our algorithms outperform k-means by a large margin in terms of accuracy, and run several times faster than approximate spectral clustering based on the Nystrom method, with comparable accuracy and significantly smaller memory footprint. Remarkably, our algorithms make it possible for a single machine to spectral cluster data sets with a million observations within several minutes
title: Effective multi-label active learning for text classification
author: Bishan Yang,Jian-Tao Sun,Tengjiao Wang,Zheng Chen
abstract: Labeling text data is quite time-consuming but essential for automatic text classification. Especially, manually creating multiple labels for each document may become impractical when a very large amount of data is needed for training multi-label text classifiers. To minimize the human-labeling efforts, we propose a novel multi-label active learning approach which can reduce the required labeled data without sacrificing the classification accuracy. Traditional active learning algorithms can only handle single-label problems, that is, each data is restricted to have one label. Our approach takes into account the multi-label information, and select the unlabeled data which can lead to the largest reduction of the expected model loss. Specifically, the model loss is approximated by the size of version space, and the reduction rate of the size of version space is optimized with Support Vector Machines (SVM). An effective label prediction method is designed to predict possible labels for each unlabeled data point, and the expected loss for multi-label data is approximated by summing up losses on all labels according to the most confident result of label prediction. Experiments on several real-world data sets (all are publicly available) demonstrate that our approach can obtain promising classification result with much fewer labeled data than state-of-the-art methods
title: Combining link and content for community detection: a discriminative approach
author: Tianbao Yang,Rong Jin,Yun Chi,Shenghuo Zhu
abstract: In this paper, we consider the problem of combining link and content analysis for community detection from networked data, such as paper citation networks and Word Wide Web. Most existing approaches combine link and content information by a generative model that generates both links and contents via a shared set of community memberships. These generative models have some shortcomings in that they failed to consider additional factors that could affect the community memberships and isolate the contents that are irrelevant to community memberships. To explicitly address these shortcomings, we propose a discriminative model for combining the link and content analysis for community detection. First, we propose a conditional model for link analysis and in the model, we introduce hidden variables to explicitly model the popularity of nodes. Second, to alleviate the impact of irrelevant content attributes, we develop a discriminative model for content analysis. These two models are unified seamlessly via the community memberships. We present efficient algorithms to solve the related optimization problems based on bound optimization and alternating projection. Extensive experiments with benchmark data sets show that the proposed framework significantly outperforms the state-of-the-art approaches for combining link and content analysis for community detection
title: Efficient methods for topic model inference on streaming document collections
author: Limin Yao,David Mimno,Andrew McCallum
abstract: Topic models provide a powerful tool for analyzing large text collections by representing high dimensional data in a low dimensional subspace. Fitting a topic model given a set of training documents requires approximate inference techniques that are computationally expensive. With today's large-scale, constantly expanding document collections, it is useful to be able to infer topic distributions for new documents without retraining the model. In this paper, we empirically evaluate the performance of several methods for topic inference in previously unseen documents, including methods based on Gibbs sampling, variational inference, and a new method inspired by text classification. The classification-based inference method produces results similar to iterative inference methods, but requires only a single matrix multiplication. In addition to these inference methods, we present SparseLDA, an algorithm and data structure for evaluating Gibbs sampling distributions. Empirical results indicate thatSparseLDA can be approximately 20 times faster than traditional LDA and provide twice the speedup of previously published fast sampling methods, while also using substantially less memory
title: Time series shapelets: a new primitive for data mining
author: Lexiang Ye,Eamonn Keogh
abstract: Classification of time series has been attracting great interest over the past decade. Recent empirical evidence has strongly suggested that the simple nearest neighbor algorithm is very difficult to beat for most time series problems. While this may be considered good news, given the simplicity of implementing the nearest neighbor algorithm, there are some negative consequences of this. First, the nearest neighbor algorithm requires storing and searching the entire dataset, resulting in a time and space complexity that limits its applicability, especially on resource-limited sensors. Second, beyond mere classification accuracy, we often wish to gain some insight into the data In this work we introduce a new time series primitive, time series shapelets, which addresses these limitations. Informally, shapelets are time series subsequences which are in some sense maximally representative of a class. As we shall show with extensive empirical evaluations in diverse domains, algorithms basedon the time series shapelet primitives can be interpretable, more accurate and significantly faster than state-of-the-art classifiers
title: Exploring social tagging graph for web object classification
author: Zhijun Yin,Rui Li,Qiaozhu Mei,Jiawei Han
abstract: This paper studies web object classification problem with the novel exploration of social tags. Automatically classifying web objects into manageable semantic categories has long been a fundamental preprocess for indexing, browsing, searching, and mining these objects. The explosive growth of heterogeneous web objects, especially non-textual objects such as products, pictures, and videos, has made the problem of web classification increasingly challenging. Such objects often suffer from a lack of easy-extractable features with semantic information, interconnections between each other, as well as training examples with category labels In this paper, we explore the social tagging data to bridge this gap. We cast web object classification problem as an optimization problem on a graph of objects and tags. We then propose an efficient algorithm which not only utilizes social tags as enriched semantic features for the objects, but also infers the categories of unlabeled objects from both homogeneous and heterogeneous labeled objects, through the implicit connection of social tags. Experiment results show that the exploration of social tags effectively boosts web object classification. Our algorithm significantly outperforms the state-of-the-art of general classification methods
title: Mining social networks for personalized email prioritization
author: Shinjae Yoo,Yiming Yang,Frank Lin,Il-Chul Moon
abstract: Email is one of the most prevalent communication tools today, and solving the email overload problem is pressingly urgent. A good way to alleviate email overload is to automatically prioritize received messages according to the priorities of each user. However, research on statistical learning methods for fully personalized email prioritization (PEP) has been sparse due to privacy issues, since people are reluctant to share personal messages and importance judgments with the research community. It is therefore important to develop and evaluate PEP methods under the assumption that only limited training examples can be available, and that the system can only have the personal email data of each user during the training and testing of the model for that user. This paper presents the first study (to the best of our knowledge) under such an assumption. Specifically, we focus on analysis of personal social networks to capture user groups and to obtain rich features that represent the social roles from the viewpoint of a particular user. We also developed a novel semi-supervised (transductive) learning algorithm that propagates importance labels from training examples to test examples through message and user nodes in a personal email network. These methods together enable us to obtain an enriched vector representation of each new email message, which consists of both standard features of an email message (such as words in the title or body, sender and receiver IDs, etc.) and the induced social features from the sender and receivers of the message. Using the enriched vector representation as the input in SVM classifiers to predict the importance level for each test message, we obtained significant performance improvement over the baseline system (without induced social features) in our experiments on a multi-user data collection. We obtained significant performance improvement over the baseline system (without induced social features) in our experiments on a multi-user data collection: the relative error reduction in MAE was 31% in micro-averaging, and 14% in macro-averaging
title: Learning patterns in the dynamics of biological networks
author: Chang hun You,Lawrence B. Holder,Diane J. Cook
abstract: Our dynamic graph-based relational mining approach has been developed to learn structural patterns in biological networks as they change over time. The analysis of dynamic networks is important not only to understand life at the system-level, but also to discover novel patterns in other structural data. Most current graph-based data mining approaches overlook dynamic features of biological networks, because they are focused on only static graphs. Our approach analyzes a sequence of graphs and discovers rules that capture the changes that occur between pairs of graphs in the sequence. These rules represent the graph rewrite rules that the first graph must go through to be isomorphic to the second graph. Then, our approach feeds the graph rewrite rules into a machine learning system that learns general transformation rules describing the types of changes that occur for a class of dynamic biological networks. The discovered graph-rewriting rules show how biological networks change over time, and the transformation rules show the repeated patterns in the structural changes. In this paper, we apply our approach to biological networks to evaluate our approach and to understand how the biosystems change over time. We evaluate our results using coverage and prediction metrics, and compare to biological literature
title: Toward autonomic grids: analyzing the job flow with affinity streaming
author: Xiangliang Zhang,Cyril Furtlehner,Julien Perez,Cecile Germain-Renaud,Michele Sebag
abstract: The Affinity Propagation (AP) clustering algorithm proposed by Frey and Dueck (2007) provides an understandable, nearly optimal summary of a dataset, albeit with quadratic computational complexity. This paper, motivated by Autonomic Computing, extends AP to the data streaming framework. Firstly a hierarchical strategy is used to reduce the complexity to O(N1+); the distortion loss incurred is analyzed in relation with the dimension of the data items. Secondly, a coupling with a change detection test is used to cope with non-stationary data distribution, and rebuild the model as needed. The presented approach StrAP is applied to the stream of jobs submitted to the EGEE Grid, providing an understandable description of the job flow and enabling the system administrator to spot online some sources of failures
title: Parallel community detection on large networks with propinquity dynamics
author: Yuzhou Zhang,Jianyong Wang,Yi Wang,Lizhu Zhou
abstract: Graphs or networks can be used to model complex systems. Detecting community structures from large network data is a classic and challenging task. In this paper, we propose a novel community detection algorithm, which utilizes a dynamic process by contradicting the network topology and the topology-based propinquity, where the propinquity is a measure of the probability for a pair of nodes involved in a coherent community structure. Through several rounds of mutual reinforcement between topology and propinquity, the community structures are expected to naturally emerge. The overlapping vertices shared between communities can also be easily identified by an additional simple postprocessing. To achieve better efficiency, the propinquity is incrementally calculated. We implement the algorithm on a vertex-oriented bulk synchronous parallel(BSP) model so that the mining load can be distributed on thousands of machines. We obtained interesting experimental results on several real network data
title: Co-evolution of social and affiliation networks
author: Elena Zheleva,Hossam Sharara,Lise Getoor
abstract: In our work, we address the problem of modeling social network generation which explains both link and group formation. Recent studies on social network evolution propose generative models which capture the statistical properties of real-world networks related only to node-to-node link formation. We propose a novel model which captures the co-evolution of social and affiliation networks. We provide surprising insights into group formation based on observations in several real-world networks, showing that users often join groups for reasons other than their friends. Our experiments show that the model is able to capture both the newly observed and previously studied network properties. This work is the first to propose a generative model which captures the statistical properties of these complex networks. The proposed model facilitates controlled experiments which study the effect of actors' behavior on the evolution of affiliation networks, and it allows the generation of realistic synthetic datasets
title: Information theoretic regularization for semi-supervised boosting
author: Lei Zheng,Shaojun Wang,Yan Liu,Chi-Hoon Lee
abstract: We present novel semi-supervised boosting algorithms that incrementally build linear combinations of weak classifiers through generic functional gradient descent using both labeled and unlabeled training data. Our approach is based on extending information regularization framework to boosting, bearing loss functions that combine log loss on labeled data with the information-theoretic measures to encode unlabeled data. Even though the information-theoretic regularization terms make the optimization non-convex, we propose simple sequential gradient descent optimization algorithms, and obtain impressively improved results on synthetic, benchmark and real world tasks over supervised boosting algorithms which use the labeled data alone and a state-of-the-art semi-supervised boosting algorithm
title: Cross domain distribution adaptation via kernel mapping
author: Erheng Zhong,Wei Fan,Jing Peng,Kun Zhang,Jiangtao Ren,Deepak Turaga,Olivier Verscheure
abstract: When labeled examples are limited and difficult to obtain, transfer learning employs knowledge from a source domain to improve learning accuracy in the target domain. However, the assumption made by existing approaches, that the marginal and conditional probabilities are directly related between source and target domains, has limited applicability in either the original space or its linear transformations. To solve this problem, we propose an adaptive kernel approach that maps the marginal distribution of target-domain and source-domain data into a common kernel space, and utilize a sample selection strategy to draw conditional probabilities between the two domains closer. We formally show that under the kernel-mapping space, the difference in distributions between the two domains is bounded; and the prediction error of the proposed approach can also be bounded. Experimental results demonstrate that the proposed method outperforms both traditional inductive classifiers and the state-of-the-artboosting-based transfer algorithms on most domains, including text categorization and web page ratings. In particular, it can achieve around 10% higher accuracy than other approaches for the text categorization problem. The source code and datasets are available from the authors
title: Mining rich session context to improve web search
author: Guangyu Zhu,Gilad Mishne
abstract: User browsing information, particularly their non-search related activity, reveals important contextual information on the preferences and the intent of web users. In this paper, we expand the use of browsing information for web search ranking and other applications, with an emphasis on analyzing individual user sessions for creating aggregate models. In this context, we introduce ClickRank, an efficient, scalable algorithm for estimating web page and web site importance from browsing information. We lay out the theoretical foundation of ClickRank based on an intentional surfer model and analyze its properties. We evaluate its effectiveness for the problem of web search ranking, showing that it contributes significantly to retrieval performance as a novel web search feature. We demonstrate that the results produced by ClickRank for web search ranking are highly competitive with those produced by other approaches, yet achieved at better scalability and substantially lower computational costs. Finally, we discuss novel applications of ClickRank in providing enriched user web search experience, highlighting the usefulness of our approach for non-ranking tasks
title: Primal sparse Max-margin Markov networks
author: Jun Zhu,Eric P. Xing,Bo Zhang
abstract: Max-margin Markov networks (M3N) have shown great promise in structured prediction and relational learning. Due to the KKT conditions, the M3N enjoys dual sparsity. However, the existing M3N formulation does not enjoy primal sparsity, which is a desirable property for selecting significant features and reducing the risk of over-fitting. In this paper, we present an l1-norm regularized max-margin Markov network (l1-M3N), which enjoys dual and primal sparsity simultaneously. To learn an l1-M3N, we present three methods including projected sub-gradient, cutting-plane, and a novel EM-style algorithm, which is based on an equivalence between l1-M3N and an adaptive M3N. We perform extensive empirical studies on both synthetic and real data sets. Our experimental results show that: (1) l1-M3N can effectively select significantfeatures; (2) l1-M3N can perform as well as the pseudo-primal sparse Laplace M3N in prediction accuracy, while consistently outperforms other competing methods that enjoy either primal or dual sparsity; and (3) the EM-algorithm is more robust than the other two in pre-diction accuracy and time efficiency
title: Augmenting the generalized hough transform to enable the mining of petroglyphs
author: Qiang Zhu,Xiaoyue Wang,Eamonn Keogh,Sang-Hee Lee
abstract: Rock art is an archaeological term for human-made markings on stone. It is believed that there are millions of petroglyphs in North America alone, and the study of this valued cultural resource has implications even beyond anthropology and history. Surprisingly, although image processing, information retrieval and data mining have had large impacts on many human endeavors, they have had essentially zero impact on the study of rock art. In this work we identify the reasons for this, and introduce a novel distance measure and algorithms which allow efficient and effective data mining of large collections of rock art
title: Influence and Correlation in Social Networks
author: Aris Anagnostopoulos,Ravi Kumar,Mohammad Mahdia
abstract: In many online social systems, social ties between users playan important role in dictating their behavior. One of theways this can happen is through social influence, the phe-nomenon that the actions of a user can induce his/her friendsto behave in a similar way. In systems where social influenceexists, ideas, modes of behavior, or new technologies can dif-fuse through the network like an epidemic. Therefore, iden-tifying and understanding social influence is of tremendousinterest from both analysis and design points of view.This is a difficult task in general, since there are fac-tors such as homophily or unobserved confounding variablesthat can induce statistical correlation between the actionsof friends in a social network. Distinguishing influence fromthese is essentially the problem of distinguishing correlationfrom causality, a notoriously hard statistical problem.In this paper we study this problem systematically. Wedefine fairly general models that replicate the aforemen-tioned sources of social correlation. We then propose twosimple tests that can identify influence as a source of socialcorrelation when the time series of user actions is available.We give a theoretical justification of one of the tests byproving that with high probability it succeeds in ruling outinfluence in a rather general model of social correlation. Wealso simulate our tests on a number of examples designedby randomly generating actions of nodes on a real socialnetwork (from Flickr) according to one of several models.Simulation results confirm that our test performs well onthese data. Finally, we apply them to real tagging data onFlickr, exhibiting that while there is significant social cor-relation in tagging behavior on this system, this correlationcannot be attributed to social influence.
title: Efficient Semi-streaming Algorithms for Local Triangle Counting in Massive Graphs
author: Luca Becchetti,Paolo Boldi,Carlos Castillo,Aristides Gionis
abstract: In this paper we study the problem of local triangle count-ing in large graphs. Namely, given a large graph G = (V,E)we want to estimate as accurately as possible the number oftriangles incident to every node v  V in the graph. Theproblem of computing the global number of triangles in agraph has been considered before, but to our knowledge thisis the first paper that addresses the problem of local tri-angle counting with a focus on the efficiency issues arisingin massive graphs. The distribution of the local numberof triangles and the related local clustering coefficient canbe used in many interesting applications. For example, weshow that the measures we compute can help to detect thepresence of spamming activity in large-scale Web graphs, aswell as to provide useful features to assess content qualityin social networks.For computing the local number of triangles we proposetwo approximation algorithms, which are based on the ideaof min-wise independent permutations (Broder et al. 1998).Our algorithms operate in a semi-streaming fashion, usingO(|V |) space in main memory and performing O(log |V |)sequential scans over the edges of the graph. The first al-gorithm we describe in this paper also uses O(|E|) space inexternal memory during computation, while the second al-gorithm uses only main memory. We present the theoreticalanalysis as well as experimental results in massive graphsdemonstrating the practical efficiency of our approach.
title: Structured Entity Identification and Document Categorization: Two Tasks with One Joint Model
author: Indrajit Bhattacharya,Shantanu Godbole,Sachindra Joshi
abstract: Traditionally, research in identifying structured entities indocuments has proceeded independently of document cat-egorization research. In this paper, we observe that thesetwo tasks have much to gain from each other. Apart fromdirect references to entities in a database, such as namesof person entities, documents often also contain words thatare correlated with discriminative entity attributes, suchage-group and income-level of persons. This happens nat-urally in many enterprise domains such as CRM, Banking,etc. Then, entity identification, which is typically vulnera-ble against noise and incompleteness in direct references toentities in documents, can benefit from document catego-rization with respect to such attributes. In return, entityidentification enables documents to be categorized accord-ing to different label-sets arising from entity attributes with-out requiring any supervision. In this paper, we propose aprobabilistic generative model for joint entity identificationand document categorization. We show how the parame-ters of the model can be estimated using an EM algorithmin an unsupervised fashion. Using extensive experimentsover real and semi-synthetic data, we demonstrate that thetwo tasks can benefit immensely from each other when per-formed jointly using the proposed model
title: Mining Adaptively Frequent Closed Unlabeled Rooted Trees in Data Streams
author: Albert Bifet,Ricard Gavalda
abstract: Closed patterns are powerful representatives of frequent pat-terns, since they eliminate redundant information. We pro-pose a new approach for mining closed unlabeled rootedtrees adaptively from data streams that change over time.Our approach is based on an efficient representation of treesand a low complexity notion of relaxed closed trees, andleads to an on-line strategy and an adaptive sliding win-dow technique for dealing with changes over time. Moreprecisely, we first present a general methodology to identifyclosed patterns in a data stream, using Galois Lattice The-ory. Using this methodology, we then develop three closedtree mining algorithms: an incremental one IncTreeNat,a sliding-window based one, WinTreeNat, and finally onethat mines closed trees adaptively from data streams, Ada-TreeNat. To the best of our knowledge this is the first workon mining frequent closed trees in streaming data varyingwith time. We give a first experimental evaluation of theproposed algorithms
title: Effective Label Acquisition for Collective Classification
author: Mustafa Bilgic,Lise Getoor
abstract: Information diffiusion, viral marketing, and collective classi-fication all attempt to model and exploit the relationshipsin a network to make inferences about the labels of nodes.A variety of techniques have been introduced and methodsthat combine attribute information and neighboring labelinformation have been shown to be effective for collectivelabeling of the nodes in a network. However, in part becauseof the correlation between node labels that the techniquesexploit, it is easy to find cases in which, once a misclassifica-tion is made, incorrect information propagates throughoutthe network. This problem can be mitigated if the system isallowed to judiciously acquire the labels for a small numberof nodes. Unfortunately, under relatively general assump-tions, determining the optimal set of labels to acquire isintractable. Here we propose an acquisition method thatlearns the cases when a given collective classification algo-rithm makes mistakes, and suggests acquisitions to correctthose mistakes. We empirically show on both real and syn-thetic datasets that this method significantly outperformsa greedy approximate inference approach, a viral market-ing approach, and approaches based on network structuralmeasures such as node degree and network clustering. In ad-dition to significantly improving accuracy with just a smallamount of labeled data, our method is tractable on largenetworks.
title: Topical Query Decomposition
author: Francesco Bonchi,Carlos Castillo,Debora Donato,Aristides Gionis
abstract: We introduce the problem of query decomposition, wherewe are given a query and a document retrieval system, andwe want to produce a small set of queries whose union ofresulting documents corresponds approximately to that ofthe original query. Ideally, these queries should representcoherent, conceptually well-separated topics.We provide an abstract formulation of the query decompo-sition problem, and we tackle it from two different perspec-tives. We first show how the problem can be instantiated asa specific variant of a set cover problem, for which we pro-vide an efficient greedy algorithm. Next, we show how thesame problem can be seen as a constrained clustering prob-lem, with a very particular kind of constraint, i.e., clusteringwith predefined clusters. We develop a two-phase algorithmbased on hierarchical agglomerative clustering followed bydynamic programming. Our experiments, conducted on aset of actual queries in a Web scale search engine, confirmthe effectiveness of the proposed solutions.
title: Unsupervised Feature Selection for Principal Components Analysis
author: Christos Boutsidis,Michael W. Mahoney,Petros Drineas
abstract: Principal Components Analysis (PCA) is the predominantlinear dimensionality reduction technique, and has been widelyapplied on datasets in all scientific domains. We consider,both theoretically and empirically, the topic of unsuper-vised feature selection for PCA, by leveraging algorithms forthe so-called Column Subset Selection Problem (CSSP). Inwords, the CSSP seeks thebestsubset of exactly k columnsfrom an mn data matrix A, and has been extensively stud-ied in the Numerical Linear Algebra community. We presenta novel two-stage algorithm for the CSSP. From a theoreticalperspective, for small to moderate values of k, this algorithmsignificantly improves upon the best previously-existing re-sults [24, 12] for the CSSP. From an empirical perspective,we evaluate this algorithm as an unsupervised feature selec-tion strategy in three application domains of modern sta-tistical data analysis: finance, document-term data, and ge-netics. We pay particular attention to how this algorithmmay be used to select representative or landmark featuresfrom an object-feature matrix in an unsupervised manner.In all three application domains, we are able to identify klandmark features, i.e., columns of the data matrix, thatcapture nearly the same amount of information as does thesubspace that is spanned by the top k eigenfeatures.
title: The Cost of Privacy: Destruction of Data-Mining Utility in Anonymized Data Publishing
author: Justin Brickell,Vitaly Shmatikov
abstract: Re-identification is a major privacy threat to public datasetscontaining individual records. Many privacy protection al-gorithms rely on generalization and suppression of quasi-identifier attributes such as ZIP code and birthdate. Theirobjective is usually syntactic sanitization: for example, k-anonymity requires that each quasi-identifier tuple appearin at least k records, while -diversity requires that the dis-tribution of sensitive attributes for each quasi-identifier havehigh entropy. The utility of sanitized data is also measuredsyntactically, by the number of generalization steps appliedor the number of records with the same quasi-identifier.In this paper, we ask whether generalization and sup-pression of quasi-identifiers offer any benefits over triv-ial sanitization which simply separates quasi-identifiersfrom sensitive attributes. Previous work showed that k-anonymous databases can be useful for data mining, butk-anonymization does not guarantee any privacy. By con-trast, we measure the tradeoff between privacy (how muchcan the adversary learn from the sanitized records?) andutility, measured as accuracy of data-mining algorithmsexecuted on the same sanitized records.For our experimental evaluation, we use the same datasetsfrom the UCI machine learning repository as were used inprevious research on generalization and suppression. Ourresults demonstrate that even modest privacy gains requirealmost complete destruction of the data-mining utility. Inmost cases, trivial sanitization provides equivalent utilityand better privacy than k-anonymity, l-diversity, and similarmethods based on generalization and suppression.
title: Generating Succinct Titles for Web URLs
author: Deepayan Chakrabarti,Ravi Kumar,Kunal Punera
abstract: How can a search engine automatically provide the best andmost appropriate title for a result URL (link-title) so thatusers will be persuaded to click on the URL? We considerthe problem of automatically generating link-titles for URLsand propose a general statistical framework for solving thisproblem. The framework is based on using informationfrom a diverse collection of sources, each of which can bethought of as contributing one or more candidate link-titlesfor the URL. It can also incorporate the context in which thelink-title will be used, along with constraints on its length.Our framework is applicable to several scenarios: obtainingsuccinct titles for displaying quicklinks, obtaining titles forURLs that lack a good title, constructing succinct sitemaps,etc. Extensive experiments show that our method is very ef-fective, producing results that are at least 20% better thannon-trivial baselines.
title: Structured Learning for Non-Smooth Ranking Losses
author: Soumen Chakrabarti,Rajiv Khanna,Uma Sawant,Chiru Bhattacharyya
abstract: Learning to rank from relevance judgment is an active re-search area. Itemwise score regression, pairwise preferencesatisfaction, and listwise structured learning are the majortechniques in use. Listwise structured learning has beenapplied recently to optimize important non-decomposableranking criteria like AUC (area under ROC curve) and MAP(mean average precision). We propose new, almost-linear-time algorithms to optimize for two other criteria widelyused to evaluate search systems: MRR (mean reciprocalrank) and NDCG (normalized discounted cumulative gain)in the max-margin structured learning framework. We alsodemonstrate that, for different ranking criteria, one mayneed to use different feature maps. Search applications shouldnot be optimized in favor of a single criterion, because theyneed to cater to a variety of queries. E.g., MRR is best fornavigational queries, while NDCG is best for informationalqueries. A key contribution of this paper is to fold multipleranking loss functions into a multi-criteria max-margin op-timization. The result is a single, robust ranking model thatis close to the best accuracy of learners trained on individ-ual criteria. In fact, experiments over the popular LETORand TREC data sets show that, contrary to conventionalwisdom, a test criterion is often not best served by trainingwith the same individual criterion.
title: Partitioned Logistic Regression for Spam Filtering
author: Ming-wei Chang,Wen-tau Yih,Christopher Meek
abstract: Naive Bayes and logistic regression perform well in different regimes.While the former is a very simple generative model which is effi-cient to train and performs well empirically in many applications,the latter is a discriminative model which often achieves better ac-curacy and can be shown to outperform naive Bayes asymptotically.In this paper, we propose a novel hybrid model, partitioned logis-tic regression, which has several advantages over both naive Bayesand logistic regression. This model separates the original featurespace into several disjoint feature groups. Individual models onthese groups of features are learned using logistic regression andtheir predictions are combined using the naive Bayes principle toproduce a robust final estimation. We show that our model is betterboth theoretically and empirically. In addition, when applying it ina practical application, email spam filtering, it improves the nor-malized AUC score at 10% false-positive rate by 28.8% and 23.6%compared to naive Bayes and logistic regression, when using theexact same training examples.
title: Learning Subspace Kernels for Classification
author: Jianhui Chen,Shuiwang Ji,Betul Ceran,Qi Li,Mingrui Wu,Jieping Ye
abstract: Kernel methods have been applied successfully in many datamining tasks. Subspace kernel learning was recently pro-posed to discover an effective low-dimensional subspace ofa kernel feature space for improved classification. In thispaper, we propose to construct a subspace kernel using theHilbert-Schmidt Independence Criterion (HSIC). We showthat the optimal subspace kernel can be obtained efficientlyby solving an eigenvalue problem. One limitation of the ex-isting subspace kernel learning formulations is that the ker-nel learning and classification are independent and the sub-space kernel may not be optimally adapted for classification.To overcome this limitation, we propose a joint optimizationframework, in which we learn the subspace kernel and sub-sequent classifiers simultaneously. In addition, we proposea novel learning formulation that extracts an uncorrelatedsubspace kernel to reduce the redundant information in asubspace kernel. Following the idea from multiple kernellearning, we extend the proposed formulations to the casewhen multiple kernels are available and need to be com-bined. We show that the integration of subspace kernelscan be formulated as a semidefinite program (SDP) whichis computationally expensive. To improve the efficiency ofthe SDP formulation, we propose an equivalent semi-infinitelinear program (SILP) formulation which can be solved ef-fficiently by the column generation technique. Experimentalresults on a collection of benchmark data sets demonstratethe effectiveness of the proposed algorithms.
title: Combinational Collaborative Filtering for Personalized Community Recommendation
author: Wen-Yen Chen,Dong Zhang,Edward Y. Chang
abstract: Rapid growth in the amount of data available on social net-working sites has made information retrieval increasinglychallenging for users. In this paper, we propose a collabora-tive filtering method, Combinational Collaborative Filtering(CCF), to perform personalized community recommenda-tions by considering multiple types of co-occurrences in so-cial data at the same time. This filtering method fuses se-mantic and user information, then applies a hybrid trainingstrategy that combines Gibbs sampling and Expectation-Maximization algorithm. To handle the large-scale dataset,parallel computing is used to speed up the model training.Through an empirical study on the Orkut dataset, we showCCF to be both effective and scalable.
title: FAST: A ROC-based Feature Selection Metric for Small Samples and Imbalanced Data Classification Problems
author: Xue-wen Chen,Michael Wasikowski
abstract: The class imbalance problem is encountered in a large number of practical applications of machine learning and data mining, for example, information retrieval and filtering, and the detection of credit card fraud. It has been widely realized that this imbalance raises issues that are either nonexistent or less severe compared to balanced class cases and often results in a classifiers suboptimal performance. This is even more true when the imbalanced data are also high dimensional. In such cases, feature selection methods are critical to achieve optimal performance. In this paper, we propose a new feature selection method, Feature Assessment by Sliding Thresholds (FAST), which is based on the area under a ROC curve generated by moving the decision boundary of a single feature classifier with thresholds placed using an even-bin distribution. FAST is compared to two commonly-used feature selection methods, correlation coefficient and RELevance In Estimating Features (RELIEF), for imbalanced data classification. The experimental results obtained on text mining, mass spectrometry, and microarray data sets showed that the proposed method outperformed both RELIEF and correlation methods on skewed data sets and was comparable on balanced data sets; when small number of features is preferred, the classification performance of the proposed method was significantly improved compared to correlation and RELIEF-based methods.
title: Semi-supervised Learning with Data Calibration for Long-Term Time Series Forecasting
author: Haibin Cheng,Pang-Ning Tan
abstract: Many time series prediction methods have focused on singlestep or short term prediction problems due to the inher-ent difficulty in controlling the propagation of errors fromone prediction step to the next step. Yet, there is a broadrange of applications such as climate impact assessmentsand urban growth planning that require long term forecast-ing capabilities for strategic decision making. Training anaccurate model that produces reliable long term predictionswould require an extensive amount of historical data, whichare either unavailable or expensive to acquire. For some ofthese domains, there are alternative ways to generate po-tential scenarios for the future using computer-driven sim-ulation models, such as global climate and traffic demandmodels. However, the data generated by these models arecurrently utilized in a supervised learning setting, where apredictive model trained on past observations is used to es-timate the future values. In this paper, we present a semi-supervised learning framework for long-term time series fore-casting based on Hidden Markov Model Regression. A co-variance alignment method is also developed to deal with theissue of inconsistencies between historical and model simu-lation data. We evaluated our approach on data sets froma variety of domains, including climate modeling. Our ex-perimental results demonstrate the efficacy of the approachcompared to other supervised learning methods for long-term time series forecasting
title: Reconstructing Chemical Reaction Networks: Data Mining meets System Identification
author: Yong Ju Cho,Naren Ramakrishnan,Yang Cao
abstract: We present an approach to reconstructing chemical reac-tion networks from time series measurements of the concen-trations of the molecules involved. Our solution strategycombines techniques from numerical sensitivity analysis andprobabilistic graphical models. By modeling a chemical re-action system as a Markov network (undirected graphicalmodel), we show how systematically probing for sensitivi-ties between molecular species can identify the topology ofthe network. Given the topology, our approach next uses de-tailed sensitivity profiles to characterize properties of reac-tions such as reversibility, enzyme-catalysis, and the precisestoichiometries of the reactants and products. We demon-strate applications to reconstructing key biological systemsincluding the yeast cell cycle. In addition to network recon-struction, our algorithm finds applications in model reduc-tion and model comprehension. We argue that our recon-struction algorithm can serve as an important primitive fordata mining in systems biology applications.
title: Automatic Record Linkage using Seeded Nearest Neighbour and Support Vector Machine Classification
author: Peter Christen
abstract: The task of linking databases is an important step in anincreasing number of data mining projects, because linkeddata can contain information that is not available otherwise,or that would require time-consuming and expensive collec-tion of specific data. The aim of linking is to match and ag-gregate all records that refer to the same entity. One of themajor challenges when linking large databases is the efficientand accurate classification of record pairs into matches andnon-matches. While traditionally classification was based onmanually-set thresholds or on statistical procedures, manyof the more recently developed classification methods arebased on supervised learning techniques. They therefore re-quire training data, which is often not available in real worldsituations or has to be prepared manually, an expensive,cumbersome and time-consuming process.The author has previously presented a novel two-step ap-proach to automatic record pair classification [6, 7]. In thefirst step of this approach, training examples of high qualityare automatically selected from the compared record pairs,and used in the second step to train a support vector ma-chine (SVM) classifier. Initial experiments showed the feasi-bility of the approach, achieving results that outperformedk-means clustering. In this paper, two variations of thisapproach are presented. The first is based on a nearest-neighbour classifier, while the second improves a SVM clas-sifier by iteratively adding more examples into the trainingsets. Experimental results show that this two-step approachcan achieve better classification results than other unsuper-vised approaches.
title: Feedback Effects between Similarity and Social Influence in Online Communities
author: David Crandall,Dan Cosley,Daniel Huttenlocher,Jon Kleinberg,Siddharth Suri
abstract: A fundamental open question in the analysis of social net-works is to understand the interplay between similarity andsocial ties. People are similar to their neighbors in a socialnetwork for two distinct reasons: first, they grow to resembletheir current friends due to social influence; and second, theytend to form new links to others who are already like them,a process often termed selection by sociologists. While bothfactors are present in everyday social processes, they are intension: social influence can push systems toward unifor-mity of behavior, while selection can lead to fragmentation.As such, it is important to understand the relative effectsof these forces, and this has been a challenge due to thedifficulty of isolating and quantifying them in real settings.We develop techniques for identifying and modeling the in-teractions between social influence and selection, using datafrom online communities where both social interaction andchanges in behavior over time can be measured. We findclear feedback effects between the two factors, with risingsimilarity between two individuals serving, in aggregate, asan indicator of future interaction  but with similarity thencontinuing to increase steadily, although at a slower rate, forlong periods after initial interactions. We also consider therelative value of similarity and social influence in modelingfuture behavior. For instance, to predict the activities thatan individual is likely to do next, is it more useful to knowthe current activities of their friends, or of the people mostsimilar to them?
title: Anomaly Pattern Detection in Categorical Datasets
author: Kaustav Das,Jeff Schneider,Daniel B. Neill
abstract: We propose a new method for detecting patterns of anoma-lies in categorical datasets. We assume that anomalies aregenerated by some underlying process which affects only aparticular subset of the data. Our method consists of twosteps: we first use a local anomaly detector to identify in-dividual records with anomalous attribute values, and thendetect patterns where the number of anomalous records ishigher than expected. Given the set of anomalies flaggedby the local anomaly detector, we search over all subsets ofthe data defined by any set of fixed values of a subset of theattributes, in order to detect self-similar patterns of anoma-lies. We wish to detect any such subset of the test datawhich displays a significant increase in anomalous activityas compared to the normal behavior of the system (as indi-cated by the training data). We perform significance testingto determine if the number of anomalies in any subset ofthe test data is significantly higher than expected, and pro-pose an efficient algorithm to perform this test over all suchsubsets of the data. We show that this algorithm is able toaccurately detect anomalous patterns in real-world hospital,container shipping and network intrusion data.
title: Bypass Rates: Reducing Query Abandonment using Negative Inferences
author: Atish Das Sarma,Sreenivas Gollapudi,Samuel Ieong
abstract: We introduce a new approach to analyzing click logs by ex-amining both the documents that are clicked and those thatare bypasseddocuments returned higher in the ordering ofthe search results but skipped by the user. This approachcomplements the popular click-through rate analysis, andhelps to draw negative inferences in the click logs. We for-mulate a natural objective that finds sets of results that areunlikely to be collectively bypassed by a typical user. Thisis closely related to the problem of reducing query aban-donment. We analyze a greedy approach to optimizing thisobjective, and establish theoretical guarantees of its perfor-mance. We evaluate our approach on a large set of queries,and demonstrate that it compares favorably to the maximalmarginal relevance approach on a number of metrics includ-ing mean average precision and mean reciprocal rank.
title: De-duping URLs via Rewrite Rules
author: Anirban Dasgupta,Ravi Kumar,Amit Sasturkar
abstract: A large fraction of the URLs on the web contain dupli-cate (or near-duplicate) content. De-duping URLs is anextremely important problem for search engines, since allthe principal functions of a search engine, including crawl-ing, indexing, ranking, and presentation, are adversely im-pacted by the presence of duplicate URLs. Traditionally,the de-duping problem has been addressed by fetching andexamining the content of the URL; our approach here is dif-ferent. Given a set of URLs partitioned into equivalenceclasses based on the content (URLs in the same equivalenceclass have similar content), we address the problem of min-ing this set and learning URL rewrite rules that transformall URLs of an equivalence class to the same canonical form.These rewrite rules can then be applied to eliminate dupli-cates among URLs that are encountered for the first timeduring crawling, even without fetching their content.In order to express such transformation rules, we proposea simple framework that is general enough to capture themost common URL rewrite patterns occurring on the web;in particular, it encapsulates the DUST (Different URLswith similar text) framework [5]. We provide an efficientalgorithm for mining and learning URL rewrite rules andshow that under mild assumptions, it is complete, i.e., ouralgorithm learns every URL rewrite rule that is correct, foran appropriate notion of correctness. We demonstrate theexpressiveness of our framework and the effectiveness of ouralgorithm by performing a variety of extensive large-scaleexperiments
title: Structured Metric Learning for High Dimensional Problems
author: Jason V. Davis,Inderjit S. Dhillon
abstract: The success of popular algorithms such as k-means clustering ornearest neighbor searches depend on the assumption that the un-derlying distance functions reflect domain-specific notions of sim-ilarity for the problem at hand. The distance metric learning prob-lem seeks to optimize a distance function subject to constraints thatarise from fully-supervised or semi-supervised information. Sev-eral recent algorithms have been proposed to learn such distancefunctions in low dimensional settings. One major shortcoming ofthese methods is their failure to scale to high dimensional prob-lems that are becoming increasingly ubiquitous in modern datamining applications. In this paper, we present metric learning algo-rithms that scale linearly with dimensionality, permitting efficientoptimization, storage, and evaluation of the learned metric. Thisis achieved through our main technical contribution which pro-vides a framework based on the log-determinant matrix divergencewhich enables efficient optimization of structured, low-parameterMahalanobis distances. Experimentally, we evaluate our methodsacross a variety of high dimensional domains, including text, statis-tical software analysis, and collaborative filtering, showing that ourmethods scale to data sets with tens of thousands or more features.We show that our learned metric can achieve excellent quality withrespect to various criteria. For example, in the context of metriclearning for nearest neighbor classification, we show that our meth-ods achieve 24% higher accuracy over the baseline distance. Ad-ditionally, our methods yield very good precision while providingrecall measures up to 20% higher than other baseline methods suchas latent semantic analysis.
title: Constraint Programming for Itemset Mining
author: Luc De Raedt,Tias Guns,Siegfried Nijssen
abstract: The relationship between constraint-based mining and con-straint programming is explored by showing how the typicalconstraints used in pattern mining can be formulated foruse in constraint programming environments. The resultingframework is surprisingly flexible and allows us to combinea wide range of mining constraints in different ways. Weimplement this approach in off-the-shelf constraint program-ming systems and evaluate it empirically. The results showthat the approach is not only very expressive, but also workswell on complex benchmark problems
title: Learning Classifiers from Only Positive and Unlabeled Data
author: Charles Elkan,Keith Noto
abstract: The input to an algorithm that learns a binary classifiernormally consists of two sets of examples, where one setconsists of positive examples of the concept to be learned,and the other set consists of negative examples. However,it is often the case that the available training data are anincomplete set of positive examples, and a set of unlabeledexamples, some of which are positive and some of which arenegative. The problem solved in this paper is how to learna standard binary classifier given a nontraditional trainingset of this nature.Under the assumption that the labeled examples are se-lected randomly from the positive examples, we show thata classifier trained on positive and unlabeled examples pre-dicts probabilities that differ by only a constant factor fromthe true conditional probabilities of being positive. We showhow to use this result in two different ways to learn a classi-fier from a nontraditional training set. We then apply thesetwo new methods to solve a real-world problem: identifyingprotein records that should be included in an incompletespecialized molecular biology database. Our experiments inthis domain show that models trained using the new meth-ods perform better than the current state-of-the-art biasedSVM method for learning from positive and unlabeled ex-amples.
title: Locality Sensitive Hash Functions Based on Concomitant Rank Order Statistics
author: Kave Eshghi,Shyamsundar Rajaram
abstract: Locality Sensitive Hash functions are invaluable tools forapproximate near neighbor problems in high dimensionalspaces. In this work, we are focused on LSH schemes wherethe similarity metric is the cosine measure. The contribu-tion of this work is a new class of locality sensitive hashfunctions for the cosine similarity measure based on the the-ory of concomitants, which arises in order statistics. Con-sider n i.i.d sample pairs, {(X1, Y1), (X2, Y2), . . . , (Xn, Yn)}obtained from a bivariate distribution f(X, Y ). Concomi-tant theory captures the relation between the order statis-tics of X and Y in the form of a rank distribution given byProb(Rank(Yi)=j|Rank(Xi)=k). We exploit properties ofthe rank distribution towards developing a locality sensitivehash family that has excellent collision rate properties forthe cosine measure.The computational cost of the basic algorithm is high forhigh hash lengths. We introduce several approximationsbased on the properties of concomitant order statistics anddiscrete transforms that perform almost as well, with sig-nificantly reduced computational cost. We demonstrate thepractical applicability of our algorithms by using it for find-ing similar images in an image repository.
title: Direct Mining of Discriminative and Essential Frequent Patterns via Model-based Search Tree
author: Wei Fan,Kun Zhang,Hong Cheng,Jing Gao,Xifeng Yan,Jiawei Han,Philip Yu,Olivier Verscheure
abstract: Frequent patterns provide solutions to datasets that do nothave well-structured feature vectors. However, frequent pat-tern mining is non-trivial since the number of unique pat-terns is exponential but many are non-discriminative andcorrelated. Currently, frequent pattern mining is performedin two sequential steps: enumerating a set of frequent pat-terns, followed by feature selection. Although many meth-ods have been proposed in the past few years on how toperform each separate step efficiently, there is still limitedsuccess in eventually finding highly compact and discrimi-native patterns. The culprit is due to the inherent nature ofthis widely adopted two-step approach. This paper discussesthese problems and proposes a new and different method. Itbuilds a decision tree that partitions the data onto differentnodes. Then at each node, it directly discovers a discrimi-native pattern to further divide its examples into purer sub-sets. Since the number of examples towards leaf level isrelatively small, the new approach is able to examine pat-terns with extremely low global support that could not beenumerated on the whole dataset by the two-step method.The discovered feature vectors are more accurate on some ofthe most difficult graph as well as frequent itemset problemsthan most recently proposed algorithms but the total size istypically 50% or more smaller. Importantly, the minimumsupport of some discriminative patterns can be extremelylow (e.g. 0.03%). In order to enumerate these low supportpatterns, state-of-the-art frequent pattern algorithm eithercannot finish due to huge memory consumption or have toenumerate 101to 103times more patterns before they caneven be found. Software and datasets are available by con-tacting the author
title: Scaling Up Text Classification for Large File Systems
author: George Forman,Shyamsundar Rajaram
abstract: We combine the speed and scalability of information retrieval with the generally superior classification accuracy offered by machine learning, yielding a two-phase text classifier that can scale to very large document corpora.  We investigate the effect of different methods of formulating the query from the training set, as well as varying the query size.  In empirical tests on the Reuters RCV1 corpus of 806,000 documents, we find runtime was easily reduced by a factor of 27x, with a somewhat surprising gain in F-measure compared with traditional text classification.
title: SPIRAL: Efficient and Exact Model Identification for Hidden Markov Models
author: Yasuhiro Fujiwara,Yasushi Sakurai,Masashi Yamamuro
abstract: Hidden Markov models (HMMs) have received considerableattention in various communities (e.g, speech recognition,neurology and bioinformatic) since many applications thatuse HMM have emerged. The goal of this work is to iden-tify efficiently and correctly the model in a given datasetthat yields the state sequence with the highest likelihoodwith respect to the query sequence. We propose SPIRAL,a fast search method for HMM datasets. To reduce thesearch cost, SPIRAL efficiently prunes a significant num-ber of search candidates by applying successive approxima-tions when estimating likelihood. We perform several ex-periments to verify the effectiveness of SPIRAL. The resultsshow that SPIRAL is more than 500 times faster than thenaive method.
title: Using Ghost Edges for Classification in Sparsely Labeled Networks
author: Brian Gallagher,Hanghang Tong,Tina Eliassi-Rad,Christos Faloutsos
abstract: We address the problem of classification in partially labeled net-works (a.k.a. within-network classification) where observed classlabels are sparse. Techniques for statistical relational learning havebeen shown to perform well on network classification tasks by ex-ploiting dependencies between class labels of neighboring nodes.However, relational classifiers can fail when unlabeled nodes havetoo few labeled neighbors to support learning (during training phase)and/or inference (during testing phase). This situation arises in real-world problems when observed labels are sparse.In this paper, we propose a novel approach to within-networkclassification that combines aspects of statistical relational learn-ing and semi-supervised learning to improve classification perfor-mance in sparse networks. Our approach works by adding ghostedges to a network, which enable the flow of information fromlabeled to unlabeled nodes. Through experiments on real-worlddata sets, we demonstrate that our approach performs well acrossa range of conditions where existing approaches, such as collectiveclassification and semi-supervised learning, fail. On all tasks, ourapproach improves area under the ROC curve (AUC) by up to 15points over existing approaches. Furthermore, we demonstrate thatour approach runs in time proportional to L  E, where L is thenumber of labeled nodes and E is the number of edges.
title: Composition Attacks and Auxiliary Information in Data Privacy
author: Srivatsava Ranjit Ganta,Shiva Prasad Kasiviswanathan,Adam Smith
abstract: Privacy is an increasingly important aspect of data publishing. Rea-soning about privacy, however, is fraught with pitfalls. One ofthe most significant is the auxiliary information (also called exter-nal knowledge, background knowledge, or side information) thatan adversary gleans from other channels such as the web, publicrecords, or domain knowledge. This paper explores how one canreason about privacy in the face of rich, realistic sources of auxil-iary information. Specifically, we investigate the effectiveness ofcurrent anonymization schemes in preserving privacy when mul-tiple organizations independently release anonymized data aboutoverlapping populations.1. We investigate composition attacks, in which an adversary usesindependently anonymized releases to breach privacy. We ex-plain why recently proposed models of limited auxiliary infor-mation fail to capture composition attacks. Our experimentsdemonstrate that even a simple instance of a composition attackcan breach privacy in practice, for a large class of currently pro-posed techniques. The class includes k-anonymity and severalrecent variants.2. On a more positive note, certain randomization-based notionsof privacy (such as differential privacy) provably resist compo-sition attacks and, in fact, the use of arbitrary side information.This resistance enables stand-alone design of anonymizationschemes, without the need for explicitly keeping track of otherreleases.We provide a precise formulation of this property, and provethat an important class of relaxations of differential privacy alsosatisfy the property. This significantly enlarges the class of pro-tocols known to enable modular design.
title: Entity categorization over large document collections
author: Venkatesh Ganti,Arnd C. Konig,Rares Vernica
abstract: Extracting entities (such as people, movies) from documents and identifying the categories (such as painter, writer) they belong to enable structured querying and data analysis over unstructured document collections. In this paper, we focus on the problem of categorizing extracted entities. Most prior approaches developed for this task only analyzed the local document context within which entities occur. In this paper, we significantly improve the accuracy of entity categorization by (i) considering an entity's context across multiple documents containing it, and (ii) exploiting existing large lists of related entities (e.g., lists of actors, directors, books). These approaches introduce computational challenges because (a) the context of entities has to be aggregated across several documents and (b) the lists of related entities may be very large. We develop techniques to address these challenges. We present a thorough experimental study on real data sets that demonstrates the increase in accuracy and the scalability of our approaches.
title: Knowledge transfer via multiple model local structure mapping
author: Jing Gao,Wei Fan,Jing Jiang,Jiawei Han
abstract: The effectiveness of knowledge transfer using classification algorithms depends on the difference between the distribution that generates the training examples and the one from which test examples are to be drawn. The task can be especially difficult when the training examples are from one or several domains different from the test domain. In this paper, we propose a locally weighted ensemble framework to combine multiple models for transfer learning, where the weights are dynamically assigned according to a model's predictive power on each test example. It can integrate the advantages of various learning algorithms and the labeled information from multiple training domains into one unified classification model, which can then be applied on a different domain. Importantly, different from many previously proposed methods, none of the base learning method is required to be specifically designed for transfer learning. We show the optimality of a locally weighted ensemble framework as a general approach to combine multiple models for domain transfer. We then propose an implementation of the local weight assignments by mapping the structures of a model onto the structures of the test domain, and then weighting each model locally according to its consistency with the neighborhood structure around the test example. Experimental results on text classification, spam filtering and intrusion detection data sets demonstrate significant improvements in classification accuracy gained by the framework. On a transfer learning task of newsgroup message categorization, the proposed locally weighted ensemble framework achieves 97% accuracy when the best single model predicts correctly only on 73% of the test examples. In summary, the improvement in accuracy is over 10% and up to 30% across different problems.
title: Banded structure in binary matrices
author: Gemma C. Garriga,Esa Junttila,Heikki Mannila
abstract: A 0--1 matrix has a banded structure if both rows and columns can be permuted so that the non-zero entries exhibit a staircase pattern of overlapping rows. The concept of banded matrices has its origins in numerical analysis, where entries can be viewed as descriptions between the problem variables; the bandedness corresponds to variables that are coupled over short distances. Banded data occurs also in other applications, for example in the physical mapping problem of the human genome, in paleontological data, in network data and in the discovery of overlapping communities without cycles.We study in this paper the banded structure of binary matrices, give a formal definition of the concept and discuss its theoretical properties. We consider the algorithmic problems of computing how far a matrix is from being banded, and of finding a good submatrix of the original data that exhibits approximate bandedness. Finally, we show by experiments on real data from ecology and other applicationsthe usefulness of the concept. Our results reveal that bands exist in real datasets and that the final obtained ordering of rows and columns have natural interpretations.
title: Quantitative evaluation of approximate frequent pattern mining algorithms
author: Rohit Gupta,Gang Fang,Blayne Field,Michael Steinbach,Vipin Kumar
abstract: Traditional association mining algorithms use a strict definition of support that requires every item in a frequent itemset to occur in each supporting transaction. In real-life datasets, this limits the recovery of frequent itemset patterns as they are fragmented due to random noise and other errors in the data. Hence, a number of methods have been proposed recently to discover approximate frequent itemsets in the presence of noise. These algorithms use a relaxed definition of support and additional parameters, such as row and column error thresholds to allow some degree of "error" in the discovered patterns. Though these algorithms have been shown to be successful in finding the approximate frequent itemsets, a systematic and quantitative approach to evaluate them has been lacking. In this paper, we propose a comprehensive evaluation framework to compare different approximate frequent pattern mining algorithms. The key idea is to select the optimal parameters for each algorithm on a given dataset and use the itemsets generated with these optimal parameters in order to compare different algorithms. We also propose simple variations of some of the existing algorithms by introducing an additional post-processing step. Subsequently, we have applied our proposed evaluation framework to a wide variety of synthetic datasets with varying amounts of noise and a real dataset to compare existing and our proposed variations of the approximate pattern mining algorithms. Source code and the datasets used in this study are made publicly available.
title: Unsupervised deduplication using cross-field dependencies
author: Rob Hall,Charles Sutton,Andrew McCallum
abstract: Recent work in deduplication has shown that collective deduplication of different attribute types can improve performance. But although these techniques cluster the attributes collectively, they do not model them collectively. For example, in citations in the research literature, canonical venue strings and title strings are dependent -- because venues tend to focus on a few research areas -- but this dependence is not modeled by current unsupervised techniques. We call this dependence between fields in a record a cross-field dependence. In this paper, we present an unsupervised generative model for the deduplication problem that explicitly models cross-field dependence. Our model uses a single set of latent variables to control two disparate clustering models: a Dirichlet-multinomial model over titles, and a non-exchangeable string-edit model over venues. We show that modeling cross-field dependence yields a substantial improvement in performance -- a 58% reduction in error over a standard Dirichlet process mixture.
title: Permu-pattern : discovery of mutable permutation patterns with proximity constraint
author: Meng Hu,Jiong Yang,Wei Su
abstract: Pattern discovery in sequences is an important problem in many applications, especially in computational biology and text mining. However, due to the noisy nature of data, the traditional sequential pattern model may fail to reflect the underlying characteristics of sequence data in these applications. There are two challenges: First, the mutation noise exists in the data, and therefore symbols may be misrepresented by other symbols; Secondly, the order of symbols in sequences could be permutated. To address the above problems, in this paper we propose a new sequential pattern model called mutable permutation patterns. Since the Apriori property does not hold for our permutation pattern model, a novel Permu-pattern algorithm is devised to mine frequent mutable permutation patterns from sequence databases. A reachability property is identified to prune the candidate set. Last but not least, we apply the permutation pattern model to a real genome dataset to discover gene clusters, which shows theeffectiveness of the model. A large amount of synthetic data is also utilized to demonstrate the efficiency of the Permu-pattern algorithm.
title: Simultaneous tensor subspace selection and clustering : the equivalence of high order svd and k-means clustering
author: Heng Huang,Chris Ding,Dijun Luo,Tao Li
abstract: Singular Value Decomposition (SVD)/Principal Component Analysis (PCA) have played a vital role in finding patterns from many datasets. Recently tensor factorization has been used for data mining and pattern recognition in high index/order data. High Order SVD (HOSVD) is a commonly used tensor factorization method and has recently been used in numerous applications like graphs, videos, social networks, etc.In this paper we prove that HOSVD does simultaneous subspace selection (data compression) and K-means clustering widely used for unsupervised learning tasks. We show how to utilize this new feature of HOSVD for clustering. We demonstrate these new results using three real and large datasets, two on face images datasets and one on hand-written digits dataset. Using this new HOSVD clustering feature we provide a dataset quality assessment on many frequently used experimental datasets with expected noise levels.
title: Bridging centrality : graph mining from element level to group level
author: Woochang Hwang,Taehyong Kim,Murali Ramanathan,Aidong Zhang
abstract: Despite the pervasiveness of networks as models for real world systems ranging from the Internet, the World Wide Web to gene regulation and scientific collaborations, only a limited number of metrics capable of characterizing these systems are available. The existing metrics for characterizing networks have broad specificity and lack the selectivity for many applications. The purpose of this paper is to identify and critically evaluate a metric, termed bridging centrality, which is highly selective for identifying bridges in networks. The properties of bridges are unique compared to the other network metrics. For a diverse range of data sets, we found that networks are highly susceptible to disruption but robust to loss structural integrity upon targeted deletion of bridging nodes. A novel graph clustering approach, termed `bridge cut', utilizing bridging edges as module boundary is also proposed. The modules identified by the bridge cut algorithm are more effective than the other graph clustering methods. Thus, bridging centrality is a network metric with unique properties that may aid in network analysis from element to group level in various areas including systems biology and national security applications.
title: Interpretable nonnegative matrix decompositions
author: Saara Hyvonen,Pauli Miettinen,Evimaria Terzi
abstract: A matrix decomposition expresses a matrix as a product of at least two factor matrices. Equivalently, it expresses each column of the input matrix as a linear combination of the columns in the first factor matrix. The interpretability of the decompositions is a key issue in many data-analysis tasks. We propose two new matrix-decomposition problems: the nonnegative CX and nonnegative CUR problems, that give naturally interpretable factors. They extend the recently-proposed column and column-row based decompositions, and are aimed to be used with nonnegative matrices. Our decompositions represent the input matrix as a nonnegative linear combination of a subset of its columns (or columns and rows).We present two algorithms to solve these problems and provide an extensive experimental evaluation where we assess the quality of our algorithms' results as well as the intuitiveness of nonnegative CX and CUR decompositions. We show that our algorithms return intuitive answers with smaller reconstruction errors than the previously-proposed methods for column and column-row decompositions.
title: Fast logistic regression for text categorization with variable-length n-grams
author: Georgiana Ifrim,Gokhan Bakir,Gerhard Weikum
abstract: A common representation used in text categorization is the bag of words model (aka. unigram model). Learning with this particular representation involves typically some preprocessing, e.g. stopwords-removal, stemming. This results in one explicit tokenization of the corpus. In this work, we introduce a logistic regression approach where learning involves automatic tokenization. This allows us to weaken the a-priori required knowledge about the corpus and results in a tokenization with variable-length (word or character) n-grams as basic tokens. We accomplish this by solving logistic regression using gradient ascent in the space of all ngrams. We show that this can be done very efficiently using a branch and bound approach which chooses the maximum gradient ascent direction projected onto a single dimension (i.e., candidate feature). Although the space is very large, our method allows us to investigate variable-length n-gram learning. We demonstrate the efficiency of our approach comparedto state-of-the-art classifiers used for text categorization such as cyclic coordinate descent logistic regression and support vector machines.
title: Probabilistic latent semantic visualization : topic model for visualizing documents
author: Tomoharu Iwata,Takeshi Yamada,Naonori Ueda
abstract: We propose a visualization method based on a topic model for discrete data such as documents. Unlike conventional visualization methods based on pairwise distances such as multi-dimensional scaling, we consider a mapping from the visualization space into the space of documents as a generative process of documents. In the model, both documents and topics are assumed to have latent coordinates in a two- or three-dimensional Euclidean space, or visualization space. The topic proportions of a document are determined by the distances between the document and the topics in the visualization space, and each word is drawn from one of the topics according to its topic proportions. A visualization, i.e. latent coordinates of documents, can be obtained by fitting the model to a given set of documents using the EM algorithm, resulting in documents with similar topics being embedded close together. We demonstrate the effectiveness of the proposed model by visualizing document and movie data sets, and quantitatively compare it with conventional visualization methods.
title: Automatic identification of quasi-experimental designs for discovering causal knowledge
author: David D. Jensen,Andrew S. Fast,Brian J. Taylor,Marc E. Maier
abstract: Researchers in the social and behavioral sciences routinely rely on quasi-experimental designs to discover knowledge from large data-bases. Quasi-experimental designs (QEDs) exploit fortuitous circumstances in non-experimental data to identify situations (sometimes called "natural experiments") that provide the equivalent of experimental control and randomization. QEDs allow researchers in domains as diverse as sociology, medicine, and marketing to draw reliable inferences about causal dependencies from non-experimental data. Unfortunately, identifying and exploiting QEDs has remained a painstaking manual activity, requiring researchers to scour available databases and apply substantial knowledge of statistics. However, recent advances in the expressiveness of databases, and increases in their size and complexity, provide the necessary conditions to automatically identify QEDs. In this paper, we describe the first system to discover knowledge by applying quasi-experimental designs that were identified automatically. We demonstrate that QEDs can be identified in a traditional database schema and that such identification requires only a small number of extensions to that schema, knowledge about quasi-experimental design encoded in first-order logic, and a theorem-proving engine. We describe several key innovations necessary to enable this system, including methods for automatically constructing appropriate experimental units and for creating aggregate variables on those units. We show that applying the resulting designs can identify important causal dependencies in real domains, and we provide examples from academic publishing, movie making and marketing, and peer-production systems. Finally, we discuss the integration of QEDs with other approaches to causal discovery, including joint modeling and directed experimentation.
title: Extracting shared subspace for multi-label classification
author: Shuiwang Ji,Lei Tang,Shipeng Yu,Jieping Ye
abstract: Multi-label problems arise in various domains such as multi-topic document categorization and protein function prediction. One natural way to deal with such problems is to construct a binary classifier for each label, resulting in a set of independent binary classification problems. Since the multiple labels share the same input space, and the semantics conveyed by different labels are usually correlated, it is essential to exploit the correlation information contained in different labels. In this paper, we consider a general framework for extracting shared structures in multi-label classification. In this framework, a common subspace is assumed to be shared among multiple labels. We show that the optimal solution to the proposed formulation can be obtained by solving a generalized eigenvalue problem, though the problem is non-convex. For high-dimensional problems, direct computation of the solution is expensive, and we develop an efficient algorithm for this case. One appealing feature of theproposed framework is that it includes several well-known algorithms as special cases, thus elucidating their intrinsic relationships. We have conducted extensive experiments on eleven multi-topic web page categorization tasks, and results demonstrate the effectiveness of the proposed formulation in comparison with several representative algorithms.
title: Mining preferences from superior and inferior examples
author: Bin Jiang,Jian Pei,Xuemin Lin,David W. Cheung,Jiawei Han
abstract: Mining user preferences plays a critical role in many important applications such as customer relationship management (CRM), product and service recommendation, and marketing campaigns. In this paper, we identify an interesting and practical problem of mining user preferences: in a multidimensional space where the user preferences on some categorical attributes are unknown, from some superior and inferior examples provided by a user, can we learn about the user's preferences on those categorical attributes? We model the problem systematically and show that mining user preferences from superior and inferior examples is challenging. Although the problem has great potential in practice, to the best of our knowledge, it has not been explored systematically before. As the first attempt to tackle the problem, we propose a greedy method and show that our method is practical using real data sets and synthetic data sets.
title: Effective and efficient itemset pattern summarization : regression-based approaches
author: Ruoming Jin,Muad Abu-Ata,Yang Xiang,Ning Ruan
abstract: In this paper, we propose a set of novel regression-based approaches to effectively and efficiently summarize frequent itemset patterns. Specifically, we show that the problem of minimizing the restoration error for a set of itemsets based on a probabilistic model corresponds to a non-linear regression problem. We show that under certain conditions, we can transform the nonlinear regression problem to a linear regression problem. We propose two new methods, k-regression and tree-regression, to partition the entire collection of frequent itemsets in order to minimize the restoration error. The K-regression approach, employing a K-means type clustering method, guarantees that the total restoration error achieves a local minimum. The tree-regression approach employs a decision-tree type of top-down partition process. In addition, we discuss alternatives to estimate the frequency for the collection of itemsets being covered by the k representative itemsets. The experimental evaluation on both real and synthetic datasets demonstrates that our approaches significantly improve the summarization performance in terms of both accuracy (restoration error), and computational cost.
title: A sequential dual method for large scale multi-class linear svms
author: S. Sathiya Keerthi,S. Sundararajan,Kai-Wei Chang,Cho-Jui Hsieh,Chih-Jen Lin
abstract: Efficient training of direct multi-class formulations of linear Support Vector Machines is very useful in applications such as text classification with a huge number examples as well as features. This paper presents a fast dual method for this training. The main idea is to sequentially traverse through the training set and optimize the dual variables associated with one example at a time. The speed of training is enhanced further by shrinking and cooling heuristics. Experiments indicate that our method is much faster than state of the art solvers such as bundle, cutting plane and exponentiated gradient methods.
title: Constructing comprehensive summaries of large event sequences
author: Jerry Kiernan,Evimaria Terzi
abstract: Event sequences capture system and user activity over time. Prior research on sequence mining has mostly focused on discovering local patterns. Though interesting, these patterns reveal local associations and fail to give a comprehensive summary of the entire event sequence. Moreover, the number of patterns discovered can be large. In this paper, we take an alternative approach and build short summaries that describe the entire sequence, while revealing local associations among events.We formally define the summarization problem as an optimization problem that balances between shortness of the summary and accuracy of the data description. We show that this problem can be solved optimally in polynomial time by using a combination of two dynamic-programming algorithms. We also explore more efficient greedy alternatives and demonstrate that they work well on large datasets. Experiments on both synthetic and real datasets illustrate that our algorithms are efficient and produce high-qualityresults, and reveal interesting local structures in the data.
title: Factorization meets the neighborhood : a multifaceted collaborative filtering model
author: Yehuda Koren
abstract: Recommender systems provide users with personalized suggestions for products or services. These systems often rely on Collaborating Filtering (CF), where past transactions are analyzed in order to establish connections between users and products. The two more successful approaches to CF are latent factor models, which directly profile both users and products, and neighborhood models, which analyze similarities between products or users. In this work we introduce some innovations to both approaches. The factor and neighborhood models can now be smoothly merged, thereby building a more accurate combined model. Further accuracy improvements are achieved by extending the models to exploit both explicit and implicit feedback by the users. The methods are tested on the Netflix data. Results are better than those previously published on that dataset. In addition, we suggest a new evaluation metric, which highlights the differences among methods, based on their performance at a top-K recommendation task.
title: The structure of information pathways in a social communication network
author: Gueorgi Kossinets,Jon Kleinberg,Duncan Watts
abstract: Social networks are of interest to researchers in part because they are thought to mediate the flow of information in communities and organizations. Here we study the temporal dynamics of communication using on-line data, including e-mail communication among the faculty and staff of a large university over a two-year period. We formulate a temporal notion of "distance" in the underlying social network by measuring the minimum time required for information to spread from one node to another - a concept that draws on the notion of vector-clocks from the study of distributed computing systems. We find that such temporal measures provide structural insights that are not apparent from analyses of the pure social network topology. In particular, we define the network backbone to be the subgraph consisting of edges on which information has the potential to flow the quickest. We find that the backbone is a sparse graph with a concentration of both highly embedded edges and long-range bridges - afinding that sheds new light on the relationship between tie strength and connectivity in social networks.
title: Angle-based outlier detection in high-dimensional data
author: Hans-Peter Kriegel,Matthias S hubert,Arthur Zimek
abstract: Detecting outliers in a large set of data objects is a major data mining task aiming at finding different mechanisms responsible for different groups of objects in a data set. All existing approaches, however, are based on an assessment of distances (sometimes indirectly by assuming certain distributions) in the full-dimensional Euclidean data space. In high-dimensional data, these approaches are bound to deteriorate due to the notorious "curse of dimensionality". In this paper, we propose a novel approach named ABOD (Angle-Based Outlier Detection) and some variants assessing the variance in the angles between the difference vectors of a point to the other points. This way, the effects of the "curse of dimensionality" are alleviated compared to purely distance-based approaches. A main advantage of our new approach is that our method does not rely on any parameter selection influencing the quality of the achieved ranking. In a thorough experimental evaluation, we compare ABOD to the well-established distance-based method LOF for various artificial and a real world data set and show ABOD to perform especially well on high-dimensional data.
title: Stream prediction using a generative model based on frequent episodes in event sequences
author: Srivatsan Laxman,Vikram Tankasali,Ryen W. White
abstract: This paper presents a new algorithm for sequence prediction over long categorical event streams. The input to the algorithm is a set of target event types whose occurrences we wish to predict. The algorithm examines windows of events that precede occurrences of the target event types in historical data. The set of significant frequent episodes associated with each target event type is obtained based on formal connections between frequent episodes and Hidden Markov Models (HMMs). Each significant episode is associated with a specialized HMM, and a mixture of such HMMs is estimated for every target event type. The likelihoods of the current window of events, under these mixture models, are used to predict future occurrences of target events in the data. The only user-defined model parameter in the algorithm is the length of the windows of events used during model estimation. We first evaluate the algorithm on synthetic data that was generated by embedding (in varying levels of noise) patterns which are preselected to characterize occurrences of target events. We then present an application of the algorithm for predicting targeted user-behaviors from large volumes of anonymous search session interaction logs from a commercially-deployed web browser tool-bar.
title: Microscopic evolution of social networks
author: Jure Leskovec,Lars Backstrom,Ravi Kumar,Andrew Tomkins
abstract: We present a detailed study of network evolution by analyzing four large online social networks with full temporal information about node and edge arrivals. For the first time at such a large scale, we study individual node arrival and edge creation processes that collectively lead to macroscopic properties of networks. Using a methodology based on the maximum-likelihood principle, we investigate a wide variety of network formation strategies, and show that edge locality plays a critical role in evolution of networks. Our findings supplement earlier network models based on the inherently non-local preferential attachment.Based on our observations, we develop a complete model of network evolution, where nodes arrive at a prespecified rate and select their lifetimes. Each node then independently initiates edges according to a "gap" process, selecting a destination for each edge according to a simple triangle-closing model free of any parameters. We show analytically that the combination of the gap distribution with the node lifetime leads to a power law out-degree distribution that accurately reflects the true network in all four cases. Finally, we give model parameter settings that allow automatic evolution and generation of realistic synthetic networks of arbitrary scale.
title: Cut-and-stitch : efficient parallel learning of linear dynamical systems on smps
author: Lei Li,Wenjie Fu,Fan Guo,Todd C. Mowry,Christos Faloutsos
abstract: Multi-core processors with ever increasing number of cores per chip are becoming prevalent in modern parallel computing. Our goal is to make use of the multi-core as well as multi-processor architectures to speed up data mining algorithms. Specifically, we present a parallel algorithm for approximate learning of Linear Dynamical Systems (LDS), also known as Kalman Filters (KF). LDSs are widely used in time series analysis such as motion capture modeling, visual tracking etc. We propose Cut-And-Stitch (CAS), a novel method to handle the data dependencies from the chain structure of hidden variables in LDS, so as to parallelize the EM-based parameter learning algorithm. We implement the algorithm using OpenMP on both a supercomputer and a quad-core commercial desktop. The experimental results show that parallel algorithms using Cut-And-Stitch achieve comparable accuracy and almost linear speedups over the serial version. In addition, Cut-And-Stitch can be generalized to othermodels with similar linear structures such as Hidden Markov Models (HMM) and Switching Kalman Filters (SKF).
title: Active learning with direct query construction
author: Charles X. Ling,Jun Du
abstract: Active learning may hold the key for solving the data scarcity problem in supervised learning, i.e., the lack of labeled data. Indeed, labeling data is a costly process, yet an active learner may request labels of only selected instances, thus reducing labeling work dramatically. Most previous works of active learning are, however, pool-based; that is, a pool of unlabeled examples is given and the learner can only select examples from the pool to query for their labels. This type of active learning has several weaknesses. In this paper we propose novel active learning algorithms that construct examples directly to query for labels. We study both a specific active learner based on the decision tree algorithm, and a general active learner that can work with any base learning algorithm. As there is no restriction on what examples to be queried, our methods are shown to often query fewer examples to reduce the predictive error quickly. This casts doubt on the usefulness of the pool in pool-based active learning. Nevertheless, our methods can be easily adapted to work with a given pool of unlabeled examples.
title: Spectral domain-transfer learning
author: Xiao Ling,Wenyuan Dai,Gui-Rong Xue,Qiang Yang,Yong Yu
abstract: Traditional spectral classification has been proved to be effective in dealing with both labeled and unlabeled data when these data are from the same domain. In many real world applications, however, we wish to make use of the labeled data from one domain (called in-domain) to classify the unlabeled data in a different domain (out-of-domain). This problem often happens when obtaining labeled data in one domain is difficult while there are plenty of labeled data from a related but different domain. In general, this is a transfer learning problem where we wish to classify the unlabeled data through the labeled data even though these data are not from the same domain. In this paper, we formulate this domain-transfer learning problem under a novel spectral classification framework, where the objective function is introduced to seek consistency between the in-domain supervision and the out-of-domain intrinsic structure. Through optimization of the cost function, the label information from the in-domain data is effectively transferred to help classify the unlabeled data from the out-of-domain. We conduct extensive experiments to evaluate our method and show that our algorithm achieves significant improvements on classification performance over many state-of-the-art algorithms.
title: Mining multi-faceted overviews of arbitrary topics in a text collection
author: Xu Ling,Qiaozhu Mei,ChengXiang Zhai,Bruce Schatz
abstract: A common task in many text mining applications is to generate a multi-faceted overview of a topic in a text collection. Such an overview not only directly serves as an informative summary of the topic, but also provides a detailed view of navigation to different facets of the topic. Existing work has cast this problem as a categorization problem and requires training examples for each facet. This has three limitations: (1) All facets are predefined, which may not fit the need of a particular user. (2) Training examples for each facet are often unavailable. (3) Such an approach only works for a predefined type of topics. In this paper, we break these limitations and study a more realistic new setup of the problem, in which we would allow a user to flexibly describe each facet with keywords for an arbitrary topic and attempt to mine a multi-faceted overview in an unsupervised way. We attempt a probabilistic approach to solve this problem. Empirical experiments on different genres of text data show that our approach can effectively generate a multi-faceted overview for arbitrary topics; the generated overviews are comparable with those generated by supervised methods with training examples. They are also more informative than unstructured flat summaries. The method is quite general, thus can be applied to multiple text mining tasks in different application domains.
title: Multi-class cost-sensitive boosting with p-norm loss functions
author: Aurelie C. Lozano,Naoki Abe
abstract: We propose a family of novel cost-sensitive boosting methods for multi-class classification by applying the theory of gradient boosting to p-norm based cost functionals. We establish theoretical guarantees including proof of convergence and convergence rates for the proposed methods. Our theoretical treatment provides interpretations for some of the existing algorithms in terms of the proposed family, including a generalization of the costing algorithm, DSE and GBSE-t, and the Average Cost method. We also experimentally evaluate the performance of our new algorithms against existing methods of cost sensitive boosting, including AdaCost, CSB2, and AdaBoost.M2 with cost-sensitive weight initialization. We show that our proposed scheme generally achieves superior results in terms of cost minimization and, with the use of higher order p-norm loss in certain cases, consistently outperforms the comparison methods, thus establishing its empirical advantage.
title: On updates that constrain the features' connections during learning
author: Omid Madani,Jian Huang
abstract: In many multiclass learning scenarios, the number of classes is relatively large (thousands,...), or the space and time efficiency of the learning system can be crucial. We investigate two online update techniques especially suited to such problems. These updates share a sparsity preservation capacity: they allow for constraining the number of prediction connections that each feature can make. We show that one method, exponential moving average, is solving a "discrete" regression problem for each feature, changing the weights in the direction of minimizing the quadratic loss. We design the other method to improve a hinge loss subject to constraints, for better accuracy. We empirically explore the methods, and compare performance to previous indexing techniques, developed with the same goals, as well as other online algorithms based on prototype learning. We observe that while the classification accuracies are very promising, improving over previous indexing techniques, the scalability benefitsare preserved.
title: Weighted graphs and disconnected components : patterns and a generator
author: Mary McGlohon,Leman Akoglu,Christos Faloutsos
abstract: The vast majority of earlier work has focused on graphs which are both connected (typically by ignoring all but the giant connected component), and unweighted. Here we study numerous, real, weighted graphs, and report surprising discoveries on the way in which new nodes join and form links in a social network. The motivating questions were the following: How do connected components in a graph form and change over time? What happens after new nodes join a network -- how common are repeated edges? We study numerous diverse, real graphs (citation networks, networks in social media, internet traffic, and others); and make the following contributions: (a) we observe that the non-giant connected components seem to stabilize in size, (b) we observe the weights on the edges follow several power laws with surprising exponents, and (c) we propose an intuitive, generative model for graph growth that obeys observed patterns.
title: Finding non-redundant, statistically significant regions in high dimensional data : a novel approach to projected and subspace clustering
author: Gabriela Moise,Jorg Sander
abstract: Projected and subspace clustering algorithms search for clusters of points in subsets of attributes. Projected clustering computes several disjoint clusters, plus outliers, so that each cluster exists in its own subset of attributes. Subspace clustering enumerates clusters of points in all subsets of attributes, typically producing many overlapping clusters. One problem of existing approaches is that their objectives are stated in a way that is not independent of the particular algorithm proposed to detect such clusters. A second problem is the definition of cluster density based on user-defined parameters, which makes it hard to assess whether the reported clusters are an artifact of the algorithm or whether they actually stand out in the data in a statistical sense.We propose a novel problem formulation that aims at extracting axis-parallel regions that stand out in the data in a statistical sense. The set of axis-parallel, statistically significant regions that exist in a given dataset is typically highly redundant. Therefore, we formulate the problem of representing this set through a reduced, non-redundant set of axis-parallel, statistically significant regions as an optimization problem. Exhaustive search is not a viable solution due to computational infeasibility, and we propose the approximation algorithm STATPC. Our comprehensive experimental evaluation shows that STATPC significantly outperforms existing projected and subspace clustering algorithms in terms of accuracy.
title: Joint latent topic models for text and citations
author: Ramesh M. Nallapati,Amr Ahmed,Eric P. Xing,William W. Cohen
abstract: In this work, we address the problem of joint modeling of text and citations in the topic modeling framework. We present two different models called the Pairwise-Link-LDA and the Link-PLSA-LDA models.The Pairwise-Link-LDA model combines the ideas of LDA [4] and Mixed Membership Block Stochastic Models [1] and allows modeling arbitrary link structure. However, the model is computationally expensive, since it involves modeling the presence or absence of a citation (link) between every pair of documents. The second model solves this problem by assuming that the link structure is a bipartite graph. As the name indicates, Link-PLSA-LDA model combines the LDA and PLSA models into a single graphical model.Our experiments on a subset of Citeseer data show that both these models are able to predict unseen data better than the baseline model of Erosheva and Lafferty [8], by capturing the notion of topical similarity between the contents of the cited and citing documents. Our experiments on two different data sets on the link prediction task show that the Link-PLSA-LDA model performs the best on the citation prediction task, while also remaining highly scalable. In addition, we also present some interesting visualizations generated by each of the models.
title: Classification with partial labels
author: Nam Nguyen,Rich Caruana
abstract: In this paper, we address the problem of learning when some cases are fully labeled while other cases are only partially labeled, in the form of partial labels. Partial labels are represented as a set of possible labels for each training example, one of which is the correct label. We introduce a discriminative learning approach that incorporates partial label information into the conventional margin-based learning framework. The partial label learning problem is formulated as a convex quadratic optimization minimizing the L2-norm regularized empirical risk using hinge loss. We also present an efficient algorithm for classification in the presence of partial labels. Experiments with different data sets show that partial label information improves the performance of classification when there is traditional fully-labeled data, and also yields reasonable performance in the absence of any fully labeled data.
title: Discrimination-aware data mining
author: Dino Pedreshi,Salvatore Ruggieri,Franco Turini
abstract: In the context of civil rights law, discrimination refers to unfair or unequal treatment of people based on membership to a category or a minority, without regard to individual merit. Rules extracted from databases by data mining techniques, such as classification or association rules, when used for decision tasks such as benefit or credit approval, can be discriminatory in the above sense. In this paper, the notion of discriminatory classification rules is introduced and studied. Providing a guarantee of non-discrimination is shown to be a non trivial task. A naive approach, like taking away all discriminatory attributes, is shown to be not enough when other background knowledge is available. Our approach leads to a precise formulation of the redlining problem along with a formal result relating discriminatory rules with apparently safe ones by means of background knowledge. An empirical assessment of the results on the German credit dataset is also provided.
title: Fast collapsed gibbs sampling for latent dirichlet allocation
author: Ian Porteous,David Newman,Alexander Ihler,Arthur Asuncion,Padhraic Smyth,Max Welling
abstract: In this paper we introduce a novel collapsed Gibbs sampling method for the widely used latent Dirichlet allocation (LDA) model. Our new method results in significant speedups on real world text corpora. Conventional Gibbs sampling schemes for LDA require O(K) operations per sample where K is the number of topics in the model. Our proposed method draws equivalent samples but requires on average significantly less then K operations per sample. On real-word corpora FastLDA can be as much as 8 times faster than the standard collapsed Gibbs sampler for LDA. No approximations are necessary, and we show that our fast sampling scheme produces exactly the same results as the standard (but slower) sampling scheme. Experiments on four real world data sets demonstrate speedups for a wide range of collection sizes. For the PubMed collection of over 8 million documents with a required computation time of 6 CPU months for LDA, our speedup of 5.7 can save 5 CPU months of computation.
title: Partial least squares regression for graph mining
author: Hiroto Saigo,Nicole Kramer,Koji Tsuda
abstract: Attributed graphs are increasingly more common in many application domains such as chemistry, biology and text processing. A central issue in graph mining is how to collect informative subgraph patterns for a given learning task. We propose an iterative mining method based on partial least squares regression (PLS). To apply PLS to graph data, a sparse version of PLS is developed first and then it is combined with a weighted pattern mining algorithm. The mining algorithm is iteratively called with different weight vectors, creating one latent component per one mining call. Our method, graph PLS, is efficient and easy to implement, because the weight vector is updated with elementary matrix calculations. In experiments, our graph PLS algorithm showed competitive prediction accuracies in many chemical datasets and its efficiency was significantly superior to graph boosting (gBoost) and the naive method based on frequent graph mining.
title: Knowledge discovery of semantic relationships between words using nonparametric bayesian graph model
author: Issei Sato,Minoru Yoshida,Hiroshi Nakagawa
abstract: We developed a model based on nonparametric Bayesian modeling for automatic discovery of semantic relationships between words taken from a corpus. It is aimed at discovering semantic knowledge about words in particular domains, which has become increasingly important with the growing use of text mining, information retrieval, and speech recognition. The subject-predicate structure is taken as a syntactic structure with the noun as the subject and the verb as the predicate. This structure is regarded as a graph structure. The generation of this graph can be modeled using the hierarchical Dirichlet process and the Pitman-Yor process. The probabilistic generative model we developed for this graph structure consists of subject-predicate structures extracted from a corpus. Evaluation of this model by measuring the performance of graph clustering based on WordNet similarities demonstrated that it outperforms other baseline models.
title: Mobile call graphs : beyond power-law and lognormal distributions
author: Mukund Seshadri,Sridhar Machiraju,Ashwin Sridharan,Jean Bolot,Christos Faloutsos,Jure Leskove
abstract: We analyze a massive social network, gathered from the records of a large mobile phone operator, with more than a million users and tens of millions of calls. We examine the distributions of the number of phone calls per customer; the total talk minutes per customer; and the distinct number of calling partners per customer. We find that these distributions are skewed, and that they significantly deviate from what would be expected by power-law and lognormal distributions.To analyze our observed distributions (of number of calls, distinct call partners, and total talk time), we propose PowerTrack , a method which fits a lesser known but more suitable distribution, namely the Double Pareto LogNormal (DPLN) distribution, to our data and track its parameters over time. Using PowerTrack , we find that our graph changes over time in a way consistent with a generative process that naturally results in the DPLN distributions we observe. Furthermore, we show that this generative process lends itself to a natural and appealing social wealth interpretation in the context of social networks such as ours. We discuss the application of those results to our model and to forecasting.
title: Efficient ticket routing by resolution sequence mining
author: Qihong Shao,Yi Chen,Shu Tao,Xifeng Yan,Nikos Anerousis
abstract: IT problem management calls for quick identification of resolvers to reported problems. The efficiency of this process highly depends on ticket routing---transferring problem ticket among various expert groups in search of the right resolver to the ticket. To achieve efficient ticket routing, wise decision needs to be made at each step of ticket transfer to determine which expert group is likely to be, or to lead to the resolver.In this paper, we address the possibility of improving ticket routing efficiency by mining ticket resolution sequences alone, without accessing ticket content. To demonstrate this possibility, a Markov model is developed to statistically capture the right decisions that have been made toward problem resolution, where the order of the Markov model is carefully chosen according to the conditional entropy obtained from ticket data. We also design a search algorithm, called Variable-order Multiple active State search(VMS), that generates ticket transfer recommendations based on our model. The proposed framework is evaluated on a large set of real-world problem tickets. The results demonstrate that VMS significantly improves human decisions: Problem resolvers can often be identified with fewer ticket transfers.
title: Get another label? improving data quality and data mining using multiple, noisy labelers
author: Victor S. Sheng,Foster Provost,Panagiotis G. Ipeirotis
abstract: This paper addresses the repeated acquisition of labels for data items when the labeling is imperfect. We examine the improvement (or lack thereof) in data quality via repeated labeling, and focus especially on the improvement of training labels for supervised induction. With the outsourcing of small tasks becoming easier, for example via Rent-A-Coder or Amazon's Mechanical Turk, it often is possible to obtain less-than-expert labeling at low cost. With low-cost labeling, preparing the unlabeled part of the data can become considerably more expensive than labeling. We present repeated-labeling strategies of increasing complexity, and show several main results. (i) Repeated-labeling can improve label quality and model quality, but not always. (ii) When labels are noisy, repeated labeling can be preferable to single labeling even in the traditional setting where labels are not particularly cheap. (iii) As soon as the cost of processing the unlabeled data is not free, even the simple strategy of labeling everything multiple times can give considerable advantage. (iv) Repeatedly labeling a carefully chosen set of points is generally preferable, and we present a robust technique that combines different notions of uncertainty to select data points for which quality should be improved. The bottom line: the results show clearly that when labeling is not perfect, selective acquisition of multiple labels is a strategy that data miners should have in their repertoire; for certain label-quality/cost regimes, the benefit is substantial.
title: iSAX : indexing and mining terabyte sized time series
author: Jin Shieh,Eamonn Keogh
abstract: Current research in indexing and mining time series data has produced many interesting algorithms and representations. However, the algorithms and the size of data considered have generally not been representative of the increasingly massive datasets encountered in science, engineering, and business domains. In this work, we show how a novel multi-resolution symbolic representation can be used to index datasets which are several orders of magnitude larger than anything else considered in the literature. Our approach allows both fast exact search and ultra fast approximate search. We show how to exploit the combination of both types of search as sub-routines in data mining algorithms, allowing for the exact mining of truly massive real world datasets, containing millions of time series.
title: Efficient computation of personal aggregate queries on blogs
author: Ka Cheung Sia,Junghoo Cho,Yun Chi,Belle L. Tseng
abstract: There is an exploding amount of user-generated content on theWeb due to the emergence of "Web 2.0" services, such as Blogger,MySpace, Flickr, and del.icio.us. The participation of a large number of users in sharing their opinion on the Web has inspired researchers to build an effective "information filter" by aggregating these independent opinions. However, given the diverse groups of users on the Web nowadays, the global aggregation of the information may not be of much interest to different groups of users. In this paper, we explore the possibility of computing personalized aggregation over the opinions expressed on the Web based on a user's indication of trust over the information sources. The hope is that by employing such "personalized" aggregation, we can make the recommendation more likely to be interesting to the users. We address the challenging scalability issues by proposing an efficient method, that utilizes two core techniques: Non-Negative Matrix Factorization and Threshold Algorithm, to compute personalized aggregations when there are potentially millions of users and millions of sources within a system. We show that, through experiments on real-life dataset, our personalized aggregation approach indeed makes a significant difference in the items that are recommended and it reduces the query computational cost significantly, often more than 75%, while the result of personalized aggregation is kept accurate enough.
title: Semi-supervised approach to rapid and reliable labeling of large data sets
author: Gyorgy J. Simon,Vipin Kumar,Zhi-Li Zhang
abstract: In this paper, we propose a method, where the labeling of the data set is carried out in a semi-supervised manner with user-specified guarantees about the quality of the labeling. In our scheme, we assume that for each class, we have some heuristics available, each of which can identify instances of one particular class. The heuristics are assumed to have reasonable performance but they do not need to cover all instances of the class nor do they need to be perfectly reliable. We further assume that we have an infallible expert, who is willing to manually label a few instances. The aim of the algorithm is to exploit the cluster structure of the problem, the predictions by the imperfect heuristics and the limited perfect labels provided by the expert to classify (label) the instances of the data set with guaranteed precision (specificed by the user) with regards to each class. The specified precision is not always attainable, so the algorithm is allowed to classify some instances as dontknow. The algorithm is evaluated by the number of instances labeled by the expert, the number of dontknow instances (global coverage) and the achieved quality of the labeling. On the KDD Cup Network Intrusion data set containing 500,000 instances, we managed to label 96.6% of the instances while guaranteeing a nominal precision of 90% (with 95% confidence) by having the expert label 630 instances; and by having the expert label 1200 instances, we managed to guarantee 95% nominal precision while labeling 96.4% of the data. We also provide a case study of applying our scheme to label the network traffic collected at a large campus network.
title: Relational learning via collective matrix factorization
author: Ajit P. Singh,Geoffrey J. Gordon
abstract: Relational learning is concerned with predicting unknown values of a relation, given a database of entities and observed relations among entities. An example of relational learning is movie rating prediction, where entities could include users, movies, genres, and actors. Relations encode users' ratings of movies, movies' genres, and actors' roles in movies. A common prediction technique given one pairwise relation, for example a #users x #movies ratings matrix, is low-rank matrix factorization. In domains with multiple relations, represented as multiple matrices, we may improve predictive accuracy by exploiting information from one relation while predicting another. To this end, we propose a collective matrix factorization model: we simultaneously factor several matrices, sharing parameters among factors when an entity participates in multiple relations. Each relation can have a different value type and error distribution; so, we allow nonlinear relationships between the parameters and outputs, using Bregman divergences to measure error. We extend standard alternating projection algorithms to our model, and derive an efficient Newton update for the projection. Furthermore, we propose stochastic optimization methods to deal with large, sparse matrices. Our model generalizes several existing matrix factorization methods, and therefore yields new large-scale optimization algorithms for these problems. Our model can handle any pairwise relational schema and a wide variety of error models. We demonstrate its efficiency, as well as the benefit of sharing parameters among relations.
title: A bayesian mixture model with linear regression mixing proportions
author: Xiuyao Song,Chris Jermaine,Sanjay Ranka,John Gums
abstract: Classic mixture models assume that the prevalence of the various mixture components is fixed and does not vary over time. This presents problems for applications where the goal is to learn how complex data distributions evolve. We develop models and Bayesian learning algorithms for inferring the temporal trends of the components in a mixture model as a function of time. We show the utility of our models by applying them to the real-life problem of tracking changes in the rates of antibiotic resistance in Escherichia coli and Staphylococcus aureus. The results show that our methods can derive meaningful temporal antibiotic resistance patterns.
title: Hypergraph spectral learning for multi-label classification
author: Liang Sun,Shuiwang Ji,Jieping Ye
abstract: A hypergraph is a generalization of the traditional graph in which the edges are arbitrary non-empty subsets of the vertex set. It has been applied successfully to capture high-order relations in various domains. In this paper, we propose a hypergraph spectral learning formulation for multi-label classification, where a hypergraph is constructed to exploit the correlation information among different labels. We show that the proposed formulation leads to an eigenvalue problem, which may be computationally expensive especially for large-scale problems. To reduce the computational cost, we propose an approximate formulation, which is shown to be equivalent to a least squares problem under a mild condition. Based on the approximate formulation, efficient algorithms for solving least squares problems can be applied to scale the formulation to very large data sets. In addition, existing regularization techniques for least squares can be incorporated into the model for improved generalization performance. We have conducted experiments using large-scale benchmark data sets, and experimental results show that the proposed hypergraph spectral learning formulation is effective in capturing the high-order relations in multi-label problems. Results also indicate that the approximate formulation is much more efficient than the original one, while keeping competitive classification performance.
title: Community evolution in dynamic multi-mode networks
author: Lei Tang,Huan Liu,Jianping Zhang,Zohreh Nazeri
abstract: A multi-mode network typically consists of multiple heterogeneous social actors among which various types of interactions could occur. Identifying communities in a multi-mode network can help understand the structural properties of the network, address the data shortage and unbalanced problems, and assist tasks like targeted marketing and finding influential actors within or between groups. In general, a network and the membership of groups often evolve gradually. In a dynamic multi-mode network, both actor membership and interactions can evolve, which poses a challenging problem of identifying community evolution. In this work, we try to address this issue by employing the temporal information to analyze a multi-mode network. A spectral framework and its scalability issue are carefully studied. Experiments on both synthetic data and real-world large scale networks demonstrate the efficacy of our algorithm and suggest its generality in solving problems with complex relationships.
title: Colibri : fast mining of large static and dynamic graphs
author: Hanghang Tong,Spiros Papadimitriou,Jimeng Sun,Philip S. Yu,Christos Faloutsos
abstract: Low-rank approximations of the adjacency matrix of a graph are essential in finding patterns (such as communities) and detecting anomalies. Additionally, it is desirable to track the low-rank structure as the graph evolves over time, efficiently and within limited storage. Real graphs typically have thousands or millions of nodes, but are usually very sparse. However, standard decompositions such as SVD do not preserve sparsity. This has led to the development of methods such as CUR and CMD, which seek a non-orthogonal basis by sampling the columns and/or rows of the sparse matrix.However, these approaches will typically produce overcomplete bases, which wastes both space and time. In this paper we propose the family of Colibri methods to deal with these challenges. Our version for static graphs, Colibri-S, iteratively finds a non-redundant basis and we prove that it has no loss of accuracy compared to the best competitors (CUR and CMD), while achieving significant savingsin space and time: on real data, Colibri-S requires much less space and is orders of magnitude faster (in proportion to the square of the number of non-redundant columns). Additionally, we propose an efficient update algorithm for dynamic, time-evolving graphs, Colibri-D. Our evaluation on a large, real network traffic dataset shows that Colibri-D is over 100 times faster than the best published competitor (CMD).
title: Can complex network metrics predict the behavior of NBA teams?
author: Pedro O.S. Vaz de Melo,Virgilio A.F. Almeida,Antonio A.F. Loureiro
abstract: The United States National Basketball Association (NBA) is one of the most popular sports league in the world and is well known for moving a millionary betting market that uses the countless statistical data generated after each game to feed the wagers. This leads to the existence of a rich historical database that motivates us to discover implicit knowledge in it. In this paper, we use complex network statistics to analyze the NBA database in order to create models to represent the behavior of teams in the NBA. Results of complex network-based models are compared with box score statistics, such as points, rebounds and assists per game. We show the box score statistics play a significant role for only a small fraction of the players in the league. We then propose new models for predicting a team success based on complex network metrics, such as clustering coefficient and node degree. Complex network-based models present good results when compared to box score statistics, which underscore the importance of capturing network relationships in a community such as the NBA.
title: Model-based document clustering with a collapsed gibbs sampler
author: Daniel David Walker,Eric K. Ringger
abstract: Model-based algorithms are emerging as a preferred method for document clustering. As computing resources improve, methods such as Gibbs sampling have become more common for parameter estimation in these models. Gibbs sampling is well understood for many applications, but has not been extensively studied for use in document clustering. We explore the convergence rate, the possibility of label switching, and chain summarization methodologies for document clustering on a particular model, namely a mixture of multinomials model, and show that fairly simple methods can be employed, while still producing clusterings of superior quality compared to those produced with the EM algorithm.
title: Building semantic kernels for text classification using wikipedia
author: Pu Wang,Carlotta Domeniconi
abstract: Document classification presents difficult challenges due to the sparsity and the high dimensionality of text data, and to the complex semantics of the natural language. The traditional document representation is a word-based vector (Bag of Words, or BOW), where each dimension is associated with a term of the dictionary containing all the words that appear in the corpus. Although simple and commonly used, this representation has several limitations. It is essential to embed semantic information and conceptual patterns in order to enhance the prediction capabilities of classification algorithms. In this paper, we overcome the shortages of the BOW approach by embedding background knowledge derived from Wikipedia into a semantic kernel, which is then used to enrich the representation of documents. Our empirical evaluation with real data sets demonstrates that our approach successfully achieves improved classification accuracy with respect to the BOW technique, and to other recently developed methods.
title: A unified approach for schema matching, coreference and canonicalization
author: Michael L. Wick,Khashayar Rohanimanesh,Karl Schultz,Andrew McCallum
abstract: The automatic consolidation of database records from many heterogeneous sources into a single repository requires solving several information integration tasks. Although tasks such as coreference, schema matching, and canonicalization are closely related, they are most commonly studied in isolation. Systems that do tackle multiple integration problems traditionally solve each independently, allowing errors to propagate from one task to another. In this paper, we describe a discriminatively-trained model that reasons about schema matching, coreference, and canonicalization jointly. We evaluate our model on a real-world data set of people and demonstrate that simultaneously solving these tasks reduces errors over a cascaded or isolated approach. Our experiments show that a joint model is able to improve substantially over systems that either solve each task in isolation or with the conventional cascade. We demonstrate nearly a 50% error reduction for coreference and a 40% error reduction for schema matching.
title: Information extraction from Wikipedia : moving down the long tail
author: Fei Wu,Raphael Hoffmann,Daniel S. Weld
abstract: Not only is Wikipedia a comprehensive source of quality information, it has several kinds of internal structure (e.g., relational summaries known as infoboxes), which enable self-supervised information extraction. While previous efforts at extraction from Wikipedia achieve high precision and recall on well-populated classes of articles, they fail in a larger number of cases, largely because incomplete articles and infrequent use of infoboxes lead to insufficient training data. This paper presents three novel techniques for increasing recall from Wikipedia's long tail of sparse classes: (1) shrinkage over an automatically-learned subsumption taxonomy, (2) a retraining technique for improving the training data, and (3) supplementing results by extracting from the broader Web. Our experiments compare design variations and show that, used in concert, these techniques increase recall by a factor of 1.76 to 8.71 while maintaining or increasing precision.
title: SAIL : summation-based incremental learning for information-theoretic clustering
author: Junjie Wu,Hui Xiong,Jian Chen
abstract: Information-theoretic clustering aims to exploit information theoretic measures as the clustering criteria. A common practice on this topic is so-called INFO-K-means, which performs K-means clustering with the KL-divergence as the proximity function. While expert efforts on INFO-K-means have shown promising results, a remaining challenge is to deal with high-dimensional sparse data. Indeed, it is possible that the centroids contain many zero-value features for high-dimensional sparse data. This leads to infinite KL-divergence values, which create a dilemma in assigning objects to the centroids during the iteration process of K-means. To meet this dilemma, in this paper, we propose a Summation-based Incremental Learning (SAIL) method for INFO-K-means clustering. Specifically, by using an equivalent objective function, SAIL replaces the computation of the KL-divergence by the computation of the Shannon entropy. This can avoid the zero-value dilemma caused by the use of the KL-divergence. Our experimental results on various real-world document data sets have shown that, with SAIL as a booster, the clustering performance of K-means can be significantly improved. Also, SAIL leads to quick convergence and a robust clustering performance on high-dimensional sparse data.
title: Asymmetric support vector machines : low false-positive learning under the user tolerance
author: Shan-Hung Wu,Keng-Pei Lin,Chung-Min Chen,Ming-Syan Chen
abstract: Many practical applications of classification require the classifier to produce a very low false-positive rate. Although the Support Vector Machine (SVM) has been widely applied to these applications due to its superiority in handling high dimensional data, there are relatively little effort other than setting a threshold or changing the costs of slacks to ensure the low false-positive rate. In this paper, we propose the notion of Asymmetric Support VectorMachine (ASVM) that takes into account the false-positives and the user tolerance in its objective. Such a new objective formulation allows us to raise the confidence in predicting the positives, and therefore obtain a lower chance of false-positives. We study the effects of the parameters in ASVM objective and address some implementation issues related to the Sequential Minimal Optimization (SMO) to cope with large-scale data. An extensive simulation is conducted and shows that ASVM is able to yield either noticeable improvement in performance or reduction in training time as compared to the previous arts.
title: Succinct summarization of transactional databases : an overlapped hyperrectangle scheme
author: Yang Xiang,Ruoming Jin,David Fuhry,Feodor F. Dragan
abstract: Transactional data are ubiquitous. Several methods, including frequent itemsets mining and co-clustering, have been proposed to analyze transactional databases. In this work, we propose a new research problem to succinctly summarize transactional databases. Solving this problem requires linking the high level structure of the database to a potentially huge number of frequent itemsets. We formulate this problem as a set covering problem using overlapped hyperrectangles; we then prove that this problem and its several variations are NP-hard. We develop an approximation algorithm HYPER which can achieve a ln(k) + 1 approximation ratio in polynomial time. We propose a pruning strategy that can significantly speed up the processing of our algorithm. Additionally, we propose an efficient algorithm to further summarize the set of hyperrectangles by allowing false positive conditions. A detailed study using both real and synthetic datasets shows the effectiveness and efficiency of our approaches in summarizing transactional databases.
title: Anonymizing transaction databases for publication
author: Yabo Xu,Ke Wang,Ada Wai-Chee Fu,Philip S. Yu
abstract: This paper considers the problem of publishing "transaction data" for research purposes. Each transaction is an arbitrary set of items chosen from a large universe. Detailed transaction data provides an electronic image of one's life. This has two implications. One, transaction data are excellent candidates for data mining research. Two, use of transaction data would raise serious concerns over individual privacy. Therefore, before transaction data is released for data mining, it must be made anonymous so that data subjects cannot be re-identified. The challenge is that transaction data has no structure and can be extremely high dimensional. Traditional anonymization methods lose too much information on such data. To date, there has been no satisfactory privacy notion and solution proposed for anonymizing transaction data. This paper proposes one way to address this issue.
title: Local peculiarity factor and its application in outlier detection
author: Jian Yang,Ning Zhong,Yiyu Yao,Jue Wang
abstract: Peculiarity oriented mining (POM), aiming to discover peculiarity rules hidden in a dataset, is a new data mining method. In the past few years, many results and applications on POM have been reported. However, there is still a lack of theoretical analysis. In this paper, we prove that the peculiarity factor (PF), one of the most important concepts in POM, can accurately characterize the peculiarity of data with respect to the probability density function of a normal distribution, but is unsuitable for more general distributions. Thus, we propose the concept of local peculiarity factor (LPF). It is proved that the LPF has the same ability as the PF for a normal distribution and is the so-called -sensitive peculiarity description for general distributions. To demonstrate the effectiveness of the LPF, we apply it to outlier detection problems and give a new outlier detection algorithm called LPF-Outlier. Experimental results show that LPF-Outlier is an effective outlier detection algorithm.
title: A family of dissimilarity measures between nodes generalizing both the shortest-path and the commute-time distances
author: Luh Yen,Marco Saerens,Amin Mantrach,Masashi Shimbo
abstract: This work introduces a new family of link-based dissimilarity measures between nodes of a weighted directed graph. This measure, called the randomized shortest-path (RSP) dissimilarity, depends on a parameter  and has the interesting property of reducing, on one end, to the standard shortest-path distance when  is large and, on the other end, to the commute-time (or resistance) distance when  is small (near zero). Intuitively, it corresponds to the expected cost incurred by a random walker in order to reach a destination node from a starting node while maintaining a constant entropy (related to ) spread in the graph. The parameter  is therefore biasing gradually the simple random walk on the graph towards the shortest-path policy. By adopting a statistical physics approach and computing a sum over all the possible paths (discrete path integral), it is shown that the RSP dissimilarity from every node to a particular node of interest can be computed efficiently by solving two linear systems of n equations, where n is the number of nodes. On the other hand, the dissimilarity between every couple of nodes is obtained by inverting an n x n matrix. The proposed measure can be used for various graph mining tasks such as computing betweenness centrality, finding dense communities, etc, as shown in the experimental section.
title: Training structural svms with kernels using sampled cuts
author: Chun-Nam John Yu,Thorsten Joachims
abstract: Discriminative training for structured outputs has found increasing applications in areas such as natural language processing, bioinformatics, information retrieval, and computer vision. Focusing on large-margin methods, the most general (in terms of loss function and model structure) training algorithms known to date are based on cutting-plane approaches. While these algorithms are very efficient for linear models, their training complexity becomes quadratic in the number of examples when kernels are used. To overcome this bottleneck, we propose new training algorithms that use approximate cutting planes and random sampling to enable efficient training with kernels. We prove that these algorithms have improved time complexity while providing approximation guarantees. In empirical evaluations, our algorithms produced solutions with training and test error rates close to those of exact solvers. Even on binary classification problems where highly optimized conventional training methods exist (e.g. SVM-light), our methods are about an order of magnitude faster than conventional training methods on large datasets, while remaining competitive in speed on datasets of medium size.
title: Stable feature selection via dense feature groups
author: Lei Yu,Chris Ding,Steven Loscalzo
abstract: Many feature selection algorithms have been proposed in the past focusing on improving classification accuracy. In this work, we point out the importance of stable feature selection for knowledge discovery from high-dimensional data, and identify two causes of instability of feature selection algorithms: selection of a minimum subset without redundant features and small sample size. We propose a general framework for stable feature selection which emphasizes both good generalization and stability of feature selection results. The framework identifies dense feature groups based on kernel density estimation and treats features in each dense group as a coherent entity for feature selection. An efficient algorithm DRAGS (Dense Relevant Attribute Group Selector) is developed under this framework. We also introduce a general measure for assessing the stability of feature selection algorithms. Our empirical study based on microarray data verifies that dense feature groups remain stable under random sample hold out, and the DRAGS algorithm is effective in identifying a set of feature groups which exhibit both high classification accuracy and stability.
title: Categorizing and mining concept drifting data streams
author: Peng Zhang,Xingquan Zhu,Yong Shi
abstract: Mining concept drifting data streams is a defining challenge for data mining research. Recent years have seen a large body of work on detecting changes and building prediction models from stream data, with a vague understanding on the types of the concept drifting and the impact of different types of concept drifting on the mining algorithms. In this paper, we first categorize concept drifting into two scenarios: Loose Concept Drifting (LCD) and Rigorous Concept Drifting (RCD), and then propose solutions to handle each of them separately. For LCD data streams, because concepts in adjacent data chunks are sufficiently close to each other, we apply kernel mean matching (KMM) method to minimize the discrepancy of the data chunks in the kernel space. Such a minimization process will produce weighted instances to build classifier ensemble and handle concept drifting data streams. For RCD data streams, because genuine concepts in adjacent data chunks may randomly and rapidly change, we propose a newOptimal Weights Adjustment (OWA) method to determine the optimum weight values for classifiers trained from the most recent (up-to-date) data chunk, such that those classifiers can form an accurate classifier ensemble to predict instances in the yet-to-come data chunk. Experiments on synthetic and real-world datasets will show that weighted instance approach is preferable when the concept drifting is mainly caused by the changing of the class prior probability; whereas the weighted classifier approach is preferable when the concept drifting is mainly triggered by the changing of the conditional probability.
title: Fastanova : an efficient algorithm for genome-wide association study
author: Xiang Zhang,Fei Zou,Wei Wang
abstract: Studying the association between quantitative phenotype (such as height or weight) and single nucleotide polymorphisms (SNPs) is an important problem in biology. To understand underlying mechanisms of complex phenotypes, it is often necessary to consider joint genetic effects across multiple SNPs. ANOVA (analysis of variance) test is routinely used in association study. Important findings from studying gene-gene (SNP-pair) interactions are appearing in the literature. However, the number of SNPs can be up to millions. Evaluating joint effects of SNPs is a challenging task even for SNP-pairs. Moreover, with large number of SNPs correlated, permutation procedure is preferred over simple Bonferroni correction for properly controlling family-wise error rate and retaining mapping power, which dramatically increases the computational cost of association study.In this paper, we study the problem of finding SNP-pairs that have significant associations with a given quantitative phenotype. We propose an efficient algorithm, FastANOVA, for performing ANOVA tests on SNP-pairs in a batch mode, which also supports large permutation test. We derive an upper bound of SNP-pair ANOVA test, which can be expressed as the sum of two terms. The first term is based on single-SNP ANOVA test. The second term is based on the SNPs and independent of any phenotype permutation. Furthermore, SNP-pairs can be organized into groups, each of which shares a common upper bound. This allows for maximum reuse of intermediate computation, efficient upper bound estimation, and effective SNP-pair pruning. Consequently, FastANOVA only needs to perform the ANOVA test on a small number of candidate SNP-pairs without the risk of missing any significant ones. Extensive experiments demonstrate that FastANOVA is orders of magnitude faster than the brute-force implementation of ANOVA tests on all SNP pairs.
title: Cuts3vm : a fast semi-supervised svm algorithm
author: Bin Zhao,Fei Wang,Changshui Zhang
abstract: Semi-supervised support vector machine (S3VM) attempts to learn a decision boundary that traverses through low data density regions by maximizing the margin over labeled and unlabeled examples. Traditionally, S3VM is formulated as a non-convex integer programming problem and is thus difficult to solve. In this paper, we propose the cutting plane semi-supervised support vector machine (CutS3VM) algorithm, to solve the S3VM problem. Specifically, we construct a nested sequence of successively tighter relaxations of the original S3VM problem, and each optimization problem in this sequence could be efficiently solved using the constrained concave-convex procedure (CCCP). Moreover, we prove theoretically that the CutS3VM algorithm takes time O(sn) to converge with guaranteed accuracy, where n is the total number of samples in the dataset and s is the average number of non-zero features, i.e. the sparsity. Experimental evaluations on several real world datasets show that CutS3VM performs better than existing S3VM methods, both in efficiency and accuracy.
title: Identifying biologically relevant genes via multiple heterogeneous data sources
author: Zheng Zhao,Jiangxin Wang,Huan Liu,Jieping Ye,Yung Chang
abstract: Selection of genes that are differentially expressed and critical to a particular biological process has been a major challenge in post-array analysis. Recent development in bioinformatics has made various data sources available such as mRNA and miRNA expression profiles, biological pathway and gene annotation, etc. Efficient and effective integration of multiple data sources helps enrich our knowledge about the involved samples and genes for selecting genes bearing significant biological relevance. In this work, we studied a novel problem of multi-source gene selection: given multiple heterogeneous data sources (or data sets), select genes from expression profiles by integrating information from various data sources. We investigated how to effectively employ information contained in multiple data sources to extract an intrinsic global geometric pattern and use it in covariance analysis for gene selection. We designed and conducted experiments to systematically compare the proposed approach with representative methods in terms of statistical and biological significance, and showed the efficacy and potential of the proposed approach with promising findings.
title: Volatile correlation computation : a checkpoint view
author: Wenjun Zhou,Hui Xiong
abstract: Recent years have witnessed increased interest in computing strongly correlated pairs in very large databases. Most previous studies have been focused on static data sets. However, in real-world applications, input data are often dynamic and must continually be updated. With such large and growing data sets, new research efforts are expected to develop an incremental solution for correlation computing. Along this line, in this paper, we propose a CHECK-POINT algorithm that can efficiently incorporate new transactions for correlation computing as they become available. Specifically, we set a checkpoint to establish a computation buffer, which can help us determine an upper bound for the correlation. This checkpoint bound can be exploited to identify a list of candidate pairs, which will be maintained and computed for correlations as new transactions are added into the database. However, if the total number of new transactions is beyond the buffer size, a new upper bound is computed by the new checkpoint and a new list of candidate pairs is identified. Experimental results on real-world data sets show that CHECK-POINT can significantly reduce the correlation computing cost in dynamic data sets and has the advantage of compacting the use of memory space
