As datasets continue to grow in size and complexity, finding succinct yet accurate data summaries poses a key challenge. Centroid-based clustering, a widely adopted approach to address this challenge, finds informative summaries of datasets in terms of few prototypes, each representing a cluster in the data. Despite their wide adoption, the resulting data summaries often contain redundancies, limiting their effectiveness particularly in datasets characterized by a large number of underlying clusters. To overcome this limitation, we introduce the Khatri-Rao clustering paradigm that extends traditional centroid-based clustering to produce more succinct but equally accurate data summaries by postulating that centroids arise from the interaction of two or more succinct sets of protocentroids. We study two central approaches to centroid-based clustering, namely the well-established k-Means algorithm and the increasingly popular topic of deep clustering, under the lens of the Khatri-Rao paradigm. To this end, we introduce the Khatri-Rao k-Means algorithm and the Khatri-Rao deep clustering framework. Extensive experiments show that Khatri-Rao k-Means can strike a more favorable trade-off between succinctness and accuracy in data summarization than standard k-Means. Leveraging representation learning, the Khatri-Rao deep clustering framework offers even greater benefits, reducing even more the size of data summaries given by deep clustering while preserving their accuracy.

The problem of approximating a matrix by a low-rank one has been extensively studied. This problem assumes, however, that the whole matrix has a low-rank structure. This assumption is often false for real-world matrices. We consider the problem of discovering submatrices from the given matrix with bounded deviations from their low-rank approximations. We introduce an effective two-phase method for this task: first, we use sampling to discover small nearly low-rank submatrices, and then they are expanded while preserving proximity to a low-rank approximation. An extensive experimental evaluation confirms that the method we introduce compares favorably to existing approaches.

An increasing body of evidence indicates that transcription and splicing are coupled and it is accepted that chromatin organization and DNA modification regulate transcription. Little is known, however, about the cross-talk between chromatin structure and splicing. We continue to examine how RNA polymerase II and DNA modifications mediate cross-talk between chromatin structure and splicing (see Schwartz et al., Nature Structural and Molecular Biology, 2009). We also study splicing-related genetic diseases like the neurodegenerative disease Familial Dysautonomia and the link between splicing and various cancer types (lung and colon cancer, for example) using molecular and computational methods. Finally, we study the role splicing plays in microRNA (miRNA) regulation as well.

In this note, we discuss the applicability of latent variable models as a tool in analyzing the structure of a research system. We consider whether tensor methods, especially Parallel Factor Analysis, are appropriate for the description of the personnel structure and publication results of different scientific disciplines in different universities. As the measured variables (personnel structure and publications) interact with both the universities and the disciplines, it is useful to view the data as a tensor. Our preliminary results suggest that tensor methods are indeed able to find meaningful structure in such data.

A chronofauna is a geographically restricted collection of interacting animal populations that maintains its base structure over a long period of time. We describe a simple computational method that can identify candidate chronofaunas on the basis of presence-absence matrices only: A candidate chronofauna is a collection of sites that share an exceptionally large number of taxa with the defining site of the chronofauna. We show examples of candidate chronofaunas in the NOW data (see http://www.helsinki.fi/science/now).

Finding out whether a word occurs significantly more often in one text or corpus than in another is an important question in analysing corpora. As noted by Kilgarriff (Language is never, ever, ever, random, Corpus Linguistics and Linguistic Theory , 2005; 1(2): 263–76.), the use of the χ2 and log-likelihood ratio tests is problematic in this context, as they are based on the assumption that all samples are statistically independent of each other. However, words within a text are not independent. As pointed out in Kilgarriff (Comparing corpora, International Journal of Corpus Linguistics , 2001; 6(1): 1–37) and Paquot and Bestgen (Distinctive words in academic writing: a comparison of three statistical tests for keyword extraction. In Jucker, A., Schreier, D., and Hundt, M. (eds), Corpora: Pragmatics and Discourse . Amsterdam: Rodopi, 2009, pp. 247–69), it is possible to represent the data differently and employ other tests, such that we assume independence at the level of texts rather than individual words. This allows us to account for the distribution of words within a corpus. In this article we compare the significance estimates of various statistical tests in a controlled resampling experiment and in a practical setting, studying differences between texts produced by male and female fiction writers in the British National Corpus. We find that the choice of the test, and hence data representation, matters. We conclude that significance testing can be used to find consequential differences between corpora, but that assuming independence between all words may lead to overestimating the significance of the observed differences, especially for poorly dispersed words. We recommend the use of the t-test, Wilcoxon rank-sum test, or bootstrap test for comparing word frequencies across corpora.

Large sparse sets of binary transaction data with millions of records and thousands of attributes occur in various domains: customers purchasing products, users visiting web pages, and documents containing words are just three typical examples. Real-time query selectivity estimation (the problem of estimating the number of rows in the data satisfying a given predicate) is an important practical problem for such databases. We investigate the application of probabilistic models to this problem. In particular, we study a Markov random field (MRF) approach based on frequent sets and maximum entropy, and compare it to the independence model and the Chow-Liu tree model. We find that the MRF model provides substantially more accurate probability estimates than the other methods but is more expensive from a computational and memory viewpoint. To alleviate the computational requirements we show how one can apply bucket elimination and clique tree approaches to take advantage of structure in the models and in the queries. We provide experimental results on two large real-world transaction datasets.

Background The object of this study was to identify temperament patterns in the Finnish population, and to determine the relationship between these profiles and life habits, socioeconomic status, and health. Methods/Principal Findings A cluster analysis of the Temperament and Character Inventory subscales was performed on 3,761 individuals from the Northern Finland Birth Cohort 1966 and replicated on 2,097 individuals from the Cardiovascular Risk in Young Finns study. Clusters were formed using the k-means method and their relationship with 115 variables from the areas of life habits, socioeconomic status and health was examined. Results Four clusters were identified for both genders. Individuals from Cluster I are characterized by high persistence, low extravagance and disorderliness. They have healthy life habits, and lowest scores in most of the measures for psychiatric disorders. Cluster II individuals are characterized by low harm avoidance and high novelty seeking. They report the best physical capacity and highest level of income, but also high rate of divorce, smoking, and alcohol consumption. Individuals from Cluster III are not characterized by any extreme characteristic. Individuals from Cluster IV are characterized by high levels of harm avoidance, low levels of exploratory excitability and attachment, and score the lowest in most measures of health and well-being. Conclusions This study shows that the temperament subscales do not distribute randomly but have an endogenous structure, and that these patterns have strong associations to health, life events, and well-being.

Traditional data mining applications consider the problem of mining a single relation between two attributes. For example, in a scientific bibliography database, authors are related to papers, and we may be interested in discovering association rules between authors. However, in real life, we often have multiple attributes related though chains of relations. For example, authors write papers, and papers concern one or more topics. Mining such relational chains poses additional challenges. In this paper we consider the following problem: given a chain of two relations R/sub 1/ (A, P) and R/sub 2/(P, T) we want to find selectors for the objects in T such that the projected relation between A and P satisfies a specific property. The motivation for our approach is that a given property might not hold on the whole dataset, but it might hold when projecting the data on a selector set. We discuss various algorithms and we examine the conditions under which the a priori technique can be used. We experimentally demonstrate the effectiveness of our methods.

Background Investigation of the environmental influences on human behavioral phenotypes is important for our understanding of the causation of psychiatric disorders. However, there are complexities associated with the assessment of environmental influences on behavior. Methods/Principal Findings We conducted a series of analyses using a prospective, longitudinal study of a nationally representative birth cohort from Finland (the Northern Finland 1966 Birth Cohort). Participants included a total of 3,761 male and female cohort members who were living in Finland at the age of 16 years and who had complete temperament scores. Our initial analyses (Wessman et al., in press) provide evidence in support of four stable and robust temperament clusters. Using these temperament clusters, as well as independent temperament dimensions for comparison, we conducted a data-driven analysis to assess the influence of a broad set of life course measures, assessed pre-natally, in infancy, and during adolescence, on adult temperament. Results Measures of early environment, neurobehavioral development, and adolescent behavior significantly predict adult temperament, classified by both cluster membership and temperament dimensions. Specifically, our results suggest that a relatively consistent set of life course measures are associated with adult temperament profiles, including maternal education, characteristics of the family’s location and residence, adolescent academic performance, and adolescent smoking. Conclusions Our finding that a consistent set of life course measures predict temperament clusters indicate that these clusters represent distinct developmental temperament trajectories and that information about a subset of life course measures has implications for adult health outcomes.

Aim  Our aims were to test: (1) the extent to which vascular plant associations are related in space to mammalian associations, and (2) whether the plant associations are more closely related than the mammalian associations to climate and to a published environmental stratification of Europe.

Traditional data mining methods consider the problem of mining a single relation that relates two different attributes. For example, in a scientific bibliography database, authors are related to papers, and we may be interested in discovering association rules between authors based on the papers that they have co-authored. However, in real life it is often the case that we have multiple attributes related through chains of relations. For example, authors write papers, and papers belong to one or more topics, defining a three-level chain of relations. In this paper we consider the problem of mining such relational chains. We formulate a generic problem of finding selector sets (subsets of objects from one of the attributes) such that the projected dataset—the part of the dataset determined by the selector set—satisfies a specific property. The motivation for our approach is that a given property might not hold on the whole dataset, but holds when projecting the data on a subset of objects. We show that many existing and new data mining problems can be formulated in the framework. We discuss various algorithms and identify the conditions when apriori technique can be used. We experi- mentally demonstrate the effectiveness and efficiency of our methods.

Multidimensional 0-1 data occurs in many domains. Typically one assumes that the order of rows and columns has no importance. However, in some applications, e.g., in ecology, there is structure in the data that becomes visible only when the rows and columns are permuted in a certain way. Examples of such structure are different forms of nestedness and bandedness. I review some of the applications, intuitions, results, and open problems in this area.

Finding who and what is "important" is an ever-occurring question. Many methods that aim at characterizing important items or influential individuals have been developed in areas such as, bibliometrics, social-network analysis, link analysis, and web search. In this paper we study the problem of attributing influence scores to individuals who accomplish tasks in a collaborative manner. We assume that individuals build small teams, in different and diverse ways, in order to accomplish atomic tasks. For each task we are given an assessment of success or importance score, and the goal is to attribute those team-wise scores to the individuals. The challenge we face is that individuals in strong coalitions are favored against individuals in weaker coalitions, so the objective is to find fair attributions that account for such biasing. We propose an iterative algorithm for solving this problem that is based on the concept of Shapley value. The proposed method is applicable to a variety of scenarios, for example, attributing influence scores to scientists who collaborate in published articles, or employees of a company who participate in projects. Our method is evaluated on two real datasets: ISI Web of Science publication data and the Internet Movie Database.

Correlation between occurrences of taxa is a fundamental concept in the analysis of presence-absence data. Such correlations can result from ecologically relevant processes, such as existence and evolution of species communities. Correlations are typically quantified by some sort of similarity index based on co-occurrence counts. We argue that the individual values of a similarity index are not useful as such: rather, we have to be able to estimate the statistical significance of the index value. Secondly, we argue that before computing the correlations one has to carefully select what is the underlying base set of locations for which the co-occurrence counts, similarity indices, and their significance is computed. We demonstrate base set selection with synthetic examples and conclude with an analysis of real data from a large database of fossil land mammals.