Selecting the appropriate visual presentation of the data such that it not only preserves the semantics but also provides an intuitive summary of the data is an important, often the final step of data analytics. Unfortunately, this is also a step involving significant human effort starting from selection of groups of columns in the structured results from analytics stages, to the selection of right visualization by experimenting with various alternatives. In this paper, we describe our DataVizard system aimed at reducing this overhead by automatically recommending the most appropriate visual presentation for the structured result. Specifically, we consider the following two scenarios: first, when one needs to visualize the results of a structured query such as SQL; and the second, when one has acquired a data table with an associated short description (e.g., tables from the Web). Using a corpus of real-world database queries (and their results) and a number of statistical tables crawled from the Web, we show that DataVizard is capable of recommending visual presentations with high accuracy.
In this work, we consider processing k-nearest-neighbor (k-NN) queries, with the additional requirement that the result objects are of a specific type. To solve this problem, we propose an approach based on a combination of an inverted index and state-of-the-art similarity search index structure for efficiently pruning the search space early-on. Furthermore, we provide a cost model, and an extensive experimental study, that analyzes the performance of the proposed index structure under different configurations, with the aim of finding the most efficient one for the dataset being searched.
General solutions to augment Knowledge Graphs (KGs) with facts extracted from Web tables aim to associate pairs of columns from the table with a KG relation based on the matches between pairs of entities in the table and facts in the KG. These approaches suffer from intrinsic limitations due to the incompleteness of the KGs. In this paper we investigate an alternative solution, which leverages the patterns that occur on the schemas of a large corpus of Wikipedia tables. Our experimental evaluation, which used DBpedia as reference KG, demonstrates the advantages of our approach over state-of-the-art solutions and reveals thatwe can extract more than 1.7M of facts with an estimated accuracy of 0.81 even from tables that do not expose any fact on the KG.
The proliferation of falsehood and misinformation, in particular through the Web, has lead to increasing energy being invested into journalistic fact-checking. Fact-checking journalists typically check the accuracy of a claim against some trusted data source. Statistic databases such as those compiled by state agencies are often used as trusted data sources, as they contain valuable, high-quality information. However, their usability is limited when they are shared in a format such as HTML or spreadsheets: this makes it hard to find the most relevant dataset for checking a specific claim, or to quickly extract from a dataset the best answer to a given query. We present a novel algorithm enabling the exploitation of such statistic tables, by (i) identifying the statistic datasets most relevant for a given fact-checking query, and (ii) extracting from each dataset the best specific (precise) query answer it may contain. We have implemented our approach and experimented on the complete corpus of statistics obtained from INSEE, the French national statistic institute. Our experiments and comparisons demonstrate the effectiveness of our proposed method.
Popular techniques for data cleaning use integrity constraints to identify errors in the data and to automatically resolve them, e. g. by using predefined priorities among possible updates and finding a minimal repair that will resolve violations. Such automatic solutions however cannot ensure precision of the repairs since they do not have enough evidence about the actual errors and may in fact lead to wrong results with respect to the ground truth. It has thus been suggested to use domain experts to examine the potential updates and choose which should be applied to the database. However, the sheer volume of the databases and the large number of possible updates that may resolve a given constraint violation, may make such a manual examination prohibitory expensive. The goal of the DANCE system presented here is to help to optimize the experts work and reduce as much as possible the number of questions (updates verification) they need to address. Given a constraint violation, our algorithm identifies the suspicious tuples whose update may contribute (directly or indirectly) to the constraint resolution, as well as the possible dependencies among them. Using this information it builds a graph whose nodes are the suspicious tuples and whose weighted edges capture the likelihood of an error in one tuple to occur and affect the other. PageRankstyle algorithm then allows us to identify the most beneficial tuples to ask about first. Incremental graph maintenance is used to assure interactive response time. We implemented our solution in the DANCE system and show its effectiveness and efficiency through a comprehensive suite of experiments.
Recommender systems play an important role in providing personalized information to users and helping address the information overload problem. Recent research has considered social theories and studied the importance of social influence in social recommendation systems. However, many publications ignored the users' roles information or just considered some single roles. In fact, users often have many different roles. Besides, different types of users (users with different roles) might have different conformity tendency. Thus, this inspires us to study how conformity tendency changes with users' roles in recommender systems. We firstly formalize conformity influence by defining a utility function and then propose a probabilistic graphical model integrating both users' roles and conformity tendency, named as Role Conformity Recommender Systems (RCRS). We evaluate the proposed model on several real-world datasets. The experimental results show that our model significantly outperforms state-of-the-art approaches.
We focus on the problem of "tell me more" information related to a given fact in Wikipedia. We use the novel notion of role to link information in an infobox with different places in the text of the same Wikipedia page (space) as well as information across different revisions of the page (time). In this way, it is possible to link together pieces of information that may not represent the same real world entity, yet have served in the same role. To achieve this, we introduce a novel structure called ladder that allows such spatial and temporal linking and we show how to effectively and efficiently construct such structures from Wikipedia data.
Current RDF knowledge bases (KBs) are highly incomplete. This incompleteness is a serious problem both for data users and producers. Users do not have guarantees that queries that are run on a KB deliver complete results. Data producers, on the other hand, are blind about the parts of the KB that are incomplete. Yet, completeness information management is poorly supported in the Semantic Web. No RDF storage engine supports reasoning with completeness statements. Moreover, SPARQL cannot express completeness constraints for queries. Motivated by these observations, this paper offers a vision on completeness-aware RDF querying. Our vision includes (1) the sketch of a method to reason about completeness in RDF knowledge bases, (2) two approaches to represent completeness information for SPARQL queries, and (3) an extension for the SPARQL language to express completeness constraints in queries.
When crowdsourcing to achieve some goal, or to gather information, there is a distinct advantage to choosing a diverse team of users. Past research has shown the advantages of diversity in the workplace, as team members bring different perspectives and points of view. Similarly, when choosing users from a crowd, user diversity must be taken into consideration. This paper studies the diverse team formation problem. More precisely, we are given a set of required skills, as wells as a large set of people, each of who has some subset of the skills. The goal is to form a team satisfying the skills, that is also diverse, as is reflected by differences in the characteristics of team members (e.g., gender, race, country of residence, economic bracket). We show that finding an optimal (diverse) team of people is an NP-complete problem. In practice, the number of candidates is likely to strongly dominate the number of skills and characteristics. Hence, we provide an algorithm that returns an optimal solution, while running in time that is indifferent to the number of candidates (but is exponential in the number of skills and characteristics). We also provide a polynomial method for approximating optimal team formation by a reduction to the problem of submodular function maximization with a matroid constraint. Extensive experimentation shows both scalability of our methods, and the quality of the solutions returned.
We address the problem of index pruning without compromising the quality of ad-hoc similarity search among sets and ranked lists. We discuss three different ways to prune the index structure and, by linking the index structure with the concept of Locality Sensitive Hashing (LSH), we introduce two solutions to query processing over the pruned index. Through a probabilistic analysis we ensure that a user-defined recall goal is still guaranteed. We are able to formulate an optimization problem that can determine the optimal pruning factor for all three pruning methods. The experimental evaluations over real-world data validate that the optimal pruning factor indeed ensures the recall goal without any significant effect on the quality of similarity search on a much smaller index.
Due to the requirements imposed by data-intensive applications, NoSQL and NewSQL databases are becoming more present in the IT Market. These products provide alternative data models to the relational databases, and most of them are intrinsically distributed. These database management systems (DBMSs) relax consistency to favor availability and performance. However, applications that use NoSQL/NewSQL databases in distributed environments have to perform consistency control to avoid anomalies, such as inconsistent data and behavior. New approaches suggest the use of replicated data types (RDTs) to control conflicting updates. Another strategy is the use of different consistencies models for each type of operation, using first-order logic and theorem provers to supply the programmer with tools to classify consistency in operations while maintaining system invariants. Notwithstanding, the use of RDT or the descriptions of application integrity constraints in languages using first-order logic is still difficult to be used by programmers. Aiming to simplify the definition of most common database constraints, this paper proposes a mechanism to extract usual integrity constraints in an intermediate model, taking into account the semantics of invariances, using a mix of RDTs and first-order logic. The aim of this paper is to demonstrate how the proposed mechanism simplifies and guarantees safer programming with consistency control being performed at the application level.
In this paper, we present a federated path querying language (FPQ) as itself an extension of the nested regular path querying language with adding an axis operator to support the federation of RDF dataset and relational database. We have proven that FPQ has more expressive power than the nested regular path query language (not to mention regular path querying language). It enjoys the same computational complexity as the regular path query language and its additional expressivity can be exactly used to characterize the conjunction and federation of nested regular path queries. Moreover, we discuss the expressivity of various fragments of FPQ and implement FPQ. Finally, we present an application scenario related to the car-pooling services in real life.
It is known that annotating entities in unstructured and semistructured datasets by their concepts improves the effectiveness of answering queries over these datasets. Ideally, one would like to annotate entities of all relevant concepts in a dataset. However, it takes substantial time and computational resources to annotate concepts in large datasets and an organization may have sufficient resources to annotate only a subset of relevant concepts. Clearly, it would like to annotate a subset of concepts that provides the most effective answers to queries over the dataset. We propose a formal framework that quantifies the amount by which annotating entities of concepts from a taxonomy in a dataset improves the effectiveness of answering queries over the dataset. Because the problem is NP-hard, we propose an efficient approximation for the problem. Our extensive empirical studies validate our framework and show the accuracy and efficiency of our algorithm.
In the web, page creators often compete for their ranking on relevant search queries, as high ranking attracts users and may lead to increased revenues. The positive effect of this competition is that it encourages page owners to improve their content. However, some techniques used for this improvement, such as keyword stuffing , are considered harmful. Current search engine literature largely ignores the interplay between the retrieval system and the pages' content evolvement. This paper studies the above phenomenon. We model pages as strategic players, competing for a better rank by content manipulations, while the search engine controls the ranking mechanism. We show that such competition may degrade retrieval effectiveness as irrelevant pages tend to outrank better ones. Further, we investigate how the search engine's choice of a ranking scheme may reduce the incentive to manipulate the page content. Finally, we propose a novel ranking solution that empirically minimizes the adverse effect on a real dataset.
A subset of the HTML tables on the Web contains relational data. The data in these tables covers a multitude of topics and is thus very useful for complementing or validating cross-domain knowledge bases, such as DBpedia, YAGO, or the Google Knowledge Graph. A large fraction of the data in these knowledge bases is time-dependent, meaning that the correctness of an attribute value depends on a point in time. Fusing data from web tables in order to determine correct values for time-dependent attributes is challenging as most web tables do not contain timestamp information. A possibility to deal with this sparsity is to exploit timestamps which appear in different locations on the web page around the table. But as these timestamps might not apply to the web table value in question, this approach introduces noise. This paper investigates the extent to which the performance of data fusion strategies that rely on voting, PageRank, and Knowledge-Based-Trust can be improved by incorporating noisy and sparse timestamp information. For this, we present a machine-learning-based approach which considers different types of noisy timestamps in the data fusion process, and experiment with propagating timestamp information between web tables in order to overcome sparsity. We evaluate the data fusion strategies using a large public corpus of web tables and a public gold standard of time-dependent attribute values. We find that our methods effectively choose and weigh timestamp information per attribute and reduce sparsity using propagation. By incorporating timestamp information into data fusion strategies that previously did not exploit temporal meta information, we are able to increase F1-measure on average by 5%.
Distributed systems are now commonly used to manage massive data flooding from physical world, such as user-generated contents from online social media and communication records from mobile phones. The new generation of distributed data management systems, such as HBase, are designed to accept tuple insertions only, such that other database operations (e.g., deletion and update) are simply simulated by appending operation logs with keys associated to the target tuples. Such append-only store architecture maximizes the processing throughput on incoming data, but potentially incur higher costs on query processing, in which additional computation is needed to generate consistent snapshots of the database. Indexing is known as the key to enable efficient query processing by fast data retrieval and aggregation under such system architecture. This paper presents a new indexing scheme for distributed append-only stores. Our new scheme utilizes traditional index structures based on B-trees and its variant without the overhead of expensive node split, by using template-based tree construction. Optimized domain partitioning and multi-thread insertion techniques are further proposed to exploit the advantages of our template B-tree structure. Empirical evaluations show that our proposal outperforms existing solutions on insertion throughput by a large margin on a variety of real and synthetic workloads.
There are a large number of implicit enterprise users in social media, who register as ordinary users but act like enterprise ones. It is a fundamental task to eliminate this type of users from the user set since their existing will severely affect the performance of many applications like network demographic investigation and targeted advertisement. Despite of its importance, this problem is surprisingly unexplored. In this paper, we present a novel two-stage framework to identify the implicit enterprise users. In the first stage, we detect this type of users by only using the profile information. In the second stage, we improve the classification accuracy by integrating more information from the user's contents. Such a framework meets well the scenario in reality: 1) due to the very nature of preprocessing, the identification of implicit enterprise users needs to be handled quickly with little cost, and 2) if enough resources such as microblog data and training/test time are provided, the problem is expected to be solved more precisely. We conduct extensive experiments on a real data set consisting of 400 users (200 ordinary users and 200 implicit enterprise users, respectively) in Sina Weibo, one of the largest social media in China. The result reaches an accuracy of 81.75% when using only profile features and is further improved to 86.00% when the content features are combined.
In this paper we present a framework for learning mixtures of Mallows models from large samples of incomplete preferences. The problem we address is of significant practical importance in social choice, recommender systems, and other domains where it is required to aggregate, or otherwise analyze, preferences of a heterogeneous user base. We improve on state-of-the-art methods for learning mixtures of Mallows models with pairwise preference data. Exact sampling from the Mallows posterior in presence of arbitrary pairwise evidence is known to be intractable even for a single Mallows. This motivated to the development of an approximate sampler called AMP. In this paper we propose AMPx, an ensemble method for approximate sampling from the Mallows posterior that combines AMP with frequency-based estimation of posterior probabilities. We experimentally demonstrate that AMPx achieves faster convergence and higher accuracy than AMP alone. We also adapt state-of-the-art clustering techniques that have not been used in this setting, for learning parameters of the Mallows mixture, and show experimentally that mixture parameters can be learned accurately and efficiently.
Many approaches have been introduced recently to automatically create or augment Knowledge Graphs (KGs) with facts extracted from Wikipedia, particularly its structured components like the infoboxes. Although these structures are valuable, they represent only a fraction of the actual information expressed in the articles. In this work, we quantify the number of highly accurate facts that can be harvested with high precision from the text of Wikipedia articles using information extraction techniques bootstrapped from the entities and relations already in a KG. Our experimental evaluation, which uses Freebase as reference KG, reveals we can augment several relations in the domain of people by more than 10%, with facts whose accuracy are over 95%. Moreover, the vast majority of these facts are missing from the infoboxes, YAGO and DBpedia.
