Real-world datasets are often incomplete due to data collection cost, privacy considerations or as a side effect of data integration/preparation. We focus on answering aggregate queries on such datasets, where data incompleteness causes the answers to be inaccurate. To address this problem, assuming typical relational data, existing work generates synthetic data to complete the database, a challenging task, especially in the presence of bias in observed data. Instead, we propose a paradigm shift by learning to directly estimate query answers, circumventing the difficult data generation step. Our approach, dubbed NeuroComplete, learns to answer queries in three steps. First, NeuroComplete generates a set of queries for which accurate answers can be computed given the incomplete dataset. Next, it embeds queries in a feature space, through which each query is effectively represented with the portion of the database that contributes to the query answer. Finally, it trains a neural network in a supervised learning fashion: both query features (input) and correct answers (labels) are known. The learned model generates accurate answers to new queries at test time, exploiting the generalizability of the learned model in the embedding space. Extensive experimental results on real datasets show up to 4 times for AVG queries and 10 times for COUNT queries error reduction compared with the state-of-the-art.
The Wearables for Health Toolkit (W4H Toolkit) is an open-source platform that provides a robust, end-to-end solution for the centralized management and analysis of wearable data. With integrated tools and frameworks, the toolkit facilitates seamless data acquisition, integration, storage, analysis, and visualization of both stored and streaming data from various wearable devices. The W4H Toolkit is designed to provide medical researchers and health practitioners with a unified framework that enables the analysis of health-related data for various clinical applications. We provide an overview of the system and demonstrate how it can be used by health researchers to import and analyze a wide range of wearable data and perform data analysis, highlighting the versatility and functionality of the system across diverse healthcare domains and applications.
In temporal interaction networks, such as financial transaction networks, vertices model entities which exchange quantities (e.g., money) over time. We study the problem of identifying the origin of the quantities that flow into the vertices of the network over time. We consider various models of flow relay, which are related to different application scenarios and develop corresponding techniques for flow provenance.
In the last decade, substantial progress has been made towards standardizing the syntax of graph query languages, and towards understanding their semantics and complexity of evaluation. In this paper, we consider temporal property graphs (TPGs) and propose temporal regular path queries (TRPQs) that incorporate time into TPG navigation. Starting with design principles, we propose a natural syntactic extension of the MATCH clause of popular graph query languages. We then formally present the semantics of TRPQs, and study the complexity of their evaluation. We show that TRPQs can be evaluated in polynomial time if TPGs are time-stamped with time points, and identify fragments of the TRPQ language that admit efficient evaluation over a more succinct interval-annotated representation. Finally, we implement a fragment of the language in a state-of-the-art dataflow framework, and experimentally demonstrate that TRPQ can be evaluated efficiently.
Recently, many knowledge graph embedding models for knowledge graph completion have been proposed, ranging from the initial translation-based models such as TransE to recent convolutional neural network (CNN) models such as ConvE. However, these models only focus on semantic information of knowledge graph and neglect the natural graph structure information. Although graph convolutional network (GCN)-based models for knowledge graph embedding have been introduced to address this issue, they still suffer from fact incompleteness, resulting in the unconnectedness of knowledge graph. To solve this problem, we propose a novel model called deep relational graph infomax (DRGI) with mutual information (MI) maximization which takes the benefit of complete structure information and semantic information together. Specifically, the proposed DRGI consists of two encoders which are two identical adaptive relational graph attention networks (ARGATs), corresponding to catching semantic information and complete structure information respectively. Our method establishes new state-of-the-art on the standard datasets for knowledge graph completion. In addition, by exploring the complete structure information, DRGI embraces the merits of faster convergence speed over existing methods and better predictive performance for entities with small indegree.
Community search (CS) on graphs returns the largest densely connected vertex subset containing a query vertex, namely k-community, where every vertex's degree in the induced subgraph is not less than $k$ . It has significant influence in many real-life applications including event organization and friend recommendation. Many complex networks such as social networks and protein-protein interaction (PPI) networks are often modeled as uncertain graphs. In this paper, we identify and study the problem of reliable community search on uncertain graphs (UCS for short). Given an uncertain graph, a query vertex $q$ , a positive integer $k$ and a probability threshold θ, the reliable community, viz., (k, θ) -community, of $q$ is the largest vertex subset, so that the probability of every vertex to be in $q$ 's k-community is not less than θ. We prove that it is a NP-hard problem. We propose two novel pruning strategies to reduce the candidate set to a much smaller size. We develop an efficient index, namely CD-index, with which the pruning process can be done in optimal time. We also present efficient sampling algorithms on top of stratified sampling and lazy sampling to accelerate the search under accuracy guarantees. Extensive experiments using four real-world datasets demonstrate the superior performance of proposed algorithms to the state-of-the-art approaches.
The sparse interactions between users and items on the web have aggravated the difficulty of their representations in recommender systems. Existing approaches leverage tags to alleviate the data sparsity problem, so as to enhance the performance and interpretability of recommendation. However, directly using flat item tags fails to fully exploit the hierarchical relations in data, but tag taxonomies are not always available. To this end, we propose TaxoRec to jointly construct a tag taxonomy automatically and perform recommendation accurately in hyperbolic space. Specifically, we first leverage hyperbolic space and enable the optimization of a discrete taxonomy structure via a representation-aware scoring function and an adaptive clustering algorithm, and preserve the hierarchical structure for interpretability. Then, we propose to capture the complex relations among users, items, and tags in a unified hyperbolic metric space, where a novel tag-enhanced aggregation mechanism and tag-enhanced metric learning algorithm for users and items are defined. Extensive experiments on four real-world benchmark datasets show drastic performance gains brought by our proposed TaxoRec framework11https://github.com/Melinda315/TaxoRec, which constantly achieves an average of 7.76% improvement over the state-of-the-art baselines regarding both Recall and NDCG metrics. Insightful case studies also show that our automatically constructed tag taxonomies are highly accurate and interpretable.
Data federations notably increase the amount of data available for data-intensive applications such as smart mobility planning and public health emergency responses. Yet they also challenge the conventional implementation of range aggregation queries because the raw data cannot be shared within the federation and the data partition at each data silo is fixed during query processing. In this work, we propose the first-of-its-kind approximate algorithms for efficient range aggregation over spatial data federation. We devise novel single-silo sampling algorithms that process queries in parallel and design a level sampling based algorithm which reduces the time complexity of local queries at each data silo to $O(\log\frac{1}{\epsilon})$ , where ∊ is the approximation ratio of the accuracy guarantee. Extensive experiments on real-world dataset validate the efficiency and effectiveness of the solutions.
Investigating the frequency and distribution of small subgraphs with a few nodes/edges, i.e., motifs, is an effective analysis method for static networks. Motif-driven analysis is also useful for temporal networks where the spectrum of motifs is significantly larger due to the additional temporal information on edges. This variety makes it challenging to design a temporal motif model that can consider all aspects of temporality. In the literature, previous works have introduced various models that handle different characteristics. In this work, we compare the existing temporal motif models and evaluate the facets of temporal networks that are overlooked in the literature. We first survey four temporal motif models and highlight their differences. Then, we evaluate the advantages and limitations of these models with respect to the temporal inducedness and timing constraints. In addition, we suggest a new lens, event pairs, to investigate temporal correlations. We believe that our comparative survey and extensive evaluation will catalyze the research on temporal network motif models.
Conjunctive path queries (CPQ) are one of the most frequently used queries for complex graph analysis. However, current graph indexes are not tailored to fully support the power of query languages to express CPQs. Consequently, current methods do not take advantage of significant pruning opportunities during $CPQ$ evaluation, resulting in poor query processing performance. We propose the CPQ-aware path index CPQx, the first path index tailored to the expressivity of CPQ. CPQx is built on the partition of the set of source-target vertex pairs of paths in a graph based on the structural notion of path-bisimulation. Path-bisimulation is an equivalence relation on paths such that each partition block induced by the relation consists of paths in the graph indistinguishable with respect to CPQs. This language-aware partitioning of the graph can significantly reduce the cost of query evaluation. We present methods to support the full index life cycle: index construction, maintenance, and query processing with our index. We also develop interest-aware CPQx to reduce index size and index construction overhead while accelerating query evaluation for queries of interest. We demonstrate through extensive experiments on 14 real graphs that our methods accelerate query processing by up to multiple orders of magnitude over the state-of-the-art methods, with smaller index sizes. Our complete C++ codebase is available as open source for further research.
Molecular dynamics (MD) has been widely used in today's scientific research across multiple domains including materials science, biochemistry, biophysics, and structural biology. MD simulations can produce extremely large amounts of data in that each simulation could involve a large number of atoms (up to trillions) for a large number of timesteps (up to hundreds of millions). In this paper, we perform an in-depth analysis of a number of MD simulation datasets and then develop an efficient error-bounded lossy compressor that can significantly improve the compression ratios. The contributions are fourfold. (1) We characterize a number of MD datasets and summarize two commonly-used execution models. (2) We develop an adaptive error-bounded lossy compression framework (called MDZ), which can optimize the compression for both execution models adaptively by taking advantage of their specific characteristics. (3) We compare our solution with six other state-of-the-art related works by using three MD simulation packages each with multiple configurations. Experiments show that our solution has up to 233 % higher compression ratios than the second-best lossy compressor in most cases. (4) We demonstrate that MDZ is fully capable of handing particle data beyond MD simulations.
Most real-world data come with explicitly defined domain orders; e.g., lexicographic for strings, numeric for integers, and chronological for time. Our goal is to discover implicit domain orders that we do not already know; for instance, that the order of months in the Chinese Lunar calendar is Corner $<$ Apricot $<$ Peach. To do so, we enhance data profiling methods by discovering implicit domain orders in data through order dependencies. We enumerate tractable special cases and show that the general case is NP-complete but can be effectively handled by a SAT solver. We also devise an interestingness measure to rank the discovered implicit domain orders. Based on an extensive suite of experiments with real-world data, we establish the efficacy of our algorithms.
Community search is a problem that seeks cohesive and connected subgraphs in a graph that satisfy certain topology constraints, e.g., degree constraints. The majority of existing works focus exclusively on the topology and ignore the nodes' influence in the communities. To tackle this deficiency, influential community search is further proposed to include the node's influence. Each node has a weight, namely influence value, in the influential community search problem to represent its network influence. The influence value of a community is produced by an aggregated function, e.g., max, min, avg, and sum, over the influence values of the nodes in the same community. The objective of the influential community search problem is to locate the top-r communities with the highest influence values while satisfying the topology constraints. Existing studies on influential community search have several limitations: (i) they focus exclusively on simple aggregation functions such as min, which may fall short of certain requirements in many real-world scenarios, and (ii) they impose no limitation on the size of the community, whereas most real-world scenarios do. This motivates us to conduct a new study to fill this gap. We consider the problem of identifying the top-r influential communities with/without size constraints while using more complicated aggregation functions such as sum or avg. We give a theoretical analysis demonstrating the hardness of the problems and propose efficient and effective heuristic solutions for our topr influential community search problems. Extensive experiments on real large graphs demonstrate that our proposed solution is significantly more efficient than baseline solutions.
Most real-world datasets contain label noise, which can negatively affect downstream ML models trained on them. To deal with this problem, one can clean the mislabeled data before training, which is not only time-consuming and expensive but also requires domain expertise. Another approach is to use a noise-robust ML training algorithm. However, existing methods have some prerequisites that may not be practical in many applications (e.g., they are tied to specific downstream model architecture or they are applicable to specific noise distributions). In this paper, we propose a model-agnostic approach for learning with noisy labels of arbitrary distributions. In particular, our approach can work with any gradient descent optimization based machine learning model and deal with any label noise distribution. We achieve them by proposing two theoretically grounded noise-robust loss functions (for different noise distributions), and we are able to automatically decide which loss function to use based on a novel noise setting detection module. We directly learn the required hyper-parameters in the loss functions via meta-learning technique to minimize the loss on a given small clean validation set, and propose several strategies to improve the efficiency of training. Experiments on multiple datasets with both real-world and injected label noise show that our method performs better than state-of-the-art approaches.
SimRank is a popular link-based similarity measure on graphs. It enables a variety of applications with different modes of querying. In this paper, we propose UISim, a unified and incremental framework for all SimRank modes based on a scheduled approximation principle. UISim processes queries with incremental and prioritized exploration of the entire computation space, and thus allows flexible tradeoff of time and accuracy. On the other hand, it creates and shares common “building blocks” for online computation without relying on indexes, and thus is efficient to handle both static and dynamic graphs. Our experiments on various real-world graphs show that to achieve the same accuracy, UISim runs faster than its respective state-of-the-art baselines, and scales well on larger graphs.
Data annotation has been a key boost for the artificial intelligence. However, difficult tasks such as fine-grained classification need lots of labeled data to train a feasible model. On the one hand, using people who have expert knowledge on the datasets to annotate all data can be costly. On the other hand, amateurs are cheaper but not able to give precise labels. Related works like machine labeling need labeled data to start up. Crowd-Model labeling can hardly solve complex tasks like fine-grained classification. Lately, combining domain experts and cost-effective crowd to solve complex tasks has become an area of increasing interest in research and industry. However, most works rarely investigate the cost gap between experts and amateurs and see how it influences the final annotation cost. In this paper, we combine both experts and amateurs to build a cost-effective data annotation system called COCA. COCA annotates the target dataset from scratch and save costs by our annotation assignment strategy. Extensive evaluations show that when reaching the same precision, COCA can reach a lower cost than SOTA automatic labeling models when the ratio of expert price to amateur price is above a certain value.
Knowledge graphs capture the complex relationships among various entities, which can be found in various real world applications, e.g., Amazon product graph, Freebase, and COVID-19. To facilitate the knowledge graph analytical tasks, a system that supports interactive and efficient query processing is always in demand. In this demonstration, we develop a prototype system, CheetahKG, that embeds with our state-of-the-art query processing engine for the top-k frequent pattern discovery. Such discovered patterns can be used for two purposes, (i) identifying related patterns and (ii) guiding knowledge exploration. In the demonstration sessions, the attendees will be invited to test the efficiency and effectiveness of the query engine and use the discovered patterns to analyze knowledge graphs on CheetahKG.
Task assignment is a crucial issue in spatial crowdsourcing. In most existing studies, the results of the task assignment cannot satisfy the workers and tasks at the same time. This is because only one-sided preferences are taken into account. Moreover, tasks are always assigned based on the locations of workers instead of the trajectories. Accordingly, they are not appropriate to the specific applications, such as carpool. Inspired by this, we investigate an interesting problem of task assignment, namely bilateral preference-aware task assignment (BPTA), with the goal of maximizing the overall satisfaction of workers and tasks by assigning tasks to suitable workers based on their routine trajectories. To tackle this problem effectively, we first propose greedy algorithms, namely task preference priority greedy and worker preference priority greedy algorithms, which are task-driven and worker-driven, respectively. Although these algorithms can solve the BPTA problem effectively, they cannot ensure the stability of the task assignment results. In other words, there can be better choices for some workers and tasks. Accordingly, we further explore deferred acceptance algorithms to find a stable matching for workers and tasks by simultaneously considering the preferences of workers and tasks. Moreover, two optimizing strategies, including a parallel strategy and a top-k strategy, are introduced to boost the performance in handling the BPTA problem. Extensive experiments on both real and synthetic datasets have validated the efficiency and effectiveness of our proposed algorithms.
We propose variational quasi-recurrent autoencoders (VQRAEs) to enable robust and efficient anomaly detection in time series in unsupervised settings. The proposed VQRAEs employs a judiciously designed objective function based on robust divergences, including a, ß, and, -divergence, making it possible to separate anomalies from normal data without the reliance on anomaly labels, thus achieving robustness and fully unsupervised training. To better capture temporal dependencies in time series data, VQRAEs are built upon quasi-recurrent neural networks, which employ convolution and gating mechanisms to avoid the inefficient recursive computations used by classic recurrent neural networks. Further, VQRAEs can be extended to bi-directional Bi VQRAEs that utilize bi-directional information to further improve the accuracy. The above design choices make VQRAEs not only robust and thus accurate, but also efficient at detecting anomalies in streaming settings. Experiments on five real-world time series offer insight into the design properties of VQRAEs and demonstrate that VQRAEs are capable of outperforming state-of-the-art methods.
Analytics on modern data analytic and data ware-house systems often need to run large complex queries on increasingly complex database schemas. A lot of progress has been made on executing such complex queries using techniques like scale out query processing, hardware accelerators like GPUs and code generation techniques. However, optimization of such queries remains a challenge. Existing optimal solutions either cannot be effectively parallelized, or are inefficient while doing a lot of unnecessary work. In this demonstration, we present our system, GPU-QO, which aims to demonstrate query optimization techniques for large analytical queries using GPUs. We first demonstrate Massively Parallel Dynamic Programming (MPDP) - a novel query optimization technique that can run on GPUs to generate optimal plans in a (massively) parallel and efficient manner. We then showcase IDP 2 -MPDP and UnionDP - two heuristic techniques, again using GPUs, that can even optimize queries containing 1000s of joins. Furthermore, we compare our techniques with current state-of-the-art solutions, and demonstrate how our techniques can reduce optimization time for optimal solutions by nearly two orders of magnitude and produce much better query plans for heuristics (up to 7x).
