Computational notebooks are commonly used for iterative workflows, such as in exploratory data analysis. This process lends itself to the accumulation of old code and hidden state, making it hard for users to reason about the lineage of, e.g., plots depicting insights or trained machine learning models. One way to reason about code used to generate various notebook data artifacts is to compute a program slice , but traditional static approaches to slicing can be both inaccurate (failing to contain relevant code for artifacts) and conservative (containing unnecessary code for an artifacts). We present nbslicer, a dynamic slicer optimized for the notebook setting whose instrumentation for resolving dynamic data dependencies is both bolt-on (and therefore portable) and switchable (allowing it to be selectively disabled in order to reduce instrumentation overhead). We demonstrate Nbslicer's ability to construct small and accurate backward slices (i.e., historical cell dependencies) and forward slices (i.e., cells affected by the "rerun" of an earlier cell), thereby improving reproducibility in notebooks and enabling faster reactive re-execution, respectively. Comparing nbslicer with a static slicer on 374 real notebook sessions, we found that nbslicer filters out far more superfluous program statements while maintaining slice correctness, giving slices that are, on average, 66% and 54% smaller for backward and forward slices, respectively.

Computational notebooks are commonly used for iterative workflows, such as in exploratory data analysis. This process lends itself to the accumulation of old code and hidden state, making it hard for users to reason about the lineage of, e.g., plots depicting insights or trained machine learning models. One way to reason about code used to generate various notebook data artifacts is to compute a program slice , but traditional static approaches to slicing can be both inaccurate (failing to contain relevant code for artifacts) and conservative (containing unnecessary code for an artifacts). We present nbslicer, a dynamic slicer optimized for the notebook setting whose instrumentation for resolving dynamic data dependencies is both bolt-on (and therefore portable) and switchable (allowing it to be selectively disabled in order to reduce instrumentation overhead). We demonstrate Nbslicer's ability to construct small and accurate backward slices (i.e., historical cell dependencies) and forward slices (i.e., cells affected by the "rerun" of an earlier cell), thereby improving reproducibility in notebooks and enabling faster reactive re-execution, respectively. Comparing nbslicer with a static slicer on 374 real notebook sessions, we found that nbslicer filters out far more superfluous program statements while maintaining slice correctness, giving slices that are, on average, 66% and 54% smaller for backward and forward slices, respectively.

Aggregating data is fundamental to data analytics, data exploration, and OLAP. Approximate query processing (AQP) techniques are often used to accelerate computation of aggregates using samples, for which confidence intervals (CIs) are widely used to quantify the associated error. CIs used in practice fall into two categories: techniques that are tight but not correct, i.e., they yield tight intervals but only offer asymptotic guarantees, making them unreliable, or techniques that are correct but not tight, i.e., they offer rigorous guarantees, but are overly conservative, leading to confidence intervals that are too loose to be useful. In this paper, we develop a CI technique that is both correct and tighter than traditional approaches. Starting from conservative CIs, we identify two issues they often face: pessimistic mass allocation (PMA) and phantom outlier sensitivity (PHOS). By developing a novel range-trimming technique for eliminating PHOS and pairing it with known CI techniques without PMA, we develop a technique for computing CIs with strong guarantees that requires fewer samples for the same width. We implement our techniques underneath a sampling-optimized in-memory column store and show how to accelerate queries involving aggregates on a real dataset with speedups of up to 124x over traditional AQP-with-guarantees and more than 1000x over exact methods.

Recently, computational notebooks have emerged as essential tools that enable scientists and engineers to perform exploratory data analysis with especially tight feedback. Jupyter [4] in particular has gained widespread popularity. With an estimated 4.7 million notebooks on GitHub as ofMarch 2019, it has been called “data scientists’ computational notebook of choice” [9]. Furthermore, its impact has been formally recognized by the ACM Software System award in 2018 [8], and continued shared interest has led to the emergence of the JupyterCon conference. All the evidence suggests that computational notebooks, and especially Jupyter, have cemented themselves as essential data tools that will be with us for years to come. Despite their popularity, notebooks have a number of drawbacks that are well-documented in academic [5, 9] and industry [2, 3, 10] literature. While the specific complaints about notebook behavior vary, they all stem from the inherent difficulty in manual management of the global state that notebooks keep persistent in memory. This global state can be highly dependent on the order in which cells are run,making it difficult to rectifywith code visible on screen. This issue is further exacerbated by the ability to reorder, rerun, edit, and delete notebook cells. However, existing approaches [5, 10] sacrifice flexible any-order execution semantics [6] of notebooks, prompting us to ask the question: can we have our cake (by reducing errors in notebooks) and eat it too (keeping existing notebook semantics)? Leveraging Wisdom from the DB Community. To reduce errors and reproducibility problems in notebooks, we propose that, just as a relational database pushes responsibility of managing data integrity from application logic to a DBMS, we need ways to push management of notebook state from the brain of the user down into smarter notebook kernels. In doing so, we can enable a number of desirable properties toward making notebooks safer. 1. Atomicity of Cell Execution. We propose that notebooks should obey transactional semantics when cells are executed. For example, if a user forgets to define a variable or import a package, we should warn the user, instead of leaving the cell in a partially-executed state — particularly egregious if the cell is not idempotent, and the user goes to re-execute it after fixing any undefined references. 2. Idempotence of Cell Executions. Idempotent cell executions are desirable in their own right, since cells can be re-executed an arbitrary number of times. For example, if a user runs a cell that increments a counter variable, we should detect that this cell is not idempotent. In this way, the user can be warned if they attempt to execute it again, in case they did so by accident. 3. Obedience of DataflowConstraints. Additionally, notebooks should be able to infer dataflow constraints without explicit guidance from users, based solely on the dependencies between variables and cells. In this way, if a dependency changes, the notebook can automatically determine which cells are unsafe to execute (because

Computational notebooks have emerged as the platform of choice for data science and analytical workflows, enabling rapid iteration and exploration. By keeping intermediate program state in memory and segmenting units of execution into so-called "cells", notebooks allow users to enjoy particularly tight feedback. However, as cells are added, removed, reordered, and rerun, this hidden intermediate state accumulates, making execution behavior difficult to reason about, and leading to errors and lack of reproducibility. We present nbsafety, a custom Jupyter kernel that uses runtime tracing and static analysis to automatically manage lineage associated with cell execution and global notebook state. nbsafety detects and prevents errors that users make during unaided notebook interactions, all while preserving the flexibility of existing notebook semantics. We evaluate nbsafety's ability to prevent erroneous interactions by replaying and analyzing 666 real notebook sessions. Of these, nbsafety identified 117 sessions with potential safety errors, and in the remaining 549 sessions, the cells that nbsafety identified as resolving safety issues were more than 7X more likely to be selected by users for re-execution compared to a random baseline, even though the users were not using nbsafety and were therefore not influenced by its suggestions.

Dataframes are a popular abstraction to represent, prepare, and analyze data. Despite the remarkable success of dataframe libraries in Rand Python, dataframes face performance issues even on moderately large datasets. Moreover, there is significant ambiguity regarding dataframe semantics. In this paper we lay out a vision and roadmap for scalable dataframe systems. To demonstrate the potential in this area, we report on our experience building MODIN, a scaled-up implementation of the most widely-used and complex dataframe API today, Python's pandas. With pandas as a reference, we propose a simple data model and algebra for dataframes to ground discussion in the field. Given this foundation, we lay out an agenda of open research opportunities where the distinct features of dataframes will require extending the state of the art in many dimensions of data management. We discuss the implications of signature data-frame features including flexible schemas, ordering, row/column equivalence, and data/metadata fluidity, as well as the piecemeal, trial-and-error-based approach to interacting with dataframes.

Bloom filters are a form of existence index which offer compressed representations of sets at the cost of some false positives. Recent work has introduced the notion of learned Bloom filters, which can leverage salient differences between in-index elements and out-of-index queries to offer further compression. In this extended abstract, we show that learned filters can handle multidimensional data particularly well, giving space savings of up to 72% in our experiments. We show how to maximize performance by leveraging three key optimizations: separate modeling of high-cardinality versus low-cardinality dimensions, Bloom filter sandwiching, and robust learning.

Machine learning workflow development is a process of trial-and-error: developers iterate on workflows by testing out small modifications until the desired accuracy is achieved. Unfortunately, existing machine learning systems focus narrowly on model training---a small fraction of the overall development time---and neglect to address iterative development. We propose Helix, a machine learning system that optimizes the execution across iterations---intelligently caching and reusing, or recomputing intermediates as appropriate. Helix captures a wide variety of application needs within its Scala DSL, with succinct syntax defining unified processes for data preprocessing, model specification, and learning. We demonstrate that the reuse problem can be cast as a Max-Flow problem, while the caching problem is NP-Hard. We develop effective lightweight heuristics for the latter. Empirical evaluation shows that Helix is not only able to handle a wide variety of use cases in one unified workflow but also much faster, providing run time reductions of up to 19x over state-of-the-art systems, such as DeepDive or KeystoneML, on four real-world applications in natural language processing, computer vision, social and natural sciences.

In exploratory data analysis, analysts often have a need to identify histograms that possess a specific distribution, among a large class of candidate histograms, e.g., find countries whose income distribution is most similar to that of Greece. This distribution could be a new one that the user is curious about, or a known distribution from an existing histogram visualization. At present, this process of identification is brute-force, requiring the manual generation and evaluation of a large number of histograms. We present FastMatch: an end-to-end approach for interactively retrieving the histogram visualizations most similar to a user-specified target, from a large collection of histograms. The primary technical contribution underlying FastMatch is a probabilistic algorithm, HistSim, a theoretically sound sampling-based approach to identify the top- k closest histograms under ℓ 1 distance. While HistSim can be used independently, within FastMatch we couple HistSim with a novel system architecture that is aware of practical considerations, employing asynchronous block-based sampling policies. FastMatch obtains near-perfect accuracy with up to 35× speedup over approaches that do not use sampling on several real-world datasets.

Development of machine learning (ML) workflows is a tedious process of iterative experimentation: developers repeatedly make changes to workflows until the desired accuracy is attained. We describe our vision for a "human-in-the-loop" ML system that accelerates this process: by intelligently tracking changes and intermediate results over time, such a system can enable rapid iteration, quick responsive feedback, introspection and debugging, and background execution and automation. We finally describe Helix, our preliminary attempt at such a system that has already led to speedups of upto 10x on typical iterative workflows against competing systems.

This paper addresses the Data-Diff problem: given a dataset and a subsequent version of the dataset, find the shortest sequence of operations that transforms the dataset to the subsequent version, under a restricted family of operations. We consider operations similar to SQL UPDATE, each with a condition (WHERE) that matches a subset of tuples and a modifier (SET) that makes changes to those matched tuples. We characterize the problem based on different constraints on the attributes and the allowed conditions and modifiers, providing complexity classification and algorithms in each case.

Recent literature has shown a wide variety of benefits to mapping traditional one-hot representations of words and phrases to lower-dimensional real-valued vectors known as word embeddings. Traditionally, most word embedding algorithms treat each word as the finest meaningful semantic granularity and perform embedding by learning distinct embedding vectors for each word. Contrary to this line of thought, technical domains such as scientific and medical literature compose words from subword structures such as prefixes, suffixes, and root-words as well as compound words. Treating individual words as the finest-granularity unit discards meaningful shared semantic structure between words sharing substructures. This not only leads to poor embeddings for text corpora that have long-tail distributions, but also heuristic methods for handling out-of-vocabulary words. In this paper we propose SubwordMine, an entropy-based subword mining algorithm that is fast, unsupervised, and fully data-driven. We show that this allows for great cross-domain performance in identifying semantically meaningful subwords. We then investigate utilizing the mined subwords within the FastText embedding model and compare performance of the learned representations in a downstream language modeling task.

We develop HINRec, a new recommendation model which is capable of incorporating extra relational information present in heterogeneous information networks (HINs) to improve recommendation quality. HINRec models sparse node behaviors in HINs, scaling with the total number of edges of all relations which participate in inference. HINRec explicitly models correlations in node behavior between different relations by allowing latent vector representations of the same node to participate in multiple relations. In contrast with previous models, HINRec captures multi-view user behavior and item perception in a completely probabilistic setting, as it allows nodes to express similar behavior between relations in different amounts. We demonstrate HINRec’s effectiveness at user-item recommendation on a variety datasets with additional relational information present (e.g., userfriend or movie-director interaction). We give a scalable variational algorithm for posterior inference, and we show that it effectively leverages the additional data present in multi-relational HINs in order to improve prediction accuracy, outperforming previous Gamma-Poisson models and other strong baselines. CCS Concepts •Information systems → Collaborative filtering;

We examine the problem of authorship attribution in collaborative documents. We seek to develop new deep learning models tailored to this task. We have curated a novel dataset by parsing Wikipedia’s edit history, which we use to demonstrate the feasiblity of deep models to multi-author attribution at the sentence-level. Though we attempt to formulate models which learn stylometric features based on both grammatical structure and vocabulary, our error analysis suggests that our models mostly learn to recognize vocabulary-based cues, making them non-competitive with baselines tailored to vocabulary-based features. We explore why this may be, and suggest directions for future models to mitigate this shortcoming.

For this project, we consider the problem of routing messages between nodes in Small World [7] and related networks – i.e., networks that are embedded in some metric space, where the probability that a given edge is constructed is inversely proportional to the distance between its corresponding nodes. Our analysis differs from existing approaches in that we assume that each node in the network is assigned a priori some probability of failure independently from some underlying distribution, possibly identically for each node, or possibly from some family indexed by some property of the node (such as its degree). Given a sequence of random (source, target) pairs, we seek to minimize the total expected cost of routing a message from source to target over a fixed number of runs, where the cost is some function of path length and number of failures. We consider the aforementioned task in the context where each node failure rate is unknown, and each node v is given O(deg(v)) memory with which to estimate the failure rate of each of its neighbors. In this setting, we derive mathematical results about how to route using only local information, and we propose a number of strategies. We further test several of these strategies on synthetic data.

In neuroscience, collaboration and data sharing are undermined by concerns over the management of protected health information (PHI) and personal identifying information (PII) in neuroimage datasets. The HIPAA Privacy Rule mandates measures for the preservation of subject privacy in neuroimaging studies. Unfortunately for the researcher, the management of information privacy is a burdensome task. Wide scale data sharing of neuroimages is challenging for three primary reasons: (i) A dearth of tools to systematically expunge PHI/PII from neuroimage data sets, (ii) a facility for tracking patient identities in redacted datasets has not been produced, and (iii) a sanitization workflow remains conspicuously absent. This article describes the XNAT Redaction Toolkit-an integrated redaction workflow which extends a popular neuroimage data management toolkit to remove PHI/PII from neuroimages. Quickshear defacing is also presented as a complementary technique for deidentifying the image data itself. Together, these tools improve subject privacy through systematic removal of PII/PHI.

We developed minimal perfect hash functions for a variety of datasets using the probabilistic process of simulated annealing (SA). The SA solution structure is a tree representing an annealed program (algorithm). This solution structure is similar to the structure used in genetic programming. When executed, the SA program produces multiple hash functions for the given data set. An initial hash function called the distribution function is generated. This function attempts to uniformly place the keys into bins in preparation for a minimal perfect hash function determined later. For each trial, and for every data set of various size tested, our algorithm annealed a minimal perfect hash function. Our algorithm is applied to datasets of strings from the English language and to a list of URL's. Bloat control is used to ensure a small fixed depth limit to our solution, to simplify function complexity, and to ensure fast evaluation. Experimental results show that our algorithm generates hash functions that outperform both widely known non-minimal, nonperfect hashing schemes as well as other recent algorithms from the literature. Keywords—Minimal Perfect Hash Functions; Differential Evolution; Simulated Annealing; Genetic Programming;