Invoking trained machine learning (ML) pipelines to perform intelligent analysis on data stored in databases has gradually become an essential requirement for many applications. Considering performance and privacy demands, prior studies translate ML pipelines into pure SQL queries for native execution. They typically rely on translating encoders into CASE expressions. Unfortunately, databases generally do not deeply optimize CASE expressions, as they are not first-class citizens in query optimizers, which may cause inefficient performance of generated queries. To overcome this limitation, we introduce join-based translation approaches, which convert encoders into joins, thus leveraging the high-performance join execution of modern databases. Furthermore, multiple encoders may have various join translation combinations, and join-based translations do not always outperform the case-based ones. This leads to a huge space of candidate translation plans that generate queries having drastically varying execution time. Hence, we propose a plan selector to address this challenge. It employs dynamic programming-based and priority-based selection strategies to identify an efficient translation plan from the huge search space. Moreover, we implement an ML2SQL framework, namely EncoderForge, deployable as a plugin across various databases. Experimental results demonstrate that queries generated by EncoderForge achieve up to an order-of-magnitude speedup compared with those produced by existing translation approaches.

In-database prediction queries that apply machine learning (ML) pipelines to perform data analysis are prevalent in many applications. Since data stored in databases is typically tabular, tree-based models are particularly well-suited and thus widely adopted for such tasks. When ML inference with a decision tree or random forest appears on a SQL predicate, existing works first perform ML inference and then determine whether the inference result satisfies the predicate. However, this leads to redundant feature tests on the tree node during the predicate evaluation. To determine whether one data record satisfies the predicate, it is possible to perform feature tests only on partial internal nodes of the tree. We identify that these redundant feature tests are caused by specific sibling and ancestor nodes. In particular, we propose the sibling-centric elimination with the merging-based subtree collapse method, and the ancestor-centric elimination with the sliding-based subtree recombination method. We implement a prototype system, called ReTree, based on DuckDB. Our experiments show that ReTree achieves a 2.56x speedup on average over DuckDB for prediction query execution and outperforms other solutions.

Multi-layer Perceptron (MLP)-based models have been widely exploited by modern recommendation applications. In practice, industrial recommendation scenarios frequently launch continuous incremental training jobs with only one epoch to capture real-time user features. This kind of job is shorter than full training and has a larger proportion of feature processing time. To fully utilize fragmentation resources, our model engineering team at Tencent explores resource-constrained CPU clusters to perform such incremental training workloads. To improve the efficiency of such workloads, we notice scheduling optimizations by overlapping feature processing and model training at the level of data processing pipelines. In particular, we propose an intra-pipeline scheduling strategy, which prefetches feature processing operators dynamically to fill the idle time of CPUs during the communication of embedding lookup. Furthermore, we propose an inter-pipeline scheduling strategy, which balances the resource demands of different pipelines. It prioritizes the execution of critical pipelines and overlaps the communication in critical pipelines with the execution of non-critical pipelines. Based on the two scheduling strategies, we implement a novel incremental recommendation training framework called RECS on top of TensorFlow. In our experimental studies, RECS achieves a speedup of 1.36x over existing solutions on industrial workloads.

Prediction queries that apply machine learning (ML) models to perform analysis on data stored in the database are prevalent with the advance of research. Current database systems introduce Python UDFs to express prediction queries and call ML frameworks for inference. However, the impedance mismatch between database engines and prediction query execution imposes a challenge for query performance. First, the database engine is oblivious to the internal semantics of prediction functions and evaluates the UDF holistically, which incurs the repetitive inference context setup. Second, the invocation of prediction functions in the database does not consider that batching inference with a desirable inference batch size achieves a high performance in ML frameworks. To mitigate the mismatch, we propose to employ a prediction-aware operator in database engines, which leverages inference context reuse cache to achieve an automatic one-off inference context setup and batch-aware function invocation to ensure desirable batching inference. We implement a prototype system, called IMBridge, based on an open-source database OceanBase. Our experiments show that IMBridge achieves a 71.4x speedup on average over OceanBase for prediction query execution and significantly outperforms other solutions.

Deploying machine learning (ML) inference pipelines in databases become increasingly prevalent in many applications. In order to avoid data transfer between the database and ML runtimes, existing ML2SQL frameworks parse ML pipelines to a graph consisting of ML operators and then translate it into pure SQL. Nevertheless, they typically rewrite the graph without operator fusion or only consider the fusion between certain operators such as StandardScaler and tree inference. However, there are various operators in ML pipelines, which have rich fusion opportunities between each other. To fully exploit operator fusion for graph rewriting, we classify widely used ML operators and design fusion rules driven by their characteristics. Moreover, rewriting the original graph by fusion rules produces candidate graphs that generate SQLs with different execution time. We employ an enumeration-based strategy to search for the graph with the lowest cost. However, this strategy may suffer from the combination explosion on search space for complex ML pipelines. To reduce this space, we propose a greedy-based strategy by exploiting the independence among ML operators. We implement a novel ML2SQL framework as a portable plugin for databases, namely Craftsman. Our experimental evaluations show that, in comparison to the existing approaches, Craftsman generates efficient SQL queries which achieves an average speedup of 2.9x on popular databases such as DuckDB.

A session-based recommendation has become a hot research topic, which seeks to recommend the next item based on anonymous behavior sequences in a short time. While previous methods have made many efforts to address the complex information relationships between items, we contend that they still suffer from two inherent limitations: 1) they fail to consider the noisy preference information typically contained in user behavior sequences and 2) they are unaware of the importance of complex high-order relationships between non-adjacent items. In light of this, we contribute a novel solution named CCL (short for Contextaugmented Contrastive Learning), which takes into account the joint effect of interest graph construction, context vectors, and contrastive learning. CCL decomposes session-based recommendation workflow into three steps. First, we adopt metric-based learning to reconstruct loose item sequences into tight item interest maps, making it easier to distinguish between the primary and secondary interests of users. Then, we propose adding a context vector to each session to provide a natural way to convey information beyond adjacent items. Finally, to improve the robustness of the model, we designed a contrastive self-supervised learning module as an auxiliary task to jointly learn the representation of items in the session. Extensive experiments have been conducted on two real-world datasets from different scenarios, demonstrating the superiority of CCL against several state-of-the-art methods.

Deep CNNs are increasingly prevalent in various application domains such as image processing. To explain a CNN prediction, it is popular to employ occlusion-based explanations (OBE). OBE helps users understand which parts of an image are important to a CNN prediction. Existing systems have explored incremental evaluation to accelerate CNN inference in OBE. However, they are oblivious that incremental evaluation does not always outperform full evaluation for certain layers. To address this issue, we propose a hybrid evaluation to efficiently interleave full and incremental evaluations during the CNN inference. Ad-ditionally, it employs a cost model to compare the overhead costs of two types of evaluations and a heuristic method to determine the efficient plan combination for common CNNs. More impor-tantly, hybrid evaluation adopts a dynamic programming-based method for attention-based CNNs. In particular, the dynamic programming-based method significantly reduces the overhead of searching for the efficient plan combination on the complex DAG structure. To demonstrate the efficiency of our techniques, we implement HyInJ, a hybrid CNN inf erence system based on PyTorch. Our experiments show that HyInf reduces execution time by up to 22% on GPU and 55% on CPU in comparison to the state-of-the-art incremental evaluation.

Deep learning based recommendation models are widely used in various applications. There are often dozens of groups of sparse features in the input of the recommendation model, and each group of features computes the embedding layer independently and applies a separate feature interaction. However, current deep learning frameworks sequentially schedule the execution of all operators into a single CUDA computational stream. Therefore, we propose StreamRec, a stream-based parallel inference system. It assigns the processing of individual features to different CUDA streams for parallel execution. Besides, StreamRec is able to visualize the execution performance and operators assignment results on the web.

Prediction queries that apply machine learning (ML) models to perform analysis on data stored in the database are prevalent with the advance of research. Thanks to the prosperity of ML frameworks in Python, current database systems introduce Python UDFs into query engines for inference invocation. However, there are impedance mismatches between database engines and prediction query execution with this approach. In particular, the database engine is oblivious to the semantics within prediction functions, which incurs the repetitive inference context setup. Moreover, the evaluation of the prediction function is coupled with the operator, which results in an undesirable inference batch size with low inference throughput. To mitigate these, we propose a system called IMBridge, which leverages aprediction function rewriter to eliminate redundant inference context setup and introduces adecoupled prediction operator to ensure that the evaluation batch size matches the desirable inference batch size. In this demonstration, we will showcase how IMBridge addresses these mismatches and boosts prediction query execution.

Synchronous distributed data parallel (SDDP) training is widely employed in distributed deep learning systems to train DNN models on large datasets. The performance of SDDP training essentially depends on the communication overhead and the statistical efficiency. However, existing approaches only optimize either the communication overhead or the statistical efficiency to accelerate SDDP training. In this paper, we adopt the advantages of those approaches and design a new approach, namely SkipSMA, that benefits from both low communication overhead and high statistical efficiency. In particular, we exploit the skipping strategy with an adaptive interval to decrease the communication frequency, which guarantees low communication overhead. Moreover, we employ the correction technique to mitigate the divergence while keeping small batch sizes, which ensures high statistical efficiency. To demonstrate the performance of SkipSMA, we integrate it into TensorFlow. Our experiments show that SkipSMA outperforms the state-of-the-art solutions for SDDP training, e.g., 6.88x speedup over SSGD.

Deep Learning based recommendation is common in various recommendation services and widely used in the industry. To predict user preferences accurately, state-of-the-art recommendation models contain an increasing number of features and various methods of feature interaction, which both lengthen inference time. We observe that the embedding lookup and feature interaction of different features in a recommendation model is independent of each other. However, current deep learning frameworks (e.g., TensorFlow, PyTorch) are oblivious to this independence, and schedule the operators to execute sequentially in a single computational stream. In this work, we exploit multiple CUDA streams to parallelize the execution of embedding lookup and feature interaction. To further overlap the processing of different sparse features and minimize synchronization overhead, we propose a topology-aware operator assignment algorithm to schedule operators to computational streams. We implement a prototype, namely StreamRec, based on TensorFlow XLA. Our experiments show that StreamRec is able to reduce latency by up to 27.8% and increase throughput by up to 52% in comparison to the original TensorFlow XLA.

在大数据治理应用中,数据分析是必不可少的一环,且具有耗时长、计算资源需求大的特点,因此,优化其执行效率至关重要.早期由于数据规模不大,数据分析师可以利用传统的矩阵计算工具执行分析算法,然而随着数据量的爆炸式增长,诸如MATLAB等传统工具已无法满足应用需求的执行效率,进而涌现出了一批面向大数据分析的分布式矩阵计算系统.从技术、系统等角度综述了分布式矩阵计算系统的研究进展.首先,从发展成熟的数据管理领域的视角出发,剖析分布式矩阵计算系统在编程接口、编译优化、执行引擎、数据存储这4个层面面临的挑战;其次,分别就这4个层面展开,探讨、总结相关技术;最后,总体分析了典型的分布式矩阵计算系统,并展望了未来研究的发展方向.

Nowadays, the size of DNN models has grown rapidly. To train a large model, pipeline parallelism-based frameworks partition the model across GPUs and slice each batch of data into multiple micro-batches. However, pipeline parallelism suffers from a bubble issue and low peak utilization of GPUs. Recent work tries to address the two issues, but fails to exploit the benefit of vanilla pipeline parallelism, i.e., overlapping communication with computation. In this work, we employ an elastic averaging-based framework which explores elastic averaging to add multiple parallel pipelines. To help the framework exploit the advantage of pipeline parallelism while reducing the memory footprints, we propose a schedule, advance forward propagation. Moreover, since the numbers of parallel pipelines and micro-batches are essential to the framework performance, we propose a profiling-based tuning method to automatically determine the settings. We integrate those techniques into a prototype system, namely AvgPipe, based on PyTorch. Our experiments show that Avg-Pipe achieves a 1.7x speedups over state-of-the-art solutions of pipeline parallelism on average.

Iterative computation in distributed graph processing systems typically incurs a long runtime. Hence, it is crucial for graph processing to tolerate and quick recover from intermittent failures. Existing solutions can be categorized into checkpoint-based and checkpoint-free solution. The former writes checkpoints periodically during execution, which leads to significant overhead. Differently, the latter requires no checkpoint. Once failure happens, it reloads input data and resets the value of lost vertices directly. However, reloading input data involves repartitioning, which incurs additional overhead. Moreover, we observe that checkpoint-free solution cannot effectively handle failures for graph algorithms with topological mutations. To address these issues, we propose ACF2 with a partition-aware backup strategy and an incremental protocol. In particular, the partition-aware backup strategy backs up the sub-graphs of all nodes after initial partitioning. Once failure happens, the partition-aware backup strategy recovers the lost sub-graphs from the backups, and then resumes computation like checkpoint-free solution. To effectively handle failures involving topological mutations, the incremental protocol logs topological mutations during normal execution which would be exploited for recovery. We implement ACF2 based on Apache Giraph and our experiments show that ACF2 significantly outperforms existing solutions.

GPU(graphics processing unit)的高并行和高吞吐特性可以提高数据库OLAP(on-line analytical processing)查询的性能.然而目前openGauss无法利用GPU等异构计算硬件的优势.因此旨在探索如何使用 GPU加速该系统的 OLAP处理过程,以实现更高的性能.针对 openGauss与 SQL为系统PostgreSQL名称的一部分,因此不能修改执行粒度的差异,提出了基于分块读取和按键分发的CPU-GPU协同并行方案,该方案可缩短GPU Scan算子的I/O(input/output)时间以缩短GPU的空闲等待时间,又可多实例运行GPU Join以支持多GPU环境.针对openGauss与PostgreSQL体系结构的差异,提出了兼容向量化引擎的异构算子加速技术,实现了可嵌入向量化执行引擎的自定义算子框架,基于此实现了可处理openGauss列式数据的向量化GPU Scan算子.实现了原型系统,验证了所提出方案的效果.

To leverage the massively parallel capability of GPU for query execution, GPU databases have been studied for over a decade. Recently, researchers proposed to execute queries with both CPU and GPU in a pipelined approach. In the pipelined query execution, the cross-processor tuple transfer plays a crucial role for the overall query execution performance. The state-of-the-art solution achieves cross-processor tuple transfer using a queue-like data structure. However, it is coarse-grained due to the use of a single spin lock to achieve thread-safety. This design causes performance issues as it prevents the threads from accessing the queue simultaneously. In this paper, we propose a fine-grained tuple transfer mechanism. It employs decoupled enqueue/dequeue to enable two threads on different processors to access the queue at the same time. Moreover, this mechanism explores subqueue-based locking to enable the threads on the same processor to access the queue at the same time. In particular, we implement a prototype system, namely QC, which adopts fine-grained tuple transfer. Our experiments show that QC achieves an order of magnitude better performance than existing GPU databases such as HeavyDB.

Pregel-like systems are developed to execute iterative applications on massive graph data, which often leads to a long runtime. These systems are usually deployed on a cluster of commodity servers, where failures are common. Hence, fault-tolerance is crucial for them. A typical fault-tolerance technique is checkpointing, which can be achieved in a blocking or an unblocking manner. Blocking checkpointing incurs notable overhead as it pauses iterative computation. Unblocking checkpointing decrease the overhead by parallelizing checkpointing and iterative computation. However, it introduces resource contention due to parallel checkpointing tasks, which may prolong overall execution time. The queuing strategy and the staleness/tardiness-aware skipping policy can effectively improve unblocking checkpointing by alleviating the resource contention and selecting an optimal checkpoint from the queued checkpoints, respectively. In this demonstration, we showcase their internal mechanisms based on Apache Giraph.

With the increasing of matrix size in large-scale data analysis, a series of Spark-based distributed matrix computation systems have emerged. Typically, these systems split a matrix into matrix blocks and save these matrix blocks into a RDD. To implement matrix operations, these systems manipulate the matrices by applying coarse-grained RDD operations. That is, these systems load the entire RDD to get a part of matrix blocks. Hence, it may cause the redundant IO when running SGD-based algorithms, since SGD only samples a min-batch data. Moreover, these systems typically employ a hash scheme to partition matrix blocks, which is oblivious to the sampling semantics. In this work, we propose a sampling-aware data loading which uses fine-grained RDD operation to reduce the partitions without sampled data, so as to decrease the redundant IO. Moreover, we exploit a semantic-based partition scheme, which gathers sampled blocks into the same partitions, to further reduce the number of accessed partitions. We modify SystemDS to implement Emacs, efficient matrix computation for SGD-based algorithms on Apache Spark. Our experimental results show that Emacs outperforms existing Spark-based matrix computation systems by 37%.

摄像设备在生活中的普及,使得视频数据快速增长,这些数据中蕴含丰富的信息.早期,研究人员基于传统的计算机视觉技术开发视频分析系统,用于提取并分析视频数据.近年来,深度学习技术在人脸识别等领域取得了突破性进展,基于深度学习的新型视频分析系统不断涌现.从应用、技术、系统等角度,综述了新型视频分析系统的研究进展.首先,回顾了视频分析系统的发展历史,指出了新型视频分析系统与传统视频分析系统的区别;其次,分析了新型视频分析系统在计算和存储两方面所面临的挑战,从视频数据的组织分布和视频分析的应用需求两方面探讨了新型视频分析系统的影响因素;再次,将新型视频分析系统划分为针对计算优化的系统和针对存储优化的系统两大类,选取其中典型的代表并介绍其核心设计理念;最后,从多个维度对比和分析了新型视频分析系统,指出了这些系统当前存在的问题,并据此展望了新型视频分析系统未来的研究和发展方向.

Distributed matrix computation solutions support query interfaces of linear algebra expressions, which often contain redundancy, i.e., common and loop-constant subexpressions. However, existing solutions fail to find all redundant subexpressions. Moreover, eliminating the found redundancy leads to new execution order of operators, which may have side effect. To exploit the benefits of redundancy elimination, we propose a new system called ReMac, which performs automatic and adaptive elimination. In particular, automatic elimination adopts a block-wise search that exploits the properties of matrix computation for speed-up. Adaptive elimination employs a cost model and a dynamic programming-based method to generate efficient plans with redundancy elimination. In this demonstration, attendees will have an opportunity to experience the effect that automatic and adaptive elimination have on distributed matrix computation.