Interactive notebook programming is universal in modern ML and AI workflows, with interactive deep learning training (IDLT) emerging as a dominant use case. To ensure responsiveness, platforms like Jupyter and Colab reserve GPUs for long-running notebook sessions, despite their intermittent and sporadic GPU usage, leading to extremely low GPU utilization and prohibitively high costs. In this paper, we introduce NotebookOS, a GPU-efficient notebook platform tailored for the unique requirements of IDLT. NotebookOS employs replicated notebook kernels with Raft-synchronized replicas distributed across GPU servers. To optimize GPU utilization, NotebookOS oversubscribes server resources, leveraging high inter-arrival times in IDLT workloads, and allocates GPUs only during active cell execution. It also supports replica migration and automatic cluster scaling under high load. Altogether, this design enables interactive training with minimal delay. In evaluation on production workloads, NotebookOS saved over 1,187 GPU hours in 17.5 hours of real-world IDLT, while significantly improving interactivity.

Interactive notebook programming is universal in modern ML (machine learning) and AI (artificial intelligence) workflows. Notebook software like Jupyter and Google Colab provides a user-friendly, interactive, web-based programming interface and is widely used across science and engineering domains. A dominant application of production notebook workloads is interactive deep learning training (IDLT). To guarantee high interactivity, modern notebook platforms typically reserve GPU resources within actively running notebook sessions. These notebook sessions are long-running but exhibit intermittent and sporadic GPU usage. Consequently, during most of their lifetimes, notebook sessions do not use the reserved GPUs, resulting in extremely low GPU utilization and prohibitively high cost. In this paper, we introduce NotebookOS, a GPU-efficient notebook platform designed to meet the unique requirements of IDLT. NotebookOS uses a replicated notebook kernel design, where each kernel consists of three replicas distributed across separate GPU servers and synchronized via Raft. To optimize GPU utilization, NotebookOS oversubscribes server resources via kernel replication to leverage the relatively high task inter-arrival times in IDLT workloads. By dynamically allocating GPUs to kernel replicas only while they are actively executing notebook cells, NotebookOS maximizes the likelihood of immediate and interactive training upon notebook notebook-cell task submission. NotebookOS also migrates kernel replicas and automatically scales the GPU cluster under overload conditions. We evaluate NotebookOS extensively using production notebook workloads. Evaluation results show that NotebookOS saves 1,187+ GPU hours over a 17.5-hour real-world IDLT workload while greatly enhancing interactivity.

Time-series forecasting models are invariably used in a variety of domains for crucial decision-making. Traditionally these models are constructed by experts with considerable manual effort. Unfortunately, this approach has poor scalability while generating accurate forecasts for new datasets belonging to diverse applications. Without access to skilled domain-knowledge, one approach is to train all the models on the new time-series data and then select the best one. However, this approach is nonviable in practice. In this work, we develop techniques for fast automatic selection of the best forecasting model for a new unseen time-series dataset, without having to first train (or evaluate) all the models on the new time-series data to select the best one. In particular, we develop a forecasting meta-learning approach called AutoForecast that allows for the quick inference of the best time-series forecasting model for an unseen dataset. Our approach learns both forecasting models performances over time horizon of the same dataset and task similarity across different datasets. The experiments demonstrate the effectiveness of the approach over state-of-the-art (SOTA) single and ensemble methods and several SOTA meta-learners (adapted to our problem) in terms of selecting better forecasting models (i.e., 2X gain) for unseen tasks for univariate and multivariate testbeds. AutoForecast has also significant reduction in inference time compared to the naïve approach (doing inference using all possible models and then selecting the best one), with median of 42X across the two testbeds. We release our meta-learning database corpus (348 datasets), performances of the 322 forecasting models on the database corpus, meta-features, and source codes for the community to access them for forecasting model selection and to build on them with new datasets and models which can help advance automating time-series forecasting problem. In our released database corpus, we unveil new traces of Adobe computing cluster usage for production workloads.

Metric embeddings are fundamental in machine learning, enabling similarity search, dimensionality reduction, and representation learning. They underpin modern architectures like transformers and large language models, facilitating scalable training and improved generalization. Theoretically, the classic problem in embedding design is mapping arbitrary metrics into l(p) spaces while approximately preserving pairwise distances. We study this problem in a fully dynamic setting, where the underlying metric is a graph metric subject to edge insertions and deletions. Our goal is to maintain an efficient embedding after each update. We present the first fully dynamic algorithm for this problem, achieving O(log(n))O-2q(log(nW))(q-1) expected distortion with O(m(1/q+o(1))) update time and O(q log(n) log(nW)) query time, where q >= 2 is an integer parameter.

Leveraging graph structures for time-series forecasting has garnered significant attention due to their effective relationship modeling between nodes and their associated time-series. However, in scenarios entities communicate in a broadcasting manner, graph models fall short of pairwise modeling. Hypergraph models address this by capturing beyond-pairwise interactions among node time-series. Nevertheless, most hypergraph models overlook the dynamics between nodes and their incident hyperedges, assuming constant node-hyperedge connections. In this paper, we introduce a novel model, Probabilistic Hypergraph Recurrent Neural Networks (PHRNN), which leverages node-hyperedge dynamics for accurate time-series forecasting. PHRNN associates each time-series with a node and models node interactions on a hypergraph, capturing beyond-pairwise interactions. Moreover, PHRNN learns a probabilistic hypergraph in which node-hyperedge relations are modeled as probabilistic distributions instead of fixed values, capturing dynamic node-hyperedge relations. PHRNN further integrates a prior knowledge KNN hypergraph as regularization when learning the probabilistic hypergraph structure. To the best of our knowledge, PHRNN is the first time-series forecasting model that incorporates hypergraph modeling and probabilistic relationship modeling. Forecasting results from extensive experiments show that PHRNN outperforms state-of-the-art graph and hypergraph baselines on real-world datasets.

We investigate the role of inaccurate priors for the classical Pandora's box problem. In the classical Pandora's box problem we are given a set of boxes each with a known cost and an unknown value sampled from a known distribution. We investigate how inaccuracies in the beliefs can affect existing algorithms. Specifically, we assume that the knowledge of the underlying distribution has a small error in the Kolmogorov distance, and study how this affects the utility obtained by the optimal algorithm.

Temporal Knowledge Graph (TKG) completion models traditionally assume access to the entire graph during training. This overlooks challenges stemming from the evolving nature of TKGs, such as: (i) the model's requirement to generalize and assimilate new knowledge, and (ii) the task of managing new or unseen entities that often have sparse connections. In this paper, we present an incremental training framework specifically designed for TKGs, aiming to address entities that are either not observed during training or have sparse connections. Our approach combines a model-agnostic enhancement layer with a weighted sampling strategy, that can be augmented to and improve any existing TKG completion method. The enhancement layer leverages a broader, global definition of entity similarity, which moves beyond mere local neighborhood proximity of GNN-based methods. The weighted sampling strategy employed in training accentuates edges linked to infrequently occurring entities. We evaluate our method on two benchmark datasets, and demonstrate that our framework outperforms existing methods in total link prediction, inductive link prediction, and in addressing long-tail entities. Notably, our method achieves a 10% improvement and a 15% boost in MRR for these datasets. The results underscore the potential of our approach in mitigating catastrophic forgetting and enhancing the robustness of TKG completion methods, especially in an incremental training context

With the advent of cloud platforms that offer GPU-as-a-Service (GPUaaS), such as Amazon EC2 and Microsoft Azure, researchers increasingly rely on virtual GPU instances for training deep learning (DL) workloads. These GPU instances vary in configuration attributes, including but not limited to the number of GPUs, the number of vCPUs, and per-hour usage cost. Identifying the appropriate GPU instance for training a DL workload becomes extremely difficult due to the huge GPU instance selection space offered by the cloud platforms and the corresponding variation in training performance or computational needs of different DL workloads. In this paper, we propose a GPU instance recommendation system called GPURank, which provides a recommended list of GPU instances to choose from for DL workloads. GPURank predicts and leverages two metrics: epoch training cost and average GPU utilization to make this choice. We curated a new benchmark dataset by profiling diverse DL workloads to train the regression models in GPURank’s prediction framework. We demonstrate that GPURank beats baselines on two pertinent problem settings: (1) unseen workloads and (2) unseen GPU instances, with a 25.89% and 20.10% higher average ranking performance on these respectively.

Text-to-image diffusion models excel in generating photo-realistic images but are hampered by slow processing times. Training-free retrieval-based acceleration methods, which leverage pre-generated "trajectories," have been introduced to address this. Yet, these methods often lack diversity and fidelity as they depend heavily on similarities to stored prompts. To address this, we present ReCON (Retrieving CONcepts), an innovative retrieval-based diffusion acceleration method that extracts visual "concepts" from prompts, forming a knowledge base that facilitates the creation of adaptable trajectories. Consequently, ReCON surpasses existing retrieval-based methods, producing high-fidelity images and reducing required Neural Function Evaluations (NFEs) by up to 40%. Extensive testing on MS-COCO, Pick-a-pick, and DiffusionDB datasets confirms that RECON consistently outperforms established methods across multiple metrics such as Pick Score, CLIP Score, and Aesthetics Score. A user study further indicates that 76% of images generated by ReCON are rated as the highest fidelity, outperforming two competing methods, a purely text-based retrieval and a noise similarity-based retrieval.

Most Temporal Knowledge Graphs (TKGs) exhibit a long-tail entity distribution, where the majority of entities have sparse connections. Existing TKG completion methods struggle with managing new or unseen entities that often lack sufficient connections. In this paper, we introduce a model-agnostic enhancement layer that can be integrated with any existing TKG completion method to improve its performance. This enhancement layer employs a broader, global definition of entity similarity, transcending the limitations of local neighborhood proximity found in Graph Neural Network (GNN) based methods. Additionally, we conduct our evaluations in a novel, realistic setup that treats the TKG as a stream of evolving data. Evaluations on two benchmark datasets demonstrate that our framework surpasses existing methods in overall link prediction, inductive link prediction, and in addressing long-tail entities. Notably, our approach achieves a 10% improvement in MRR on one dataset and a 15% increase on another.

Temporal Knowledge Graph (TKG) completion models traditionally assume access to the entire graph during training. This overlooks challenges stemming from the evolving nature of TKGs, such as: (i) the model's requirement to generalize and assimilate new knowledge, and (ii) the task of managing new or unseen entities that often have sparse connections. In this paper, we present an incremental training framework specifically designed for TKGs, aiming to address entities that are either not observed during training or have sparse connections. Our approach combines a model-agnostic enhancement layer with a weighted sampling strategy, that can be augmented to and improve any existing TKG completion method. The enhancement layer leverages a broader, global definition of entity similarity, moving beyond mere local neighborhood proximity of GNN-based methods. The weighted sampling strategy employed in training accentuates edges linked to infrequently occurring entities. Our evaluations, conducted on two benchmark datasets, demonstrate that our framework outperforms existing methods in overall link prediction, inductive link prediction, and in addressing long-tail entities. Notably, our method achieves a 10\% improvement in MRR for one dataset and a 15\% boost for another. The results underscore the potential of our approach in mitigating catastrophic forgetting and enhancing the robustness of TKG completion methods, especially in an incremental training context.

In real-world scenarios, labeled samples for dialogue summarization are usually limited (i.e., few-shot) due to high annotation costs for high-quality dialogue summaries. To efficiently learn from few-shot samples, previous works have utilized massive annotated data from other downstream tasks and then performed prompt transfer in prompt tuning so as to enable cross-task knowledge transfer. However, existing general-purpose prompt transfer techniques lack consideration for dialogue-specific information. In this paper, we focus on improving the prompt transfer from dialogue state tracking to dialogue summarization and propose Skeleton-Assisted Prompt Transfer (SAPT), which leverages skeleton generation as extra supervision that functions as a medium connecting the distinct source and target task and resulting in the model's better consumption of dialogue state information. To automatically extract dialogue skeletons as supervised training data for skeleton generation, we design a novel approach with perturbation-based probes requiring neither annotation effort nor domain knowledge. Training the model on such skeletons can also help preserve model capability during prompt transfer. Our method significantly outperforms existing baselines. In-depth analyses demonstrate the effectiveness of our method in facilitating cross-task knowledge transfer in few-shot dialogue summarization.

Recent successes of foundation models in large language models have inspired researchers to apply similar technologies to time-series forecasting. Unlike conventional time-series forecasting models, which are trained on the training subset of the target dataset, foundation models are trained on a large collection of source datasets that do not necessarily include the target dataset, with the assumption that foundation models can capture the complex patterns between the input time-series values and the desired predictions. Although many foundation models have claimed superior prediction performance compared to conventional models, one question remains unanswered: Do foundation models for time-series forecasting, which train on many datasets other than the target dataset, perform better than conventional models that train on only (the training subset of) the target dataset? To answer this question, this paper adapts a diffusion-based foundation model and conducts extensive experiments using both small datasets and a large collection of over 100 datasets. Our results show that training on large-scale datasets does not necessarily guarantee a better performance than a conventional model that trains only on the dataset from the same domain. Hence, this paper provides insights for future foundation model research in large-scale time-series forecasting, emphasizing that the usage of target datasets should be considered in addition to training on large-scale source datasets.

Graph Recurrent Neural Networks (GRNN) excel in time-series prediction by modeling complicated non-linear relationships among time-series. However, most GRNN models target small datasets that only have tens of time-series or hundreds of time-series. Therefore, they fail to handle large-scale datasets that have tens of thousands of time-series, which exist in many real-world scenarios. To address this scalability issue, we propose Evolving Super Graph Neural Networks (ESGNN), which target large-scale datasets and significantly boost model training. Our ESGNN models multivariate time-series based on super graphs, where each super node is associated with a set of time-series that are highly correlated with each other. To further precisely model dynamic relationships between time-series, ESGNN quickly updates super graphs on the fly by using the LSH algorithm to construct the super edges. The embeddings of super nodes are learned through end-to-end learning and are then used with each target time-series for forecasting. Experimental result shows that ESGNN outperforms previous state-of-the-art methods with a significant runtime speedup ( 3 × – 40 × faster) and space-saving ( 5 × – 4600 × less), while only sacrificing little or negligible prediction accuracy. An ablation study is also conducted to investigate the effectiveness of the number of super nodes and the graph update interval.

Many existing deep graph models have shown that forecasting time-series values benefits from modeling the mutual relations between time-series. For example, graph neural networks can exploit the correlations between two CPU utilization time-series, facilitating more accurate predictions. However, the implied pairwise interactions between entities in the graph structure do not always reflect the actual interactions. In a cloud system, for instance, computing tasks are assigned to groups of machines, and CPU utilization time-series within the same group simultaneously interact with one another. Hence, such interactions are beyond-pairwise. In this paper, we propose a novel model called Hypergraph Recurrent Neural Networks (HGRNN) for time-series forecasting. Our model employs a hypergraph to model beyond-pairwise relations, which naturally reflect the actual interactions among entities. We also introduce a novel semi-principled hypergraph construction method to address the challenge of missing hypergraph information. Our model adopts the encoder-decoder framework where historical time-series are digested into an encoded state which is decoded to yield prediction. We further integrate a temporal component to enhance learning from temporal locality. Extensive experiments on large-scale datasets show that our model achieves better forecasting performance compared to state-of-the-art baselines.

Effective time-series forecasting methods are of significant importance to solve a broad spectrum of research problems. Deep probabilistic forecasting techniques have recently been proposed for modeling large collections of time-series. However, these techniques explicitly assume either complete independence (local model) or complete dependence (global model) between time-series in the collection. This corresponds to the two extreme cases where every time-series is disconnected from every other time-series in the collection or likewise, that every time-series is related to every other time-series resulting in a completely connected graph. In this work, we propose a deep hybrid probabilistic graph-based forecasting framework called Graph Deep Factors (GraphDF) that goes beyond these two extremes by allowing nodes and their time-series to be connected to others in an arbitrary fashion. GraphDF is a hybrid forecasting framework that consists of a relational global and relational local model. In particular, a relational global model learns complex non-linear time-series patterns globally using the structure of the graph to improve both forecasting accuracy and computational efficiency. Similarly, instead of modeling every time-series independently, a relational local model not only considers its individual time-series but also the time-series of nodes that are connected in the graph. The experiments demonstrate the effectiveness of the proposed deep hybrid graph-based forecasting model compared to the state-of-the-art methods in terms of its forecasting accuracy, runtime, and scalability. Our case study reveals that GraphDF can successfully generate cloud usage forecasts and opportunistically schedule workloads to increase cloud cluster utilization by 47.5% on average. Furthermore, we target addressing the common nature of many time-series forecasting applications where time-series are provided in a streaming version; however, most methods fail to leverage the newly incoming time-series values and result in worse performance over time. In this article, we propose an online incremental learning framework for probabilistic forecasting. The framework is theoretically proven to have lower time and space complexity. The framework can be universally applied to many other machine learning-based methods.

Online recommender systems have proven to have ubiquitous applications in various domains. To provide accurate recommendations in real time it is imperative to constantly train and deploy models with the latest data samples. This retraining involves adjusting the model weights by incorporating newly-arrived streaming data into the model to bridge the accuracy gap. To provision resources for the retraining, typically the compute is hosted on VMs, however, due to the dynamic nature of the data arrival patterns, stateless functions would be an ideal alternative over VMs, as they can instantaneously scale on demand. However, it is non-trivial to statically configure the stateless functions because the model retraining exhibits varying resource needs during different phases of retraining. Therefore, it is crucial to dynamically configure the functions to meet the resource requirements, while bridging the accuracy gap. In this paper, we propose Sandpiper, an adaptive framework that leverages stateless functions to deliver accurate predictions at low cost for online recommender systems. The three main ideas in Sandpiper are (i) we design a data-drift monitor that automatically triggers model retraining at required time intervals to bridge the accuracy gap due to incoming data drifts; (ii) we develop an online configuration model that selects the appropriate function configurations while maintaining the model serving accuracy within the latency and cost budget; and (iii) we propose a dynamic synchronization policy for stateless functions to speed up the distributed model retraining leading to cloud cost minimization. A prototype implementation on AWS shows that Sandpiper maintains the average accuracy above 90%, while 3.8× less expensive than the traditional VM-based schemes.

Sequencing technologies are prone to errors, making error correction (EC) necessary for downstream applications. EC tools need to be manually configured for optimal performance. We find that the optimal parameters (e.g., k-mer size) are both tool- and dataset-dependent. Moreover, evaluating the performance (i.e., Alignment-rate or Gain) of a given tool usually relies on a reference genome, but quality reference genomes are not always available. We introduce Lerna for the automated configuration of k-mer-based EC tools. Lerna first creates a language model (LM) of the uncorrected genomic reads; then, calculates the perplexity metric to evaluate the corrected reads for different parameter choices. Next, it finds the one that produces the highest alignment rate without using a reference genome. The fundamental intuition of our approach is that the perplexity metric is inversely correlated with the quality of the assembly after error correction. Results: First, we show that the best k-mer value can vary for different datasets, even for the same EC tool. Second, we show the gains of our LM using its component attention-based transformers. We show the model's estimation of the perplexity metric before and after error correction. The lower the perplexity after correction, the better the k-mer size. We also show that the alignment rate and assembly quality computed for the corrected reads are strongly negatively correlated with the perplexity, enabling the automated selection of k-mer values for better error correction, and hence, improved assembly quality. Additionally, we show that our attention-based models have significant runtime improvement for the entire pipeline -- 18X faster than previous works, due to parallelizing the attention mechanism and the use of JIT compilation for GPU inferencing.

In this work, we develop techniques for fast automatic selection of the best forecasting model for a new unseen time-series dataset, without having to first train (or evaluate) all the models on the new time-series data to select the best one. In particular, we develop a forecasting meta-learning approach called AutoForecast that allows for the quick inference of the best time-series forecasting model for an unseen dataset. Our approach learns both forecasting models performances over time horizon of same dataset and task similarity across different datasets. The experiments demonstrate the effectiveness of the approach over state-of-the-art (SOTA) single and ensemble methods and several SOTA meta-learners (adapted to our problem) in terms of selecting better forecasting models (i.e., 2X gain) for unseen tasks for univariate and multivariate testbeds.