Acquiring tabular datasets for machine learning research can be surprisingly challenging. For example, executable SQL code paired with tabular data is still limited by manual annotations by human experts, which limits its scale. Furthermore, tabular machine learning (ML) problems often impose domain-specific constraints, making customized datasets even harder to obtain. To tackle the tabular dataset labeling problem, we propose a scalable and customizable methodology that synthesizes tabular ML datasets using language model annotations. Our approach leverages the abundance of existing tabular data and the instruction-following capabilities of language models to generate datasets tailored to specific research needs. We apply this method to create a three-part dataset for three tabular ML studies, progressively exploring how language models can augment tabular data. Notably, our method has produced the largest executable SQL dataset to date. In these studies, we demonstrate the scientific utility of our augmented datasets and assess the correctness of LLM-synthesized labels, highlighting the models' ability to understand tabular data and support ML research.
Modern HPC workloads exchange vast data to drive scientific discoveries, relying on diverse storage infrastructures. Current monitoring tools only capture the I/O interfaces, which is insufficient to explain suboptimal I/O performance, as the bottleneck can be within one or more layers of the parallel I/O stack. To address this, we analyze HPC ranking lists to highlight storage diversity and build the Master Architectural Plan (MAP) that enables identification of current gaps in data paths in monitoring tools. Finally, we present DataCrumbs, a lightweight tool that fills these gaps by examining the user and kernel I/O stack to uncover causal relationships to achieve eXplainable I/O (XIO).
Optimizing the I/Os is essential to avoid performance bottlenecks in High-Performance Computing. Advisory hints, such as POSIX fadvise and Lustre ladvise, are designed to improve I/O performance by allowing applications to provide the file system with additional information on their access patterns. These mechanisms are supposed to enable more efficient cache utilization and reduce I/O performance bottlenecks. However, in practice, their effectiveness depends on how individual file systems interpret these hints and on how developers use them. In this paper, we offer a comprehensive investigation of the impact of advisory hints on Lustre and GPFS parallel file systems. Our contributions include: (1) a methodology to evaluate the performance impact of such hints on these widely-used parallel file systems, (2) the introduction of a benchmark that implements this methodology, and (3) we analyze the result of our benchmark, leading to practical guidelines for application developers to improve I/O performance through proper use of those hints. Our findings demonstrate that the effect of these hints varies significantly between evaluated state-of-the-art parallel file systems and highlight the need for tailored approaches in leveraging advisory hints in order to achieve optimal I/O performance.
Underground mines are highly accident-prone, and navigating miners to safety in post-disaster scenarios is hard. Delay-tolerant Networks (DTNs) based communication and navigation systems can be used by the miners (referred to humans and/or humanoids) to get to the nearest exit/shelter in the shortest possible time. This paper presents an Obstacle-based Goal-Conditioned Reinforcement Learning framework for underground mines (OGLe-Mine) that takes advantage of the Contrastive Learning (CL) technique for an obstacle Goal-Conditioned Reinforcement Learning (GCRL) framework. We add an obstacle representation term to the traditional GCRL framework, an augmented Markov Decision Process, to allow the model to learn about the kind and intensity of obstacles. We use Soft Actor-Critic (SAC) to implement the GCRL framework that will enable us to exploit replay buffer for sample efficient learning. The pre-disaster mine map is updated based on sensor information and miner annotation as the miner navigates in a post-disaster mine. Using maps, the trained CL framework generates representations, which are then used by the GCRL framework to predict the direction in which the miner should move. Our model was compared with CURL-SAC, Goal conditioned RL and Graph GC planner methods with different blockage percentages. OGLe-Mine outperformed these other models for single-agent and multiple-agent cases in success rate and escape times and has better path optimality values (find more efficient routes) with one, three, and five exits.
Understanding and mitigating biodiversity loss is an urgent and pressing global-scale problem, exacerbated by rapid changes in planetary climate and human development. Accessing and integrating diverse, multimodal data across an ever-proliferating landscape of repositories has become a critical need for coordinated biodiversity efforts. Even when applicable repositories exist and are known, findability of data inside them is a tremendous challenge, impeding complex research questions and collaborative science. This challenge is hindered not just by the volume and variety of the data but also by real-world hurdles in data fragmentation, governance, and metadata annotation. Current centralized data repositories, while useful, are insufficient, especially for low-resource sub-fields and locally focused initiatives. In this paper, we share case studies and challenges encountered with current approaches towards biodiversity data management. We share our vision, OmniMesh, that builds upon recent progress in embeddings and zero-shot models to provide a pragmatic, easy-to-adopt, standards-based solution to distributed search. Finally, we share a call-to-action, identifying open problems and areas of need where the data management community can assist with global biodiversity efforts.
To verify the reproducibility of an application, it is often necessary to execute it multiple times, each time with a different input, and evaluate whether the application outcome changes. Given the complexity of the software, any unexpected behavior in the outcome requires quick insight into application behavior. Efficient execution tracing is instrumental but traces must often be compared to attain that insight. Comparing execution traces is a significant challenge, especially for parallel MPI applications as tasks may exchange messages in a non-deterministic order. Current methods for comparing execution traces assume same input across application runs. In this paper, we present a method to compare execution traces of a parallel MPI application that have two sources of non-determinism: a changed input and variations in message exchange order due to running the application at a different time. We show that to compare traces from such application runs, we need selective replay-a method to replay message exchanges only when the new execution, with a changed input, aligns with the recorded execution in terms of its execution path. We propose two methods for deciding selective replay, each of which vary in the amount of execution state maintained. Our results demonstrate that, without selective replay, multiple sources of non-determinism lead to numerous false positives in comparing two runs. While traditional record-and-replay methods can pinpoint the first location of divergence, they fail to identify subsequent points. Through selective replay, we uncover and explain all divergences and convergences, achieving a reduction in false positives by more than 50%.
Awareness regarding niche user preferences, city events, and points of interest (POIs), along with their contextual relevance, is critical for efficiently executing spatial search. Standard tourist applications rely on spatial indexes (such as the R-tree or the R*-tree) to identify POIs based on their spatial relevance. However, these indexes typically struggle to simultaneously integrate POI metadata, user preferences, and contextual relevance into their spatial search, leading to a significant search overhead. In order to mitigate such overheads and streamline POI search, this work presents a novel index that leverages a bitmap structure to enrich an underlying spatial index with POI categories, their respective sub-categories, as well as contextual information. The proposed index structure helps integrate both spatial and context-relevance in the node-traversal process, and performs as efficiently as the underlying spatial index in the worst-case scenario. Next, this work introduces a novel tourist navigation application, designated as eSahyatri (translation: e-companion), which exploits our proposed indexing technique and LLMs to generate personalized, context-aware stories about POIs in real-time. A theoretical analysis and performance study highlight the overall effectiveness of the proposed indexing technique.
Graph reordering is a technique that enhances data access efficiency by adjusting the arrangement of vertices in a graph. However, finding the optimal ordering is NP-hard. To address this challenge, researchers have proposed various heuristic methods. One notable approach is Gorder, which significantly improves computational performance. Another is the degree-based DBG ordering, which effectively reduces reordering overhead. In this study, we consider amortized cost as a criterion to evaluate the practical applicability of graph reordering. Amortized cost indicates how many times a graph application must be executed to offset the computational cost of reordering. Surprisingly, previous studies have primarily focused on comparing acceleration performance with Gorder and RCM. They have not considered amortized cost in their comparison with BFS ordering. In this work, we evaluate our approach using 12 real-world graphs and 5 common graph applications. Our findings demonstrate that BFS ordering achieves a well-balanced trade-off between acceleration and reordering time. Furthermore, we employ random graph models to analyze how various graph parameters influence the performance of BFS ordering.
Conventional database indexes, although popular, are not memory efficient and usually do not take the data distribution of the underlying workloads into account, which may lead to performance issues. In recent years, substantial research has focused on enhancing the performance of index lookups and reducing the memory footprint of the indexes. One promising research area is the development of learned indexes, which can deliver notable performance gains over traditional indexing methods. Methods like ALEX and LIPP facilitate updates in learned index frameworks but at the cost of higher memory consumption and slower query performance. We introduce a lightweight updatable learned index, called FALCON (FPGA Accelerated Lightweight COmpression-based learNed index). FALCON is the first FPGA-based lightweight learned index that supports updates and insertion. This approach devises an effective compression technique facilitating faster key lookups. Our experiments involved first evaluating a CPU-based implementation of our approach against existing leading learned index approaches. With a CPU-based implementation of FALCON, we observed an average performance boost of 25x during the Build phase and more than 2.2x speedup during the Lookup phase on SOSD benchmark datasets, along with a 26x improvement in memory efficiency. We also evaluated FALCON's performance with frequent updates, in which it achieved around 3.5x better memory efficiency compared to existing updatable learned indexes, such as ALEX. For the datasets exhibiting uniform distribution, we achieved a similar update throughput as ALEX, while in the case of skewed distribution, ALEX performed better. Nonetheless, runtime and memory usage usually escalate as data size scales, prompting us to employ FPGA to accelerate the build phase of our approach. We then evaluated FALCON's FPGA-based implementation against the CPU-based counterpart, in which it achieved a build stage speedup of 17x, and a total speedup of 5.5x. Against the FPGA-based RadixSpline, we achieved a total of 7.6x better performance on FPGA.
Scientific workflows are becoming increasingly complex, including utilization of multiple binaries, multiple clusters, and wide ranges of input parameters. This complexity will likely increase with the emergence of Artificial Intelligence and Machine Learning (AI/ML) simulations as well as the convergence of High-Performance Computing (HPC) and Cloud technologies. Existing performance tools target traditional bulk-synchronous parallel single-binary applications. Such tools do not integrate easily with upcoming complex workflows, for reasons including steep user-level learning curves, source code readability issues, and lack of support for composability and native visualization of traces. We introduce PerfFlowAspect, a lightweight, open-source, and aspect-oriented performance analysis tool for complex scientific workflows. PerfFlowAspect is easy-to-use and can provide both coarse and fine-grained performance results ranging from traditional to complex multi-cluster, multi-binary workflows. We demonstrate its usability with two uses cases: (1) Merlin ICECap: an ML-workflow based on a scalable framework designed for multi-cluster simulations, and (2) Autonomous Multi-Scale (AMS): an application to integrate ML surrogate models in HPC physics simulations. Our findings demonstrate that PerfFlowAspect is effective in supporting complex, multi-cluster and multi-binary workflows in a user-friendly manner with minimal overhead (0.9% on average).
We propose a simplified user interaction framework for the indistinguishability query. The indistinguishability query determines all of the user's near optimal tuples without explicit knowledge of their utility function. This approach uses a simple interactive framework where the user picks their favorite tuple during rounds of questions. We propose an alternative interactive framework that is more truthful and focuses on comparing two attributes at a time. This allows us to use more powerful 2D techniques for higher-dimensional data while also simplifying the decision-making on the part of the user. We provide a strongly truthful algorithm that displays the user with only real tuples from the database. In addition, we give an algorithm that displays synthetic tuples that are more realistic than previous work. We introduce a new definition - tolerably truthful - that guarantees that all tuples have attribute values within the ranges found within the database. Our tolerably truthful algorithm has a provable approximation guarantee for the indistinguishable output set. We also verify the efficacy of our algorithms with experiments on both real and synthetic data sets and through a user study.
The discovery of functional dye materials with superior optical properties is crucial for advancing technologies in biomedical imaging, organic photovoltaics, and quantum information systems. Recent advancements highlight the need to accelerate this discovery process by integrating computational strategies with experimental methods. In this regard, we have employed a computational approach to explore the latent space of dye materials, utilizing swarm optimization techniques to efficiently navigate complex chemical spaces and identify optimal values of molecular properties using machine learning methods based on target properties, such as high extinction coefficients (epsilon). The latent space based evaluation outperformed all available features of a domain. This approach enhances inverse material design by systematically correlating molecular parameters with desired optical characteristics by implementing VAEs. In this process, by defining target properties as inputs, the model effectively determines the key molecular features necessary for engineering high-performance dye compounds.
Inspired by the emergence of scientific data in fields like as astronomy, climate research, and genomics, which presents significant challenges for conventional database systems. This vision paper investigates the adaptation of vector databases in order to describe, handle, and query large-scale scientific data. Moreover, we propose a road-map for embedding-centric data infrastructures, along with an analysis of current advancements and identification of important future research directions, including explainability and scalability. This research provides a foundational basis for next-generation interdisciplinary scientific discovery.
HPC, Big Data Analytics, and Machine Learning have become increasingly intertwined as popular models such as LLMs and Diffusion Models have been driving discovery in scientific fields. However, each of these domains has its own storage infrastructure with unique I/O interfaces and storage systems, requiring feature sets that are often incompatible. Users with experience in one domain lack the expertise to change their applications to match the data stacks of the other domains, necessitating expensive conversions. There is a need for a transparent solution for the unification of disparate data stacks for the triple convergence of HPC, Big Data, and ML that can provide the required functionality while achieving higher performance. To better support converged HPC, Big Data, and ML workflows, this paper proposes DTIO, a scalable I/O runtime that unifies the disparate I/O stack for modern scientific ML workflows. DTIO utilizes a unique DataTask abstraction to express the movement of data, its ordering, and its dependencies on other data as a task. DTIO achieves a unification of scientific and ML workflows by utilizing intelligent mapping of interfaces, and automatically determines the best method to relate their unique semantics. DTIO's online translation with DataTask caching can improve performance by 49.6% compared to offline translation methods. DTIO also offers numerous optimizations, such as asynchronous I/O and aggregation.
Scientific workflows are pivotal for managing complex computational tasks, including data analysis, processing, simulation, and visualization. However, their design and administration typically demand substantial programming expertise, limiting access for domain scientists. Many such workflow systems also lack real-time execution tracking, and streamlined data integration capabilities, hindering efficiency and repeatability in scientific experimentation. In response, we introduce VisFlow 2.0, a next-generation platform derived from the original VisFlow. We compare VisFlow 2.0 to traditional alternatives through a well-studied computational pipeline, highlighting its usability, flexibility, and effectiveness, especially for non-expert users.
Many scientific data applications show a cyclic oscillating workload that oscillates between read-heavy and write-heavy constantly. This oscillation introduces a challenge for the existing indexing techniques. Indexes, e.g., the B+-tree and the LSM-tree, cannot perform equally well all the time. Conventional adaptive indexing does not solve this issue as it focuses on adapting in one direction. This paper studies how to support oscillating workloads that arise in scientific data applications with adaptive indexes that adapt the underlying index structures in both directions. With the observation that real-world datasets are skewed, the focus is to optimize the index within the hotspot regions. The Adaptive Hotspot-Aware Tree (or AHA-tree, for short) is introduced, where its adaptation is bi-directional. Experimental evaluation shows that AHA-tree can behave competitively as compared to an LSM-tree for write-heavy transactional workloads. Upon switching to a read-heavy analytical workload, AHA-tree can gradually adapt and behave competitively, and can match the B+-tree's read performance.
More and more scientific applications store datasets in scientific data formats such as HDF5 and netCDF. However, existing search methods for scientific data formats generally require researchers to be familiar with the formats and metadata structure, resulting in a steep learning curve. Therefore, researchers need a natural language query method to query scientific data. In this paper, we propose ICEAGE, a novel Intelligent Contextual Exploration and Answer Generation Engine that bridges the gap between natural language querying and scientific data and metadata retrieval. Based on a retrieval-augmented generation framework, ICEAGE generates reliable and human-readable responses without requiring extensive domain-specific fine-tuning by applying unique indexing method for scientific datasets. Our experimental results demonstrate that ICEAGE significantly outperforms existing methods in terms of accuracy, throughput in both CPU-GPU and CPU-only environments.
Scientists are automating instrument operation using distributed scientific workflows. To improve workflow response time, we present a new scheduling method called FastFlow. Although there has been much prior work on scheduling, we introduce a new insight: prioritizing critical flow paths and their interactions-in/out flow path-senables a linear-time scheduling method that achieves high quality results. Since many workflows are executed repeatedly, our method is based on a monitor-analyze-optimize strategy. After monitoring a workflow's execution-time data flow, our method identifies response-critical paths and their in/out data flows using a linear-time partitioning algorithm. For each partition, a greedy linear-time scheduler selects between the better of flow parallelism and flow locality. The resulting schedules are high quality because the greedy decisions avoid delaying or shifting the critical flow. We evaluate a range of representative workflows and compare against state-of-the-art methods. Our experiments demonstrate mean speedups of 1.15x, 3.5x, 1.04x, and 1.07x compared to the next best, which are not linear time. Compared to popular linear-time methods, speedups are up to 1.28x, 87x, 1.4x, and 5x.
Automated and reproducible performance modeling plays a vital role in HPC research as it provides objective insights into network, I/O and computing behavior. Yet, performance data is often used only once and then discarded, limiting its long-term value. This work introduces a streamlined ML-based workflow for HPC systems that automates and simplifies performance modeling while maximizing data reuse. Implemented as an interactive console application, the workflow handles data generation, preprocessing, model training and validation, all in one place. Its modular design supports different machine learning algorithms for both regression and classification, providing flexibility for different modeling tasks. Our case study demonstrates that the proposed solution effectively automates the creation of performance models and supports various use cases such as performance prediction and parameter reduction. Overall, it reduces the time, effort and complexity typically associated with HPC performance modeling.
The k-MIPS (k Maximum Inner Product Search) problem has been employed in many fields. Recently, its reverse version, the reverse k-MIPS problem, has been proposed. Given an item vector (i.e., query), it retrieves all user vectors such that their k-MIPS results contain the item vector. Consider the cardinality of a reverse k-MIPS result. A large cardinality means that the item is potentially popular, because it is included in the k-MIPS results of many users. This mining is important in recommender systems, market analysis, and new item development. Motivated by this, we formulate a new problem. In this problem, the score of each item is defined as the cardinality of its reverse k-MIPS result, and the N items with the highest score are retrieved. A straightforward approach is to compute the scores of all items, but this is clearly prohibitive for large numbers of users and items. We remove this inefficiency issue and propose a fast algorithm for this problem. Because the main bottleneck of the problem is to compute the score of each item, we devise a new upper-bounding technique that is specific to our problem and filters unnecessary score computations. We conduct extensive experiments on real datasets and show the superiority of our algorithm over competitors.
