Modern High-Performance Computing (HPC) centers are facing significant challenges in ingesting large and diverse data streams, leading to bandwidth bottlenecks and scientific delays. To address the shortcomings of traditional static allocation and simple queuing methods, this work introduces a dynamic, value-based approach to bandwidth allocation. We propose two new auction-based mechanisms: the computationally efficient Greedy Value Density Auction and the theoretically robust Vickrey-Clarke-Groves (VCG) Knapsack Auction. Both mechanisms utilize user-provided bids that specify data requirements and scientific value, with the goal of maximizing the total value of successful data transfers. Our simulation results, based on realistic workload characteristics, demonstrate that these auction mechanisms significantly outperform standard First-Come, First-Served (FCFS) baselines. In high-load scenarios, our methods reduce average and tail completion delays by over 80% and improve predictability by decreasing the coefficient of variation of delay by up to 85%. Furthermore, network stability is enhanced, with the peak-to-average load ratio dropping by as much as 70%. This value-driven, adaptive strategy alleviates congestion, improves bandwidth utilization, and ensures that access is prioritized based on scientific importance, ultimately accelerating scientific discovery.
Clustering is the process of finding natural groups within a data set such that patterns within a group are more similar than patterns belonging to different groups. It has been used in a wide range of scientific and engineering disciplines. Yet, clustering is also a difficult unsupervised problem without an absolute ground truth. In practice, the "best" clustering method is the one that produces the most interpretable results. There is no universal optimum way to select the number of clusters. Our ultimate goal is to enable domain experts in high-stakes fields to make informed decisions when choosing a final, insightful, and actionable clustering solution for complex problems. In such challenging scenarios, ensemble clustering is a technique where multiple clustering results are combined to produce a more robust and stable final clustering. Here, inspired by a medical application, we take a novel approach to ensemble clustering; rather than focusing on an optimum global clustering, we identify persistent cluster(s) by multiple techniques across multiple clustering experiments among patients from Rush University Medical Center in Chicago. The key novelty resides in reducing the uncertainty in clustering, in the absence of ground truth, by looking at persistent clusters across the ensemble clustering. In healthcare, physicians aim to treat a patient based on a series of symptoms that contribute to a single disease. Because most clinical guidelines focus on individual diseases, managing patients with multimorbidity—when a person has multiple (chronic) conditions—can be challenging. We aim to discover clusters of diseases with some level of confidence to assist physicians in diagnosing and treating patients with multimorbidity. In this case study, we identify and thoroughly characterize a cluster of diseases that affects primarily women and persists through clustering the data from k=16 to k=20 clusters using three clustering techniques. Additionally, our first proposed multimorbidity cluster identified a network connecting asthma, breast cancer, hip/pelvic fracture, endometrial cancer, and non-Alzheimer’s dementia. We use multiple clustering techniques, consensus metrics, as well as graph analysis and visualization to provide confidence and foster trust from domain experts in our results. We are working with physicians to provide a medical explanation for this data-driven disease multimorbidity discovery.
Timely and coordinated flood response is often hindered by fragmented data and delayed situational awareness. This paper presents a multi-agent system (MAS) that integrates geolocated social media posts and IoT sensor data to enable dynamic and high-confidence flood detection and alerting. Each agent is responsible for a specific function, including filtering noisy tweets, validating water level readings, and clustering incident reports. In a simulated urban flood scenario, the system significantly improved event coverage, reduced response time, and lowered false alarms by validating information across multiple sources. These results highlight the potential of intelligent agent collaboration to enhance real-time disaster monitoring and response.
This work bridges the gap between multi-GPU computing and high-productivity, performance-portable programming solutions. Our goal is to enhance scientific applications with a productive and portable solution-program once, deploy everywhere-for multi-GPU programming with no cost to programmability. To accomplish this, we implemented JACC.Multi, which is part of the Julia for ACCelerators (JACC) performance-portable framework. JACC.Multi is the only high-level, portable metaprogramming solution that targets multi-GPU environments and is integrated in a readily accessible programming language (e.g., Julia language). With transparent GPU-to-GPU communication, JACC.Multi is optimized for scientific application workloads and is portable for NVIDIA and AMD accelerators. For the evaluation, we use two modern multi-GPU systems: Hudson, which features two NVIDIA H100 Hopper GPUs per node, and Frontier, which features four AMD MI250X GPUs per node, each with two Graphics Compute Dies (GCDs) for a total of eight GCDs per node. Additionally, as part of the evaluation, we use JACC (one GPU), MPI+JACC, and JACC.Multi codes that implement well-known and widely used scientific algorithms/kernels such as the conjugate gradient algorithm and an explicit forward Euler solver that requires GPU-to-GPU communication. Overall, JACC.Multi codes achieve better performance than MPI+JACC codes and significant speedups over JACC (one GPU), with up to 1.9x on Hudson and 6x on Frontier.
Atmospheric composition is an essential part of weather, climate and Earth system modeling. However, modeling atmospheric composition is a computationally expensive and time-consuming task that requires a significant amount of energy. As models scale to finer spatial and temporal resolutions, maintaining real-time performance becomes increasingly challenging. To address this, optimization and acceleration techniques are essential. One promising approach is the use of deep neural networks, which have demonstrated the capability to efficiently approximate complex systems with high accuracy. Predictions using these neural networks are notably faster compared to traditional methods, significantly reducing the computational burden. In this study, we present the development of a surrogate model designed to emulate ISORROPIA, a traditional model used for calculating the concentrations of chemical compounds in the ICON-ART (ICOsahedral Nonhydrostatic model with Aerosol and Reactive Trace gases) model. Specifically, ISORROPIA is an aerosol thermodynamic equilibrium model used by ART that requires substantial computational resources, occupying a significant portion of the overall calculation time, making it particularly well-suited for emulation. The methodology involved generating a comprehensive dataset using the traditional model, which served as the training data for the neural network. This dataset encompassed a wide range of chemical concentrations and conditions, ensuring the neural network could effectively learn the underlying patterns and relationships for real-life scenarios. A simple feedforward architecture was used and fine-tuned with the primary goal of maintaining a low approximation error while also striving to achieve the lowest possible inference timing. After training, the new neural network model was compared to ISORROPIA on ICON-ART simulation data. The results demonstrated that the neural network model successfully achieved the desired outcomes, maintaining low approximation error across the globe and efficient inference timing.
