Modern High-Performance Computing (HPC) centers are facing significant challenges in ingesting large and diverse data streams, leading to bandwidth bottlenecks and scientific delays. To address the shortcomings of traditional static allocation and simple queuing methods, this work introduces a dynamic, value-based approach to bandwidth allocation. We propose two new auction-based mechanisms: the computationally efficient Greedy Value Density Auction and the theoretically robust Vickrey-Clarke-Groves (VCG) Knapsack Auction. Both mechanisms utilize user-provided bids that specify data requirements and scientific value, with the goal of maximizing the total value of successful data transfers. Our simulation results, based on realistic workload characteristics, demonstrate that these auction mechanisms significantly outperform standard First-Come, First-Served (FCFS) baselines. In high-load scenarios, our methods reduce average and tail completion delays by over 80% and improve predictability by decreasing the coefficient of variation of delay by up to 85%. Furthermore, network stability is enhanced, with the peak-to-average load ratio dropping by as much as 70%. This value-driven, adaptive strategy alleviates congestion, improves bandwidth utilization, and ensures that access is prioritized based on scientific importance, ultimately accelerating scientific discovery.
Clustering is the process of finding natural groups within a data set such that patterns within a group are more similar than patterns belonging to different groups. It has been used in a wide range of scientific and engineering disciplines. Yet, clustering is also a difficult unsupervised problem without an absolute ground truth. In practice, the "best" clustering method is the one that produces the most interpretable results. There is no universal optimum way to select the number of clusters. Our ultimate goal is to enable domain experts in high-stakes fields to make informed decisions when choosing a final, insightful, and actionable clustering solution for complex problems. In such challenging scenarios, ensemble clustering is a technique where multiple clustering results are combined to produce a more robust and stable final clustering. Here, inspired by a medical application, we take a novel approach to ensemble clustering; rather than focusing on an optimum global clustering, we identify persistent cluster(s) by multiple techniques across multiple clustering experiments among patients from Rush University Medical Center in Chicago. The key novelty resides in reducing the uncertainty in clustering, in the absence of ground truth, by looking at persistent clusters across the ensemble clustering. In healthcare, physicians aim to treat a patient based on a series of symptoms that contribute to a single disease. Because most clinical guidelines focus on individual diseases, managing patients with multimorbidity—when a person has multiple (chronic) conditions—can be challenging. We aim to discover clusters of diseases with some level of confidence to assist physicians in diagnosing and treating patients with multimorbidity. In this case study, we identify and thoroughly characterize a cluster of diseases that affects primarily women and persists through clustering the data from k=16 to k=20 clusters using three clustering techniques. Additionally, our first proposed multimorbidity cluster identified a network connecting asthma, breast cancer, hip/pelvic fracture, endometrial cancer, and non-Alzheimer’s dementia. We use multiple clustering techniques, consensus metrics, as well as graph analysis and visualization to provide confidence and foster trust from domain experts in our results. We are working with physicians to provide a medical explanation for this data-driven disease multimorbidity discovery.
Timely and coordinated flood response is often hindered by fragmented data and delayed situational awareness. This paper presents a multi-agent system (MAS) that integrates geolocated social media posts and IoT sensor data to enable dynamic and high-confidence flood detection and alerting. Each agent is responsible for a specific function, including filtering noisy tweets, validating water level readings, and clustering incident reports. In a simulated urban flood scenario, the system significantly improved event coverage, reduced response time, and lowered false alarms by validating information across multiple sources. These results highlight the potential of intelligent agent collaboration to enhance real-time disaster monitoring and response.
This work bridges the gap between multi-GPU computing and high-productivity, performance-portable programming solutions. Our goal is to enhance scientific applications with a productive and portable solution-program once, deploy everywhere-for multi-GPU programming with no cost to programmability. To accomplish this, we implemented JACC.Multi, which is part of the Julia for ACCelerators (JACC) performance-portable framework. JACC.Multi is the only high-level, portable metaprogramming solution that targets multi-GPU environments and is integrated in a readily accessible programming language (e.g., Julia language). With transparent GPU-to-GPU communication, JACC.Multi is optimized for scientific application workloads and is portable for NVIDIA and AMD accelerators. For the evaluation, we use two modern multi-GPU systems: Hudson, which features two NVIDIA H100 Hopper GPUs per node, and Frontier, which features four AMD MI250X GPUs per node, each with two Graphics Compute Dies (GCDs) for a total of eight GCDs per node. Additionally, as part of the evaluation, we use JACC (one GPU), MPI+JACC, and JACC.Multi codes that implement well-known and widely used scientific algorithms/kernels such as the conjugate gradient algorithm and an explicit forward Euler solver that requires GPU-to-GPU communication. Overall, JACC.Multi codes achieve better performance than MPI+JACC codes and significant speedups over JACC (one GPU), with up to 1.9x on Hudson and 6x on Frontier.
Atmospheric composition is an essential part of weather, climate and Earth system modeling. However, modeling atmospheric composition is a computationally expensive and time-consuming task that requires a significant amount of energy. As models scale to finer spatial and temporal resolutions, maintaining real-time performance becomes increasingly challenging. To address this, optimization and acceleration techniques are essential. One promising approach is the use of deep neural networks, which have demonstrated the capability to efficiently approximate complex systems with high accuracy. Predictions using these neural networks are notably faster compared to traditional methods, significantly reducing the computational burden. In this study, we present the development of a surrogate model designed to emulate ISORROPIA, a traditional model used for calculating the concentrations of chemical compounds in the ICON-ART (ICOsahedral Nonhydrostatic model with Aerosol and Reactive Trace gases) model. Specifically, ISORROPIA is an aerosol thermodynamic equilibrium model used by ART that requires substantial computational resources, occupying a significant portion of the overall calculation time, making it particularly well-suited for emulation. The methodology involved generating a comprehensive dataset using the traditional model, which served as the training data for the neural network. This dataset encompassed a wide range of chemical concentrations and conditions, ensuring the neural network could effectively learn the underlying patterns and relationships for real-life scenarios. A simple feedforward architecture was used and fine-tuned with the primary goal of maintaining a low approximation error while also striving to achieve the lowest possible inference timing. After training, the new neural network model was compared to ISORROPIA on ICON-ART simulation data. The results demonstrated that the neural network model successfully achieved the desired outcomes, maintaining low approximation error across the globe and efficient inference timing.
Discovering novel molecules with targeted properties remains a formidable challenge in materials science, often likened to finding a needle in a haystack. Traditional experimental approaches are slow, costly, and inefficient. In this study, we present an inverse design framework based on a molecular graph conditional variational autoencoder (CVAE) that enables the generation of new molecules with user-specified optical properties, particularly molar extinction coefficient (epsilon). Our model encodes molecular graphs, derived from SMILES strings, into a structured latent space, and then decodes them into valid molecular structures conditioned on a target epsilon value. Trained on a curated dataset of known molecules with corresponding extinction coefficients, the CVAE learns to generate chemically valid structures, as verified by RDKit. Subsequent Density Functional Theory (DFT) simulations confirm that many of the generated molecules exhibit the electronic structures similar to those molecules with desired epsilon values. We have also verified the epsilon values of the generated molecules using a graph neural network (GNN) and the synthesizability of those molecules using an open-source module named ASKCOS. This approach demonstrates the potential of CVAEs to accelerate molecular discovery by enabling user-guided, property-driven molecule generation - offering a scalable, data-driven alternative to traditional trial-and-error synthesis.
Members of the Western Michigan Transformative Interdisciplinary Human+AI Research Group have been engaged in two consecutive NSF-funded projects to promote AI readiness in diverse STEM disciplines. Putting equal emphasis on theory and practice, our goal is to instill knowledge and competency in safe, secure, and reliable AI across a wide range of learners from high school students through to university students and practitioners who wish to upskill. The second project that is currently underway has a specific focus on machine-assisted processing of massive data. This presentation focuses on the development of immersive learning experiences.
Scientific discovery is undergoing a fundamental transformation, driven by an unprecedented deluge of data from next-generation experimental facilities (such as High-Luminosity Large Hadron Collider), high-fidelity simulations (novel weather prediction), and ubiquitous sensor networks (such as National Ecological Observatory Network). From the petabytes generated by the Large Hadron Collider to the terabytes streamed daily from genomic sequencers and climate models, the scale and complexity of scientific data have outpaced traditional methods of analysis. In response, the scientific community is increasingly turning to Artificial Intelligence (AI), particularly Machine Learning (ML) and Deep Learning (DL), as an indispensable tool for knowledge extraction. AI is not merely accelerating existing processes; it is enabling entirely new modes of inquiry. It allows physicists to identify faint particle signatures in noisy detector data, helps biologists predict protein structures from amino acid sequences, and empowers astronomers to classify celestial objects in vast sky surveys.
We introduce FedCostAware, a cost-aware scheduling algorithm designed to optimize synchronous federated learning (FL) on cloud spot instances, which addresses the challenges of training on spot instances and different client budgets by employing intelligent management of the lifecycle of spot instances. This approach minimizes idle resource time and overall expenses. Experiments on real-world medical datasets demonstrate that FedCostAware significantly reduces cloud computing costs compared to conventional spot and on-demand schemes, enhancing the accessibility and affordability of FL.
Recent innovations in 3D reconstruction and the rise in popularity of Digital Twins present a unique opportunity for the integration of the two technologies for high quality data visualization. In this work, I present an architecture for integrating Gaussian Splat reconstructions with dynamic data and user interaction, implemented in Unreal Engine. I also explore the application of this method to digital twin creation and how it addresses specific visualization challenges.
Earth observation (EO) scientists use sophisticated algorithms, large datasets, and high-performance data analytics (HPDA) clusters to develop and execute complex EO workflows. To improve portability and reusability within the domain, the Open Geospatial Consortium (OGC) published a set of best practices for developing EO workflows. Even though EO data products are often shared openly, representative EO workflow implementations are hardly available.In this paper, we contribute a case study on implementing and scaling a complex EO workflow, adhering to the OGC best practices and demonstrating its portability by deploying it both locally and on the HPDA platform "terrabyte" at the Leibniz Supercomputing Centre. Contentwise, the contributed workflow analyzes settlement patterns by first delineating coherent settlements derived from leveraged satellite data that maps builtup areas and subsequently calculating centrality measures to characterize the spatial arrangement and hierarchy of settlements in a given region.We demonstrate the workflow’s scalability and variance in resource demands by analyzing its time-to-result, total CPU time, and resource efficiency under different inputs and configurations. Interestingly, implicit parameters hidden in the input data semantics, like the number and area of settlements in the region of interest, significantly impact the time required to complete the processing. Concurrent processing is restricted to connected components of the settlement graph in the analysis stage, leading to an unbalanced workload distribution when analyzing large urban areas, showcasing the scalability challenges EO scientists face.
Mixed reality (MR) enables immersive, intuitive engagement with scientific data. When paired with AI-driven assistants, it has the potential to transform traditional workflows. In this paper, we introduce a modular agentic architecture for scientific visualization in MR, designed to balance general-purpose flexibility with domain-specific extensibility. Our modular architecture supports composable tools, contextual reasoning, and dynamic task execution. We outline a three-layer design, domain module integration, and orchestration of multistep workflows. We demonstrate the system’s capabilities through use cases in biology and general-purpose scientific visualization, including protein interaction networks and remote ParaView-based rendering. The result is a flexible and extensible foundation for spatial scientific computing.
This paper presents major enhancements to frances, a platform for exploring digitized historical collections using LLM-based semantic and spatial enrichments. We focus on three corpora from the National Library of Scotland (NLS): the Gazetteers of Scotland, the Broadsides, and the Encyclopaedia Britannica. Using a flexible, prompt-based LLM pipeline, we extract over 50,000 structured articles from the Gazetteers, adapting to diverse typographic layouts. For Broadsides, we align OCR-derived text (extracted using defoe) with manually corrected transcriptions published by NLS, linking both sources for improved access and reuse. Historical text is further improved using a fine-tuned Llama2 model selected after evaluating several LLMs for post-OCR correction. To enrich geographic information, we combine Stanza NER, in-text coordinate parsing, and the Edinburgh Geoparser, enabling both modern georesolution and preservation of historical geography. We extend the Heritage Textual Ontology (HTO) to support article-level records, spatiotemporal entities, and in-text annotations via CRMgeo and Web Annotation standards. Resulting RDF knowledge graphs are deployed on a GeoSPARQL-enabled Fuseki server and indexed in Elasticsearch to support full-text, semantic, and spatial search. The upgraded frances interface offers entity highlighting, historical maps, and provenance tracking, demonstrating a scalable pipeline for structured, enriched access to OCRed heritage texts.
Notebooks have become widely adopted in the scientific community due to their interactive interface and ease of sharing. However, using notebooks to execute large-scale scientific workflows remains challenging. Scientific workflows are typically distributed and require resource provisioning and data management prior to execution. Because notebooks do not natively embed workflow specifications, users often resort to inserting custom configuration steps directly within notebook cells to enable provisioning. This practice undermines reproducibility, as the same notebook may not run consistently across different cluster environments. In this paper, we introduce the concept of a notebook backpack—a companion specification that captures the embedded workflow along with all relevant configuration elements. We describe how notebook tracing can be leveraged to automatically populate the backpack. We then describe an integrated tool that provisions a backpack on distributed resources. Using real-world case studies, we demonstrate that the backpack abstraction enables minimal modification of the notebook, portable execution, and cross-site reproducibility of notebook-based workflows on HPC clusters without significantly increasing notebook execution time.
GEOtiled enables efficient computation of high-resolution terrain parameters from digital elevation models (DEMs) by decomposing large regions into smaller, parallelizable tiles. Originally developed as GEOtiled-G with support for three parameters (Slope, Aspect, Hillshade) via GDAL, we present GEOtiled-SG, an enhanced version that integrates the SAGA GIS library to compute over 15 parameters, expanding its utility in Earth science. To offset SAGA’s computational overhead, GEOtiled-SG introduces three optimizations: concurrent DEM cropping, buffer-aware mosaicking, and unified concurrency across workflow stages. Evaluations show that GEOtiled-SG maintains GEOtiled-G’s performance on the original parameters and offers consistent speedups across the expanded set. The framework is open source on GitHub, with data hosted on Dataverse, supporting reproducible, scalable terrain analysis.
Scientific data spans heterogeneous modalities and is distributed across diverse storage systems. Vector databases have emerged as powerful tools to index high-dimensional embeddings for semantic search, but most existing systems are centralized and focus on indexing a single modality. We introduce SMURF, a system for federated multimodal indexing and retrieval. SMURF aligns pre-computed embeddings from different scientific modalities into a unified embedding space. Our method extends semi-supervised learning techniques to connect disparate embeddings by combining geometry-preserving alignment with weak supervision using a small set of semantically matched data. We evaluate SMURF on cross-modal retrieval tasks using scientific datasets and find that aligned embeddings substantially improve performance-on average, top-5 accuracy nearly doubles (19.18%.38.82%), and nDCG@10, a ranking quality metric, increases by 78% (0.18.0.32) across various retrieval settings. These results highlight the promise of bridging modality gaps to support more effective scientific discovery in federated environments.
Scientific applications are turning to GPUs to analyze and produce data due to advances in AI technologies. However, getting data to the GPU from storage is a challenge. Although technologies such as NVIDIA GDS and NIXL exist, limited research has been conducted on the actual benefits and trade-offs of these technologies. Our motivation is to explore the benefits and trade-offs of using GDS and the NIXL library by running benchmarking experiments.
Global changes are increasing the frequency and intensity of extreme weather events, posing challenges for forecasting localized phenomena with sub-grid resolution. The Hi-WeFAI project addresses this by combining high-performance computing, Federated Artificial Intelligence, and heterogeneous sensor networks to improve short-term precipitation forecasting and flood nowcasting. In this paper, we present preliminary results using a transformer-based radar prediction model coupled with a flood model to generate high-resolution early warning maps. The results from the Naples pilot site improved accuracy and detail, highlighting the potential of the hybrid AI and HPC approach to support quasi-real-time decision-making and disaster risk reduction.
Structural color, which arises from the interaction of light with micro- and nanostructured materials, has been recognized as an environmentally sustainable alternative to traditional dyes and pigments. In this study, Direct Ink Writing has been utilized as the additive manufacturing technique for fabricating structurally colored materials which here, are bottlebrush block copolymers. The experimental workflow has been integrated with an artificial intelligence-driven optimization process, enabling an efficient exploration of the printing parameter space. Image analysis techniques have been employed to characterize the resultant color profiles, with hue in the HSV color space serving as the primary objective function for optimization. The development of software architecture that facilitates real-time data collection and feedback has been undertaken in collaboration with the National Center for Supercomputing Applications (NCSA). Preliminary results have demonstrated the feasibility of the proposed approach in tuning structural coloration, while ongoing efforts aim to enhance automation capabilities and expand the scope of parameter optimization.
Monitoring soil health at scale requires tools that are both scientifically robust and computationally efficient. Accurate and efficient estimation of soil chemical properties is essential for sustainable agricultural practices and environmental management. Traditional laboratory-based soil testing methods are labor-intensive, time-consuming, and often impractical for large-scale analysis. Although mid-infrared (MIR) spectroscopy offers a rapid and cost-effective alternative, conventional modeling techniques suffer from inconsistent predictive performance across diverse soil properties. To address this gap, we introduce FTIRNet, a novel multi-task Transformer-based architecture designed to predict multiple soil chemical properties from Fourier-transform infrared (FTIR) spectral data. FTIRNet employs a shared encoder block with task-specific Transformer encoders, enabling the model to learn both general and task-focused spectral representations. These multi-task Transformer encoders incorporate science-informed task relation learning through specialized fusion layers that capture known biogeochemical interdependencies among soil properties. Across a broad set of soil properties, FTIRNet demonstrates consistently superior predictive accuracy compared to traditional approaches.
