This paper proposes FastAlign, a faster, cheaper, and practical end-to-end solution for sequence alignment using commercial CPUs. It introduces two key innovations: a multistage seeding algorithm that improves search performance while maintaining low memory consumption, and an intraquery parallel seed-extension algorithm that eliminates redundancy and increases SIMD utilization. Evaluation results show that FastAlign achieves 2.27x similar to 3.28x throughput speedup and 2.54x similar to 5.65x cost reduction compared to stateof-the-art CPU and GPU baselines while guaranteeing 100% identical output to the de facto software BWA-MEM. FastAlign is open-sourced at https://github.com/zzhofict/ BWAFastAlign.git.

As training scales grow, collective communication libraries (CCL) increasingly face anomalies arising from complex interactions among hardware, software, and environmental factors. These anomalies typically manifest as slow/hang communication, the most frequent and time-consuming category to diagnose. However, traditional diagnostic methods remain inaccurate and inefficient, frequently requiring hours or even days for root cause analysis. To address this, we propose CCL-D, a high-precision diagnostic system designed to detect and locate slow/hang anomalies in large-scale distributed training. CCL-D integrates a rank-level real-time probe with an intelligent decision analyzer. The probe measures cross-layer anomaly metrics using a light-weight distributed tracing framework to monitor communication traffic. The analyzer performs automated anomaly detection and root-cause location, precisely identifying the faulty GPU rank. Deployed on a 4,000-GPU cluster over one year, CCL-D achieved near-complete coverage of known slow/hang anomalies and pinpointed affected ranks within 6 minutes-substantially outperforming existing solutions.

Molecular crystal structure prediction (CSP) faces a persistent computational bottleneck: it requires exhaustive sampling of vast packing landscapes while resolving energy differences of only a few kJ mol-1. We introduce BOMLIP-CSP, an open-source Python framework that integrates machine learning interatomic potentials (MLIPs) with a tailored batched optimization strategy, enabling rapid, unbiased structure prediction across the full crystal density range. By introducing tailored parallelism into modern MLIPs, BOMLIP-CSP achieves a similar to 2.1-2.3 & times; acceleration in large-scale CSP searches without compromising accuracy. In benchmarks covering 34 experimental structures from six CSP blind tests, more than 50% of the experimental crystal structures can be recovered using foundation MLIPs when the correct space group and Z ' are included in the search, with more than 70% of the experimental structures recovered by at least one of the tested MLIPs under the present benchmark conditions. Importantly, our results suggest that MLIPs with comparable equilibrium-energy accuracy can yield strikingly different CSP outcomes, indicating that predictive success may depend not only on local energy fidelity but also on how the MLIP energy surface is organised. Together, these results establish BOMLIP-CSP as a broadly accessible platform for accelerated CSP and provide new insight into the interplay between MLIP characteristics and crystal structure discovery.

The solution of sparse linear systems constitutes the dominant computational bottleneck in interior point methods (IPMs), frequently consuming over 70% of the total solution time. As optimization problems scale to millions of variables, direct solvers encounter prohibitive fill-in, excessive memory consumption, and limited parallel scalability. We present SDSL-Solver, a scalable distributed sparse linear solver framework designed for IPMs. SDSL-Solver employs Krylov subspace methods, combined with numerics-based sparse filtering and diagonal correction techniques that produce high-quality preconditioners. To accommodate diverse problem characteristics, SDSL-Solver offers two complementary distributed parallel methods: Block Jacobi for well-conditioned, diagonally dominant systems, and Bordered Block Diagonal (BBD) for ill-conditioned problems requiring globally coupled preconditioning via Schur complement techniques. A preconditioner reuse strategy further amortizes construction costs across consecutive IPMs iterations. We evaluate SDSL-Solver on benchmark problems with matrix dimensions ranging from tens of thousands to over five million on multi-node clusters equipped with X86 processors. The experimental results show that under the Block Jacobi and BBD distributed methods, SDSL-Solver on a four-node configuration achieves average speedups of 6.23× and 7.77×, respectively, compared to PETSc running on the same number of nodes. Relative to the single-node PARDISO, the average speedups reach 97.54× and 5.85×, respectively.

Burst buffers (BBs) act as an intermediate storage layer between compute nodes and parallel file systems (PFS), effectively alleviating the I/O performance gap in high-performance computing (HPC). As scientific simulations and AI workloads generate larger checkpoints and analysis outputs, BB capacity shortages and PFS bandwidth bottlenecks are emerging, and CPU-based compression is not an effective solution due to its high overhead. We introduce Computational Burst Buffers (CBBs), a storage paradigm that embeds hardware compression engines such as application-specific integrated circuit (ASIC) inside computational storage drives (CSDs) at the BB tier. CBB transparently offloads both lossless and error-bounded lossy compression from CPUs to CSDs, thereby (i) expanding effective SSD-backed BB capacity, (ii) reducing BB-PFS traffic, and (iii) eliminating contention and energy overheads of CPU-based compression. Unlike prior CSD-based compression designs targeting databases or flash caching, CBB co-designs the burst-buffer layer and CSD hardware for HPC and quantitatively evaluates compression offload in BB-PFS hierarchies. We prototype CBB using a PCIe 5.0 CSD with an ASIC Zstd-like compressor and an FPGA prototype of an SZ entropy encoder, and evaluate CBB on a 16-node cluster. Experiments with four representative HPC applications and a large-scale workflow simulator show up to 61% lower application runtime, 8-12x higher cache hit ratios, and substantially reduced compute-node CPU utilization compared to software compression and conventional BBs. These results demonstrate that compression-aware BBs with CSDs provide a practical, scalable path to next-generation HPC storage.

Transformer architecture gradually dominates the LLM field. Recent advances in training optimization for Transformer-based large language models (LLMs) primarily focus on architectural modifications or optimizer adjustments. However, these approaches lack systematic optimization of weight patterns during training. Weight pattern refers to the distribution and relative magnitudes of weight parameters in a neural network. To address this issue, we propose a Weight Scaling method called WISCA to enhance training efficiency and model quality by strategically improving neural network weight patterns without changing network structures. By rescaling weights while preserving model outputs, WISCA indirectly optimizes the model's training trajectory. Experiments demonstrate that WISCA significantly improves convergence quality (measured by generalization capability and loss reduction), particularly in LLMs with Grouped Query Attention (GQA) architectures and LoRA fine-tuning tasks. Empirical results show 5.6

Long-context LLM training suffers from a load-balancing problem that sequence packing does not solve. Packing samples into fixed-token sequences balances memory and linear-cost operators, but the dominant attention cost scales with the sum of squared sequence lengths. Thus, equally sized packed sequences drawn from a long-tailed corpus can carry substantially different attention workloads, creating data-parallel stragglers and pipeline bubbles. Existing approaches either balance at the granularity of sequences or microbatches, where an outlier can dominate an assignment, or disaggregate attention over a global worker pool whose communication domain grows with the data-parallel (DP) degree. We present Libra, which operationalizes the law of large numbers (LLN) as a scaling principle for load balancing: the attention-balancing pool need not grow with the DP degree. Libra groups packed sequences and their CP groups into fixed-size sequence pools. As DP scales out, Libra adds pools rather than enlarging each one, bounding every attention exchange. Variance-Reduced Sequence Placement makes this effective for finite, long-tailed workloads by co-locating sequences with complementary attention workloads to reduce residual inter-pool skew. Within each pool, Tiled Attention Pooling dispatches sequence-head SH-Tiles across GPUs, while a pipelined runtime overlaps tile exchange with attention. Libra exposes a drop-in context-parallel attention operator and a pluggable data sampler, requiring no changes to model layers, optimizers, or pipeline schedules. On Qwen3-Turbo training with 256K- and 1M-token workloads, Libra improves end-to-end throughput by up to 2.54x over Ulysses, with up to 3.14x worst-step straggler-attention speedup in microbenchmarks. Libra has run for hundreds of thousands of GPU-hours in production on jobs spanning 32K to 1M tokens while preserving training semantics.

How can we determine whether a trained neural network is already deep enough? We study this under a fixed function-preserving residual-growth protocol specifying insertion locations, residual families, zero-output initializations, and zero-state first-order updates. We define first-order residual depth saturation as the absence of a strict local decrease from every admissible insertion. We prove residual non-degeneracy is necessary and sufficient: additional depth has first-order value exactly when conditional activation gradients have a nonzero projection onto at least one admissible residual tangent space. This boundary is shared by descent-compatible zero-state updates and invariant under regular local reparameterizations preserving that tangent space. Under residual-signal realizability, raw activation-gradient vanishing exactly certifies saturation. Across ResNets, GPT-2-style models, and continued-pretrained Pythia checkpoints, the maximum activation-gradient norm decreases toward a low-signal regime with depth. Function-preserving growth also achieves converged performance competitive with training from scratch. These results support activation-gradient magnitude as a conservative diagnostic of the remaining empirical first-order value of residual depth.

Foundation atomistic models—pretrained, general-purpose machine-learned interatomic potentials—offer a promising path toward universal, data-efficient atomistic simulation across chemistry and materials science. However, their ability to capture rare, collective phenomena in complex molecular systems remains largely untested. Here, we systematically evaluate the transferability and fine-tuning of four state-of-the-art foundation atomistic models (MACE-MP-0b, MACE-OFF-small, SevenNet, and CHGNet) for simulating the reversible α ⇌ β polymorphic phase transition in the prototypical organic molecular crystal 2,4,5-triiodo-1H-imidazole (tIIm). Out-of-the-box, none of the models can reproduce the complete reversible transition, reflecting their limitations in capturing subtle and finely balanced non-covalent interactions outside their pretraining domain. By contrast, targeted fine-tuning—especially using our recently developed parameter-efficient equivariant low-rank adaptation (ELoRA) method—enables robust simulation of both transition directions and recovers the full collective dynamics of the phase change. Mechanistic analysis reveals a stepwise transition pathway with pronounced asymmetry in energy barriers, consistent with experimental observations. Our results establish practical guidelines for adapting foundation atomistic models to challenging materials phenomena and demonstrate that efficient fine-tuning protocols are essential for realizing the promise of universal, transferable machine-learned interatomic potentials in molecular materials simulation.

Recent advancement of distributed deep learning have heightened the demands for efficient cluster resource management. Existing frameworks are primarily designed for deterministic jobs, assuming exclusive resource assignments over predictable execution durations. However, they overlook one unique characteristic of DNN model exploration that it is a trial-and-error process. Exploratory training jobs constitute a significant portion of the deep learning workload. Unlike deterministic jobs, they are usually much shorter, in terms of execution time. This is caused by early-terminations during the exploration process, e.g., hyperparameter tuning and code debugging. This makes exploratory jobs more sensitive to queuing latency than deterministic jobs, causing both poor user-experience and low system utilization. This paper revisits the resource management for exploratory training jobs from a novel perspective by enabling exploratory jobs to opportunistically share resources assigned to deterministic jobs. A novel framework, CouchSurfer, is proposed to host exploratory jobs as resource-agnostic, low-priority surfers attach to long-running deterministic jobs. CouchSurfer provides low queuing latency by bypassing traditional scheduling process for exploratory jobs. This paper focuses on solving two key challenges: (1) identifying appropriate placements for exploratory jobs, and (2) mitigating performance interference caused by co-location. We introduce Exploratory Job Placement Problem (EJPP), targeting on low-latency model exploration, and present a threshold-guided algorithm to compute fast and efficient placement plans. Furthermore, CouchSurfer incorporates a Job Fusion Mechanism (JFM) that performs fine-grained kernel-level scheduling to achieve low-interference job co-location. Both testbed experiments and large-scale simulations are conducted to demonstrate CouchSurfer's effectiveness that it reduces 85.2% job queuing latency than other schedulers.

We have optimized the parallel threshold ILU algorithm (ParILUT) for GPUs. The optimizations are for three building blocks: candidate search and ILU residual computation, adding and removing elements, and threshold selection. Firstly, we fuse candidate search and ILU residual computation by modifying the ParILUT algorithm and extending the register-aware SpGEMM algorithm to calculate it. At the same time, we developed a GPU bin search algorithm to make the register-aware SpGEMM algorithm perform better in ParILUT. Secondly, we adopt a warp-row-parallel approach to add elements to new L and U and remove elements from candidates instead of the thread-row-parallel approach. And used the efficient GPU instructions to locate the positions of elements. Thirdly, we proposed a balanced classification tree in the threshold selection to balance the buckets’ data, when a large number of elements with the same value. Finally,we experimented with the performance of each optimization and the whole ParILUT. And verified the correctness of the optimized ParILUT. The result indicates that the optimized ParILUT average speedup is 4.03 times over the original version, and the speedup increases with the amount of fill-in.

Compared with replication-based storage, erasure-coded storage suffers from high update overhead due to long update paths and intensive random I/O. Existing parity logging methods often fail to satisfy the performance requirements of modern applications because of high latency and low throughput. To address this issue, we propose TSUE+, an efficient update framework with a swift recycling mechanism. TSUE+ divides updates into synchronous and asynchronous stages. In the synchronous stage, updates are stored as replica-style data logs, transforming random I/O into sequential writes through a space-for-time tradeoff. In the asynchronous stage, logs are recycled and merged into original data and parity blocks, reclaiming temporary storage overhead. To further reduce recycling costs, TSUE+ employs a three-layer log structure and locality-aware recycling that exploits spatiotemporal access patterns. We evaluated TSUE+ against state-of-the-art update mechanisms under diverse erasure coding schemes across heterogeneous storage devices, including HDDs, SATA SSDs, NVMe SSDs, and PMEM. Experiments were conducted using real-world traces from MSR Cambridge, Alibaba Cloud, and Tencent Cloud, as well as synthetic worst-case workloads. Results show that TSUE+ consistently achieves substantial performance improvements across all platforms while maintaining strong robustness under tail-latency and worst-case evaluations. Moreover, by enabling prompt log recycling, avoiding unnecessary overwrites, and improving update granularity, TSUE+ also mitigates SSD write wear and extends device lifespan.

AI-driven methods have demonstrated considerable success in tackling the central challenge of accurately solving the Schrödinger equation for complex many-body systems. Among neural network quantum state (NNQS) approaches, the NNQS-SCI (Selected Configuration Interaction) method stands out as a state-of-the-art technique, recognized for its high accuracy and scalability. However, its application to larger systems is severely constrained by a hybrid CPU-GPU architecture. Specifically, centralized CPU-based global de-duplication creates a severe scalability barrier due to communication bottlenecks, while host-resident coupled-configuration generation induces prohibitive computational overheads. We introduce QiankunNet-cuSCI, a fully GPU-accelerated SCI framework designed to overcome these bottlenecks. It first integrates a distributed, load-balanced global de-duplication algorithm to minimize redundancy and communication overhead at scale. To address compute limitations, it employs specialized, fine-grained CUDA kernels for exact coupled configuration generation. Finally, to break the single-GPU memory barrier exposed by this full acceleration, it incorporates a GPU memory-centric runtime featuring GPU-side pooling, streaming mini-batches, and overlapped offloading. This design enables much larger configuration spaces and shifts the bottleneck from host-side limitations back to on-device inference. Our evaluation demonstrates that our work fundamentally expands the scale of solvable problems. On an NVIDIA A100 cluster with 64 GPUs, our work achieves up to 2.32X end-to-end speedup over the highly-optimized NNQS-SCI baseline while preserving the same chemical accuracy. Furthermore, it demonstrates excellent distributed performance, maintaining over 90

With the rapid expansion of genomic sequencing data over the years, the costs associated with storage, transmission, and bandwidth are becoming the primary bottlenecks in genomic research and applications. Data compression is widely used to alleviate this burden, provided it achieves a sufficiently high compression ratio and fast compression speed. MPEG-G is a genome-specific compression standard that offers a higher compression ratio than general-purpose compression tools (4.3x), however, at the cost of performance reduction (5x). Following common strategies in compression acceleration, we design to the best of our knowledge, the first hardware accelerator for the MPEG-G genomic data compression pipeline utilizing in-SRAM (Static Random-Access Memory) computing, referred to as iSCoder. We identify and analyze MatchC (Match Coding) and LutC (Lut Coding) as two bottleneck algorithms within this pipeline, propose two optimized in-SRAM algorithms, and design a unified hardware architecture for these algorithms, considering the characteristics of genomic data. Compared with 72-core Intel processors operating at 3.0 GHz, experimental results demonstrate that iSCoder achieves an average speedup of 131x for MatchC and 191x for LutC.

Discovering atom-level phenomena requires molecular dynamics (MD) simulations with ab initio accuracy. Machine learning interatomic potentials (MLIPs) enable stable, high-accuracy MD simulations, and their models exhibit scaling-law trends similar to large language models. However, the lack of scalable and efficient distributed training systems for conservative MLIPs makes them difficult to scale. This is because conservative MLIPs inherently follow a double-backward execution pattern, which involves computing gradients during the forward pass. This pattern creates a mismatch with existing distributed training systems, especially for pipeline parallelism. Therefore, we present JanusPipe, an efficient 3D-parallel (PP/DP/GP) training system tailored for conservative MLIPs. It integrates SymFold to enable memory-efficient pipeline parallelism for conservative MLIPs, and WaveK to reduce pipeline bubbles by balancing the four-phase compute time. Experimental results on 32 GPUs show that JanusPipe improves throughput by 1.51× and 1.45× on average over 1F1B and Hanayo, respectively.

Collective communication is critical to scaling large language model (LLM) training across various parallelism strategies, including data, tensor, and pipeline parallelism on GPU clusters. However, as model sizes and training scales increase, communication overhead is emerging as a major performance bottleneck. While compression is a promising mitigation strategy, existing solutions often lack user-transparency, hinder deployment and extensibility, and are not co-designed with communication algorithms. To address these limitations, we present COCCL, a high-performance collective communication library built on top of NCCL. COCCL introduces a novel programming model that can easily integrate compression into communication workflows with flexible configurability. It features a suite of compression-aware collective algorithms and runtime overlap mechanisms that mitigate error propagation and reduce computational overhead. We integrate well-established compression techniques into COCCL and tune the compression configurations during 3D-parallel training on GPT and Qwen models with up to 7 billion parameters. Using the optimal configuration (COCCL-3D), we achieve 1.24x throughput improvement while maintaining training accuracy.

Flexible macromolecular docking offers high-fidelity predictions of biomolecular interactions, but remains prohibitively expensive at scale. Among existing approaches, LightDock leverages Glowworm Swarm Optimization (GSO) for accuracy, yet suffers from limited parallelism, irregular computation, and severe load imbalance, preventing efficient execution on GPU supercomputers. We present SparkleDock, a scalable GSO-based docking framework enabling near-real-time flexible docking. We redesign GSO to expose massive fine-grained parallelism at the glowworm-agent level, and restructure the dominant energy scoring computation into a Tensor Core-compatible formulation, enabling efficient execution of irregular pairwise interactions through structured matrix operations. We further introduce a performance-model-driven scheduling for load balancing and out-of-core scaling across GPUs. SparkleDock achieves 9.7 × and 18.9 × speedups over LightDock on single A100 and H100 GPU, and delivers over two orders of magnitude acceleration at scale. On 512 GPUs, it reduces docking time from hours to seconds, enabling large-scale, high-fidelity virtual screening previously impractical with flexible docking.

CUDA-to-SYCL translation is important for performance portability, yet rule-based migration tools often struggle to preserve parallel hierarchy, locality, and CUDA-specific optimization structure. We present LLM-SYCL, a search-driven SYCL generation backend for QiMeng-Xpiler. Starting from a normalized loop representation recovered from CUDA code, LLM-SYCL reconstructs SYCL thread binding, local-memory tiling, and subgroup-oriented arithmetic using target-aware prompts, guarded post-processing, and compile-and-run validation. On NVIDIA A100, all 20 regular GEMM cases compile, run, and pass correctness, with type-level geometric-mean retention of 67.5

Non-dedicated training environments such as public clouds have become increasingly important for research communities because they are both user-friendly and budget-friendly. But the performance difference from heterogeneous devices in these environments causes straggler issues that can severely hinder parallel training performance, especially for data parallelism. Existing solutions either temporarily relax the consistency of the global model state to tolerant transient stragglers or adjust the local batch size (lbs) among parallel tasks to mitigate long-term stragglers. These approaches inevitably cause different degrees of degradation in model accuracy. This paper first establishes the Batch Uniformity Theorem to quantify the accuracy deviation caused by non-uniform lbs, and then proposes a novel training framework DelTa, which enforces parallel DNN training with uniform lbs to maintain accuracy. A scalable Thread-level Worker (TW) is proposed to support fine-grained resource sharing, along with a device-level Worker Fusion Mechanism (WFM) for concurrent TWs. We introduce a novel TW placement problem that seeks load-balanced TW configurations with the shortest training iteration time and propose an algorithm as a solution. The results of extensive testbed experiments demonstrate that DelTa improves training throughput and test accuracy by 19.7% and 12.9%, respectively, compared with state-of-the-art systems.