The widespread use of GPUs in cloud and high-performance computing makes memory isolation a critical security requirement. While the programming model assumes that each thread local memory is private, the underlying hardware does not always enforce this guarantee. Weaknesses in address translation can allow one thread to access another local memory, creating a semantic gap that enables cross-thread corruption and exploitation. To address these challenges, we propose CAMI, a hardware-level framework that integrates fine-grained execution context into the memory translation pipeline. CAMI enforces a binding between the execution context of each memory access and the ownership of its target memory page, ensuring that even subtle inconsistencies in translation cannot be exploited. By introducing an efficient hardware enforcement unit within the MMU and extending page table entries with ownership metadata, CAMI achieves strong, fine-grained isolation while maintaining low performance overhead. We implement CAMI in a cycle-accurate GPU simulator and conduct comprehensive evaluations. Results show that CAMI effectively eliminates cross-thread memory access vulnerabilities with minimal runtime cost, offering a practical path toward secure and high-performance GPU architectures.

AI-driven methods have demonstrated considerable success in tackling the central challenge of accurately solving the Schrödinger equation for complex many-body systems. Among neural network quantum state (NNQS) approaches, the NNQS-SCI (Selected Configuration Interaction) method stands out as a state-of-the-art technique, recognized for its high accuracy and scalability. However, its application to larger systems is severely constrained by a hybrid CPU-GPU architecture. Specifically, centralized CPU-based global de-duplication creates a severe scalability barrier due to communication bottlenecks, while host-resident coupled-configuration generation induces prohibitive computational overheads. We introduce QiankunNet-cuSCI, a fully GPU-accelerated SCI framework designed to overcome these bottlenecks. It first integrates a distributed, load-balanced global de-duplication algorithm to minimize redundancy and communication overhead at scale. To address compute limitations, it employs specialized, fine-grained CUDA kernels for exact coupled configuration generation. Finally, to break the single-GPU memory barrier exposed by this full acceleration, it incorporates a GPU memory-centric runtime featuring GPU-side pooling, streaming mini-batches, and overlapped offloading. This design enables much larger configuration spaces and shifts the bottleneck from host-side limitations back to on-device inference. Our evaluation demonstrates that our work fundamentally expands the scale of solvable problems. On an NVIDIA A100 cluster with 64 GPUs, our work achieves up to 2.32X end-to-end speedup over the highly-optimized NNQS-SCI baseline while preserving the same chemical accuracy. Furthermore, it demonstrates excellent distributed performance, maintaining over 90

The rapid progress of large language models (LLMs) has driven their deployment on heterogeneous computing platforms using accelerators such as GPUs and NPUs to meet growing computational demands. However, this paradigm transition introduces critical security and privacy risks that are uniquely intensified by the characteristics of LLM workloads. These models embed high intellectual property value, making them prime targets for adversaries. Furthermore, their autoregressive generation process makes them exceptionally sensitive to latency, magnifying the performance impact of any security overhead. Existing confidential computing solutions are predominantly CPU-centric, leaving critical computation on accelerators unprotected. This paper provides a survey of technologies for securing LLMs on heterogeneous confidential computing platforms. We analyze the threat landscape of heterogeneous LLM deployment, identifying risks across hardware interconnects, complex software stacks, and LLM-specific properties. These risks include data leakage during transfer, interference in multi-tenant execution, and limited trust in accelerator components. To address these challenges, we review key enabling technologies including secure I/O, tenant isolation, and device attestation, which together form the foundation for building trusted execution paths across heterogeneous platforms. We evaluate representative academic and industrial systems for GPUs and NPUs, summarizing their protection strategies and current limitations. Finally, we highlight open research directions such as fine-grained protection of LLM runtime state, low-overhead cryptographic engines, and formal methods for building verifiable systems. Our goal is to support secure and efficient LLM execution through the integration of confidential computing as a foundational element in heterogeneous system design.

The growing demand for high-bandwidth and largecapacity memory access in data-intensive workloads has driven the development and deployment of Processing-in-Memory (PIM) architectures. However, existing DIMM-based PIM systems suffer from the severe communication bottleneck between the processing elements (PEs) near the PIM banks due to their requirement on host CPU forwarding. This bottleneck limits the efficiency of collective operations and degrades scalability and performance for workloads that require inter-PE communication. To address the communication limitation, we propose CoCoTree, a computation-capable architecture for collective communication in scalable DIMM-based PIM. CoCoTree supports direct and high-throughput inter-PE communication without host intervention. CoCoTree accelerates key collective communication using novel hierarchical binary tree topology and lightweight in-network computation support. We design and implement microarchitectures for the main building blocks: Co-Leaf and Co-Node, to efficiently handle the data packing, routing, and processing in CoCoTree. Furthermore, we also introduce a packet-based communication protocol tailored to the CoCoTree architecture, which decouples control and data through a two-phase configuration-computation communication mechanism to efficiently support a wide range of collective communication operations. CoCoTree effectively mitigates inter-PE communication bottlenecks, enabling scalable PIM systems capable of meeting the demands of growing data size. Experimental results show that CoCoTree achieves up to 95.6x improvement for collective operations and improves end-to-end application performance by up to 10.5x across various workloads over the baseline PIM, while outperforming state-of-the-art PIM communication architectures in both performance and scalability.

The design of XPU-PIM integrated Domain-Specific Accelerators relies heavily on high-performance simulators. However, existing simulation tools often suffer from limited integration capability and insufficient simulation speed. To address these challenges, this paper proposes HOPESim, a lightweight and fast simulator specifically designed for XPU-PIM accelerators. We introduce a Register Synchronous Simulation Model (RSSM), which abstracts multiple Processing Elements (PEs) and their interconnections into parallel blocks, enabling seamless integration with third-party simulators through a modular structure. A non-global timestamp strategy is proposed to break cycle dependencies between components, while a decoupled lightweight coroutine library leveraging modern C++ features is adopted to enhance large-scale simulation performance. Experimental results demonstrate that HOPESim achieves over 27× speedup compared to RTL simulation while maintaining compatibility with third-party simulation tools.

In modern cloud and heterogeneous distributed infrastructures, container images are widely used as the deployment unit for machine learning applications. An image bundles the application with its entire platform-specific execution environment and can be directly launched into a container instance. However, this approach forces developers to build and maintain separate images for each target deployment platform. This limitation is particularly evident for widely used interpreted languages such as Python and R in data analytics and machine learning, where application code is inherently cross-platform, yet the runtime dependencies are highly platform-specific. With emerging computing paradigms such as sky computing and edge computing, which demand seamless workload migration and cross-platform deployment, traditional images not only introduce inefficiencies in storage and network usage, but also impose substantial burdens on developers, who must repeatedly craft and manage platform-specific builds. To address these challenges, we propose a lazy-build approach that defers platform-specific construction to the deployment stage, thus keeping the image itself cross-platform. To enable this, we introduce a new image format, CIR (Container Intermediate Representation), together with its pre-builder and lazy-builder. CIR targets interpreted-language applications and only stores the identifiers of the application's direct dependencies, leaving platform adaptation to the lazy-builder, which at deployment time assembles the actual dependencies into runnable containers. A single CIR can therefore be deployed across heterogeneous platforms while reducing image size by 95

Molecular Dynamics (MD) simulations with first-principles accuracy are widely applied in various fields, including materials science and molecular pharmacology. Current research focus on reducing the solution time of ab initio molecular dynamics (AIMD) from both algorithmic and software perspectives. However, these optimizations are still far from meeting the ever-increasing performance demands for AIMD. In this paper, a fine-grained pipeline architecture is proposed to further optimize the solution time of MD. We design a high-utilization systolic line that eliminates the injection and evacuation time during each matrix operation. Besides, in conjunction with the MD dataflow, a computation migration strategy is introduced to reduce the storage overhead. Furthermore, we leverage dataflow rearrangement and preloading to eliminate the matrix transpose costs. The MD-pipe architecture has been implemented and verified using AMD VPK180 FPGA and ASIC synthesis tools. Evaluation results show that our work on FPGA and ASIC outperforms the NVIDIA A100 GPU by 2.97x and 23.77x respectively. Moreover, the ASIC implementation achieves a simulation speed of 67.6 mu s/ day, outperforming state-of-the-art work on Fugaku supercomputer by 454 times under extremely strong scaling deployment (one-atom-per-core).

It is widely believed that an open source hardware ecosystem can reduce development costs and lower barriers to innovation. However, developing an open source industrial-grade high-performance processor is a challenging undertaking. For mass adoption, such an IP needs to have an advanced out-of-order microarchitecture for high performance, a robust verification infrastructure for reliable quality, and high configurability to accommodate the myriad use cases. With a best-in-class performance, XiangShan is an open source project for RISC-V processors that fully meets these requirements. To maximize overall efficiency, XiangShan adopted a collaborative hardware development model, partnering with industry on processor design, implementation, and verification. With innovations in Agile development processes and tools, the design of the processors can be evolved, optimized, and verified quickly, ensuring high quality and enabling architectural innovation and rapid commercialization.

Deep neural networks (DNNs) increasingly rely on parallel structures to enhance performance and efficiency. However, existing machine learning compilers (MLCs) face challenges in optimizing these structures due to limited parallel fusion scopes and insufficient consideration of intra-operator information. This paper introduces Magneto, a novel framework designed to accelerate parallel structures in DNNs through the co-optimization of parallel operators. By expanding the scope of parallel operator fusion and introducing a dedicated co-tuning algorithm, Magneto unlocks new opportunities for co-optimization. Experimental results demonstrate that Magneto outperforms NVIDIA TensorRT and AMD MIGraphX, achieving speedups of 3.02x and 4.19x, respectively.

With the rapid development of machine learning and big data technologies, ensuring user privacy has become a pressing challenge. Secure multi-party computation offers a solution to this challenge by enabling privacy-preserving computations, but it also incurs significant performance overhead, thus limiting its further application. Our analysis reveals that the oblivious transfer protocol accounts for up to 96.64% of execution time. To address these challenges, we propose POTA, a high-performance pipelined OT hardware acceleration architecture supporting the silent OT protocol. Finally, we implement a POTA prototype on Xilinx VCU129 FPGAs. Experimental results demonstrate that under various network settings, POTA achieves significant speedups, with maximum improvements of 22.67x for OT efficiency and 192.57x for basic operations in MPC applications.

Processing-In-Memory (PIM) architectures alleviate the memory bottleneck in the decode phase of large language model (LLM) inference by performing operations like GEMV and Softmax in memory. However, the fragmented data layout in current PIM architectures limits end-to-end acceleration for long-context LLMs. In this paper, we propose BlockPIM, a cross-channel block memory layout strategy that maximizes memory utilization and eliminates the context length constraint. Additionally, we introduce a cross-channel attention computation scheme that is compatible with the current architecture to support distributed attention operations on BlockPIM. Experimental results demonstrate that our approach achieves a 62% average throughput increase compared to existing state-of-the-art PIM solutions, enabling efficient and scalable deployment of large language models on PIM architectures.

Coldstart introduces a significant latency penalty in cloud computing. While several previous works have proposed mechanisms such as warm start, fast snapshot recovery, lightweight isolation, and fast image download to avoid or mitigate this issue, image provisioning remains underexplored despite being critical. In this paper, we propose Poby, a software-hardware collaborative system that offloads and accelerates critical operations of image provisioning using SmartNICs. Specifically, Poby embodies a disaggregated architecture that offloads different image provisioning operations to the appropriate hardware such as embedded CPUs and domain-specific hardware accelerators for optimal performance. It uses a pipeline-based, data-driven workflow to eliminate delays caused by the serial execution of image provisioning operations. Moreover, it contains a distributed image provisioning scheme to alleviate the performance bottlenecks of conventional centralized registries. We implement the entire Poby system using Blue-Field SmartNICs and evaluate its performance using various microservice and FaaS benchmark suites. The results demonstrate that Poby outperforms two industry-standard container platforms, containerd and iSulad, with speedups of 13.2x and 8.0x, respectively. In addition, compared to iSulad, it reduces host CPU usage by 87.5%.

Recent advances in 3D Gaussian Splatting (3DGS) have demonstrated significant potential over traditional rendering techniques, attracting widespread attention from both industry and academia. However, real-time rendering with 3DGS remains a challenging problem, particularly in large-scale, high-resolution scenes due to the presence of numerous anisotropic Gaussian representations, and it has not been extensively explored. To address this challenge, we introduce FlashGS, an open-source CUDA library with Python bindings, featuring comprehensive algorithm design and optimizations, including redundancy elimination, adaptive scheduling, and efficient pipelining. First, we eliminate substantial redundant computations through precise Gaussian intersection tests, leveraging the intrinsic mechanism of the 3DGS rasterizer. During task partitioning, we propose an adaptive scheduling strategy that accounts for variations in Gaussian size and shape. Additionally, we design a multi-stage pipelining strategy for color computation in the rendering process, further accelerating performance. We conduct an extensive evaluation of FlashGS across a diverse range of synthetic and real-world 3D scenes, encompassing scene sizes of up to 2.7 km(2) cityscape and resolutions of up to over 4K. Our approach improves 3DGS rendering performance by an order of magnitude, achieving an average speedup of 7.2x, and rendering at a minimum of 125.9 FPS, setting a new state-of-the-art in real-time 3DGS rendering. https://github.com/InternLandMark/FlashGS.

The Text-to-Video (T2V) model aims to generate dynamic and expressive videos from textual prompts. The generation pipeline typically involves multiple modules, such as language encoder, Diffusion Transformer (DiT), and Variational Autoencoders (VAE). Existing serving systems often rely on monolithic model deployment, while overlooking the distinct characteristics of each module, leading to inefficient GPU utilization. In addition, DiT exhibits varying performance gains across different resolutions and degrees of parallelism, and significant optimization potential remains unexplored. To address these problems, we present DDiT, a flexible system that integrates both inter-phase and intra-phase optimizations. DDiT focuses on two key metrics: optimal degree of parallelism, which prevents excessive parallelism for specific resolutions, and starvation time, which quantifies the sacrifice of each request. To this end, DDiT introduces a decoupled control mechanism to minimize the computational inefficiency caused by imbalances in the degree of parallelism between the DiT and VAE phases. It also designs a greedy resource allocation algorithm with a novel scheduling mechanism that operates at the single-step granularity, enabling dynamic and timely resource scaling. Our evaluation on the T5 encoder, OpenSora SDDiT, and OpenSora VAE models across diverse datasets reveals that DDiT significantly outperforms state-of-the-art baselines by up to 1.44x in p99 latency and 1.43x in average latency.

Time series analysis (TSA) is an important technique for extracting information from data in various domains. TSA is memory-bound on conventional platforms due to excessive off-chip data movement between processing units and the main memory of the system. Near data processing (NDP) has emerged as a promising solution to alleviate the bottleneck of memory access for data-intensive applications by enabling processing to be performed near the data within memory. In this work, we propose and implement upTSA, a parallel near data processing accelerator on real-world commercial DRAM Dual-Inline Memory Module (DIMM) NDP hardware. Our solution offers high bandwidth with large memory capacity and multi-level parallelism to accelerate TSA. We begin with a detailed characterization of TSA on conventional CPUs. We then design and implement a multi-level parallel accelerator on the real-world DIMM-based NDP hardware UPMEM. We further explore the hardware enhancements to improve the computational capability of current DIMM NDP hardware. Experimental results show that upTSA improves performance by 2.1 × compared to the server-class CPU baseline. The enhanced DIMM NDP architecture achieves up to 3.3 × speedup on TSA compared with current commercial NDP hardware.

Sparse matrix-vector semiring computation is a key operation in sparse matrix computations, with performance strongly dependent on both program design and the features of the sparse matrices. Given the diversity of sparse matrices, designing a tailored program for each matrix is challenging. To address this, we propose SRSparse1 an program generator that creates tailored programs by automatically combining program designing methods to fit specific input matrices. It provides two components: the problem definition configuration, which declares the computation, and the scheduling language, which can be leveraged by an auto-tuner to specify the program designs. The two are lowered to the intermediate representations of SRSparse, the Format IR and Kernel IR, which respectively generates format conversion routine and kernel code. We evaluate SRSparse on four representative sparse kernels and three format conversion routines. For sparse kernels, SRSparse achieves median speedups over handwritten programs: COO (3.50 ×), CSR-Adaptive (5.36 ×), CSR5 (2.06 ×), ELL (1.63 ×), Gunrock (1.57 ×), and GraphBLAST (1.96 ×); over an auto-tuner: AlphaSparse (1.16 ×); and over a compiler: TACO (1.71 ×). For format conversion routines, SRSparse achieves median speedups over handwritten implementations: Intel MKL (7.60 ×), SPARSKIT (2.61 ×), CUSP (2.77 ×), and Ginkgo (1.74 ×); and over a compiler: TACO (4.04 ×).

Parallel structures have become a key pattern in deep neural networks (DNNs), offering improved efficiency and scalability. However, existing machine learning compilers (MLCs) face challenges in optimizing these structures due to limited parallel fusion scope and insufficient analysis of intra-operator characteristics. This paper introduces Magneto, a framework designed to accelerate DNN inference by co-optimizing parallel operators. Magneto broadens the fusion scope and incorporates a specialized co-tuning algorithm to optimize operators jointly. Our approach addresses the unique challenges inherent in optimizing parallel structures, enabling significant performance improvements across various hardware platforms. Experimental results show that Magneto outperforms state-of-the-art NVIDIA TensorRT and AMD MIGraphX, achieving geometric mean speedups of 2.27 × and 2.88 ×, respectively.

Inter-operator optimization in deep neural networks (DNNs) relies on accurate data dependency analysis. Traditional machine learning compilers (MLCs) perform static data dependency analysis at the element and operator levels, leading to two key limitations: complex dependencies that hinder efficient inter-operator optimizations, and overlooked parallelizable computations that underutilize GPU resources. We introduce BlockDepend, a novel MLC framework that addresses these issues through block-level dependency analysis. By examining the lower-level phases of compilation, BlockDepend extracts crucial block-level dependency information, simplifying complex relationships between operators and uncovering hidden parallelization opportunities. This allows for targeted optimization strategies that enhance memory access efficiency and improve GPU utilization. Our experiments demonstrate BlockDepend's effectiveness, achieving speedups of 1.71x and 2.88x compared to NVIDIA TensorRT and AMD MIGraphX, respectively, across various workloads.