Sparse-dense Matrix Multiplication (SpMM) is a fundamental operation in various fields. Recently, leveraging Tensor Cores (TCs) on GPUs to accelerate SpMM has attracted growing interest. Given the block-wise computation nature of TCs, existing TC-based SpMM methods typically apply preprocessing techniques, such as similarity-based row reordering, to cluster similar rows and improve data locality, with the goal of forming denser matrix blocks that better utilize TCs and enhance SpMM performance. However, these methods typically apply TCs only at the final computation stage, without fully accounting for the block-wise execution characteristic during the reordering process. As a result, the clustering outcomes often misalign with the actual computation pattern of TCs, limiting potential performance gains. To this end, we propose BlockSmith , an end-to-end framework for accelerating SpMM by leveraging TCs. The name BlockSmith reflects its core design principle: to forge high-density matrix blocks from sparse matrices. The key innovation lies in the Ultra-dense Block Transformation (UBT) approach, which significantly improves the data locality of sparse matrices by integrating a novel row-reordering algorithm and column compression technique, while also taking into account the block-wise characteristics of TCs. Moreover, we design a novel storage format to maximize memory access efficiency, together with an optimized GPU kernel that leverages low-level PTX instructions to fully exploit the capabilities of TCs. Extensive evaluations on modern GPUs, including RTX 4090 and RTX 5090, show that BlockSmith outperforms the state-of-the-art SpMM method Acc-SpMM, achieving average speedups of 1.89× and 4.35×, respectively, with the maximum speedup exceeding 100×. Moreover, for GNN inference tasks, BlockSmith-GCN achieves over 10× speedup compared with mainstream frameworks such as PyG and DGL. The source code of BlockSmith is publicly available at: https://github.com/CGCL-codes/BlockSmith.git .

Mining temporal motifs in temporal graphs is essential for many critical applications. Although several solutions have been proposed to handle temporal motif mining, they still suffer from substantial inefficiencies due to significant redundant graph traversals and fragmented memory access, both caused by irregular search tree expansions across different motif matching tasks. In this work, we observe that data accesses issued by these tasks exhibit strong spatial similarity and temporal monotonicity. Based on these observations, this paper proposes an efficient data-centric temporal motifmining system DTMiner, which introduces a novel LoadExplore-Synchronize (LES) execution model to efficiently regularize data accesses to the common temporal graph data among different tasks. Specifically, DTMiner enables the temporal graph chunks to be sequentially loaded into the cache in temporal order and then triggers all relevant tasks to explore only these loaded data for search tree expansions in a fine-grained synchronization mechanism. In this way, different tasks can share the graph traversal corresponding to the same chunks, while fragmented memory accesses are restricted to the graph data residing in the cache, significantly reducing data access overhead. Experimental results demonstrate that DTMiner achieves 1.14x-11.98x performance improvement in comparison with the state-of-the-art temporal motif mining solutions.

Front-end development constitutes a substantial portion of software engineering, yet converting design mockups into production-ready *User Interface* (UI) code remains tedious and time-costly. While recent work has explored automating this process with *Multimodal Large Language Models* (MLLMs), existing approaches typically rely solely on design images. As a result, they must infer complex UI details from images alone, often leading to degraded results. In real-world development workflows, however, design mockups are usually delivered as Figma files—a widely used tool for front-end design—that embed rich multimodal information (e.g., metadata and assets) essential for generating high-quality UI. To bridge this gap, we introduce Figma2Code, a new task that generalizes *design-to-code* into a multimodal setting and aims to automate *design-to-code* in the wild. Specifically, we collect paired design images and their corresponding metadata files from the Figma community. We then apply a series of processing operations, including rule-based filtering, human and MLLM-based annotation and screening, and metadata refinement. This process yields 3,055 samples, from which designers curate a balanced dataset of 213 high-quality cases. Using this dataset, we benchmark ten state-of-the-art open-source and proprietary MLLMs. Our results show that while proprietary models achieve superior visual fidelity, they remain limited in layout responsiveness and code maintainability. Further experiments across modalities and ablation studies corroborate this limitation, partly due to models’ tendency to directly map primitive visual attributes from Figma metadata.

The acceleration of Sparse-dense Matrix Multiplication (SpMM) using Tensor Cores (TCs) in GPUs has recently garnered significant attention. TCs are designed for block-wise matrix multiplication, however, block partitioning of general unstructured sparse matrices often results in low-level density, causing a substantial waste of computational resources. Sparse Tensor Cores (SpTCs) can mitigate this issue by skipping 50% of zero values, however, SpTCs are limited to strict 2:4 or 1:2 structured sparsity. To bridge this gap, we propose MP-SpMM, a novel Matching and Padding approach that transforms general sparse matrices into structured sparsity, drawing inspiration from the maximum matching problem in graph theory. Moreover, we introduce a novel storage format and a highly optimized GPU kernel that fully exploits the capabilities of SpTCs. Extensive experiments on modern GPUs demonstrate that MP-SpMM outperforms state-of-the-art SpMM libraries, DTC-SpMM and RoDe, with an average speedup of 2.42x (up to 7.65x) and 1.92x (up to 8.60x).

Graph Neural Networks (GNNs) have achieved remarkable successes in various graph-based learning tasks, thanks to their ability to leverage advanced GPUs. However, GNNs currently face challenges arising from the concurrent use of advanced Tensor Cores (TCs) and CUDA Cores (CDs) in GPUs. These challenges are further exacerbated due to repeated, inefficient, and redundant aggregations in GNN that result from the high sparsity and irregular non-zero distribution of real-world graphs. We propose RT-GNN, a GNN framework based on the fusion of advanced TC and CD units, to eliminate the aforementioned redundancies by exploiting the properties of an adjacency matrix. First, a novel GNN representation technique, hierarchical embedding graph (HEG) is proposed to manage the intermediate aggregation results hierarchically, which can further avoid redundancy in intermediate aggregations elegantly. Next, to address the inherent sparsity of graphs, RT-GNN places the blocks (a.k.a. tiles) in HEG onto TCs and CDs according to their sparsity by a new block-based row-wise multiplication approach, which assembles TCs and CDs to work concurrently. Experimental results demonstrate that HEG outperforms HAG by an average speedup of 19.3x for redundancy elimination performance, especially up to 72x speedup on the dataset of ARXIV. Moreover, for overall performance, RTGNN outperforms state-of-the-art GNN frameworks (including DGL, HAG, GNNAdvisor, and TC-GNN) by an average factor of 3.1x while maintaining or even improving the task accuracy.

Automating the synthesis of User Interfaces (UIs) plays a crucial role in enhancing productivity and accelerating the development lifecycle, reducing both development time and manual effort. Recently, the rapid development of Multimodal Large Language Models (MLLMs) has made it possible to generate front-end Hypertext Markup Language (HTML) code directly from webpage designs. However, real-world webpages encompass not only a diverse array of HTML tags but also complex stylesheets, resulting in significantly lengthy code. The lengthy code poses challenges for the performance and efficiency of MLLMs, especially in capturing the structural information of UI designs. To address these challenges, this paper proposes UICopilot, a novel approach to automating UI synthesis via hierarchical code generation from webpage designs. The core idea of UICopilot is to decompose the generation process into two stages: first, generating the coarse-grained HTML hierarchical structure, followed by the generation of fine-grained code. To validate the effectiveness of UICopilot, we conduct experiments on a real-world dataset, i.e., WebCode2M. Experimental results demonstrate that UICopilot significantly outperforms existing baselines in both automatic evaluation metrics and human evaluations. Specifically, statistical analysis reveals that the majority of human annotators prefer the webpages generated by UICopilot over those produced by GPT-4V.

This paper explores the potential and trends of novel dataflow architectures in multi-domain integrated computing.Traditional von Neumann and domain-specific architectures struggle to meet the high performance and flexibility demands of emerging technologies like artificial intelligence,graph computing,and big data.We review current dataflow chip design methods,discussing their implementations based on specialization vs.generalization and execution granularity.Based on this,we propose a dataflow abstract machine model using concurrent code blocks,with a complete instruction set and microarchitecture.This model achieves unified intermediate representation across domains and integrates multiple operator fusion strategies,enhancing efficiency in tasks such as graph neural networks,large model computations,and real-time signal processing.Experimental results show that our processor outperforms existing general-purpose architectures in performance and power consumption.We conclude by highlighting the broad application prospects of dataflow architectures in future computing systems and their significant impact on efficient computing.

In recent years, Neural Architecture Search (NAS) has emerged as a promising approach for automatically discovering superior model architectures for deep Graph Neural Networks (GNNs). Different methods have paid attention to different types of search spaces. However, due to the time-consuming nature of training deep GNNs, existing NAS methods often fail to explore diverse search spaces sufficiently, which constrains their effectiveness. To crack this hard nut, we propose CAS-DGNN, a novel comprehensive architecture search method for deep GNNs. It encompasses four kinds of search spaces that are the composition of aggregate and update operators, different types of aggregate operators, residual connections, and hyper-parameters. To meet the needs of such a complex situation, a phased and hybrid search strategy is proposed to accommodate the diverse characteristics of different search spaces. Specifically, we divide the search process into four phases, utilizing evolutionary algorithms and Bayesian optimization. Meanwhile, we design two distinct search methods for residual connections (All-connected search and Initial Residual search) to streamline the search space, which enhances the scalability of CAS-DGNN. The experimental results show that CAS-DGNN achieves higher accuracy with competitive search costs across ten public datasets compared to existing methods.

Sparse-Dense Matrix Multiplication (SpMM) is a fundamental computational operation in various domains, and leveraging Tensor cores or CUDA cores on GPU to accelerate SpMM has become common practice. While Tensor cores offer notable advantages in dense matrix multiplication, their efficiency significantly decreases when handling sparse matrices. Besides, the computational power of CUDA cores should not be overlooked. Nevertheless, the differences in supported data formats present challenges in effectively leveraging both types of cores to accelerate SpMM. To this end, we propose BRP-SpMM, a block and row partition approach designed for efficient SpMM on GPU. BRP-SpMM partitions the sparse matrix into two parts: TC Block and Residual Row part, which are computed on Tensor cores and CUDA cores separately. Meanwhile, a customized storage format is proposed to manage these two distinct parts. Our two GPU kernels incorporate several advanced techniques including load balance, register remapping, and 1-D tiling. BRP-SpMM can achieve higher memory access efficiency and more rational computing resource utilization of GPU. Extensive experiments on modern NVIDIA A800 GPU show that BRP-SpMM outperforms SOTA libraries by up to 2.9x (on average 2.1x). Furthermore, BRP-SpMM accelerates end-to-end GNN training by up to 1.9x compared to popular frameworks.

Converting webpage designs into code (design-to-code) plays a vital role in User Interface (UI) development for front-end developers, bridging the gap between visual design and functional implementation. While recent Multimodal Large Language Models (MLLMs) have shown significant potential in design-to-code tasks, they often fail to accurately preserve the layout during code generation. To this end, we draw inspiration from the Chain-of-Thought (CoT) reasoning in human cognition and propose LaTCoder, a novel approach that enhances layout preservation in webpage design during code generation with Layout-as-Thought (LaT). Specifically, we first introduce a simple yet efficient algorithm to divide the webpage design into image blocks. Next, we prompt MLLMs using a CoT-based approach to generate code for each block. Finally, we apply two assembly strategies-absolute positioning and an MLLM-based method-followed by dynamic selection to determine the optimal output. We evaluate the effectiveness of LaTCoder using multiple backbone MLLMs (i.e., DeepSeek-VL2, Gemini, and GPT-4o) on both a public benchmark and a newly introduced, more challenging benchmark (CC-HARD) that features complex layouts. The experimental results on automatic metrics demonstrate significant improvements. Specifically, TreeBLEU scores increased by 66.67% and MAE decreased by 38% when using DeepSeek-VL2, compared to direct prompting. Moreover, the human preference evaluation results indicate that annotators favor the webpages generated by LaTCoder in over 60% of cases, providing strong evidence of the effectiveness of our method.

Existing reinforcement learning strategies based on outcome supervision have proven effective in enhancing the performance of large language models(LLMs) for code generation. While reinforcement learning based on process supervision has shown great promise in handling multi-step reasoning tasks, its effectiveness in code generation remains largely underexplored and underjustified. The primary obstacle stems from the resource-intensive nature of constructing high-quality process-supervised data, which demands substantial human expertise and computational resources. In response to this challenge, we propose a "statement mutation/refactoring-compile and execution verification" strategy: mutating and refactoring code line-by-line through a teacher model, and utilizing compiler execution results to automatically label each line, resulting in line-by-line process-supervised data, which is pivotal for training a process-supervised reward model. The trained reward model is then integrated into the PRLCoder framework, followed by experimental validation on several benchmarks. Experimental results demonstrate that process-supervised reinforcement learning significantly surpasses methods relying solely on outcome supervision. Notably, in tackling complex code generation tasks, process-supervised reinforcement learning shows a clear advantage, ensuring both the integrity of the code generation process and the correctness of the generation results.

Graph neural networks (GNNs) have been widely applied in graph-based learning tasks. However, the feature collection stage is increasingly becoming the bottleneck of the existing GNN systems when training large-scale graphs. To solve this problem, this paper proposes FCGraph, a novel efficient GNN training strategy by combining a feature cache policy of multi-hop (k-hop) neighbors and a hierarchical GPU-centric data access method. First, a k-hop neighbors based cache policy is proposed by exploring vertex data access characteristics in k-layer GNN models, which focuses on reducing CPU-GPU data transfer overhead. Second, based on the cache policy, node features are partitioned according to node access frequency, and a novel GPU-centric data access method is presented for feature collection. Further, FCGraph is scaled to multi-GPU systems equipped with NVLink, in which the memory access hierarchies are explored. The evaluation of some representative datasets shows that FCGraph can improve end-to-end training performance over the advanced GNN training systems of DGL and PyTorch-Direct by 3.72 × and 1.46 × on average, respectively.

Automatically generating webpage code from webpage designs can significantly reduce the workload of front-end developers, and recent Multimodal Large Language Models (MLLMs) have shown promising potential in this area. However, our investigation reveals that most existing MLLMs are constrained by the absence of high-quality, large-scale, real-world datasets, resulting in inadequate performance in automated webpage code generation. To fill this gap, this paper introduces WebCode2M, a new dataset comprising 2.56 million instances, each containing a design image along with the corresponding webpage code and layout details. Sourced from real-world web resources, WebCode2M offers a rich and valuable dataset for webpage code generation across a variety of applications. The dataset quality is ensured by a scoring model that filters out instances with aesthetic deficiencies or other incomplete elements. To validate the effectiveness of WebCode2M, we introduce a baseline model based on the Vision Transformer (ViT), named WebCoder, and establish a benchmark for fair comparison. Additionally, we introduce a new metric, TreeBLEU, to measure the structural hierarchy recall. The benchmarking results demonstrate that our dataset significantly improves the ability of MLLMs to generate code from webpage designs, confirming its effectiveness and usability for future applications in front-end design tools. Finally, we highlight several practical challenges introduced by our dataset, calling for further research. The code and dataset are publicly available at our project homepage: https://webcode2m.github.io.

Graph convolutional networks (GCNs) are promising to enable machine learning on graph data. GCNs show potential vertex-level and intra-vertex parallelism for GPU acceleration, but their irregular memory accesses arising in aggregation operations and the inherent sparsity for vertex features of graphs cause inefficiencies on the GPU. In this paper, we present gPIM, which aims to accelerate GCNs inference through a processing-in-memory (PIM) enabled architecture. gPIM is expected to perform compute-intensive combination on the GPU while aggregation and memory-bound combination are offloaded to the PIM-featured hybrid memory cubes (HMCs). To maximize the efficiency of such GPU-HMC architecture, gPIM is novel with two key designs: 1) A GCN-induced graph partitioning that minimizes communication overheads between cubes, 2) A programmer-transparent performance estimation mechanism that predicts the performance bound of operations accurately for workload offloading. Experimental results show that gPIM significantly outperforms Intel Xeon E5-2680v3 CPU (8,979.52×), NVIDIA Tesla V100 GPU (96.01×), and a state-of-the-art GCN accelerator AWB-GCN (4.18×).

Network pruning is an efficient approach to adapting large-scale deep neural networks (DNNs) to resource-constrained systems; the networks are pruned using the predefined pruning criteria or a flexible network structure is explored with the help of neural architecture search, (NAS). However, the former crucially relies on the human expert knowledge, while the latter usually requires one to make many simplifications to ensure the efficiency of the search, resulting in limited performance. This paper presents a new pruning approach called Progressive Differentiable Architecture Search (PDAS) that realizes a better balance between computation efficiency and model performance. First, a joint search-update scheme for search optimization is presented; it constantly refines the candidate number of channels in each layer by performing differentiable searching and evolutionary updating alternately. The latter can provide new high-probability candidates continuously to avoid local minimum point. Second, a two-stage constrained progressive search strategy is presented for some complex nonlinear networks (such as ResNet) that are more difficult to prune for existing approaches; it effectively avoids the over-fitting problem caused by the excessive search space and largely reduces the consumption of 1x1 convolution in the skip connections of the residual blocks with little accuracy loss. Extensive experiments on some representative datasets (such as CIFAR-10, CIFAR-100, and ImageNet) approve the superior performance of PDAS compared to most existing network pruning algorithms available. Notably, compared to the state-of-the-art LFPC, PDAS can even prune about 8% more FLOPs on ResNet-110 (on CIFAR-10) and ResNet-50 (on ImageNet), respectively, while coming with almost identical and ignorable accuracy losses.

Benefitting from the combination of the idea of pipeline with model parallelism and data parallelism, pipeline parallelism improves the efficiency of distributed deep learning systems significantly. However, suffering from the bubbles and gaps caused by the imbalance of different stages in pipeline, it can not output ideal performance yet. To explore the potential of pipeline parallelism further, we propose a novel approach called PipePrune, which adds a convolutional layer pruning strategy to pipeline creatively to reduce the bubbles and gaps. For the convolutional layers with heavy overheads, some unimportant kernels are pruned by the measurement of the L1-norm. This approach makes the processing overheads of different pipeline stages more balanced. The experimental results show that, compared with state-of-the-art pipeline methods, PipePrune can improve the training speeds obviously (e.g. for ResNet50 on ImageNet, about 30% speed improvement is realized with only 1.1% loss of training accuracies).

Deep learning has achieved great success in various areas and its success is closely linked to the availability of massive data. But in general, a large dataset could include sensitive data and therefore the model should have the capability to avoid privacy leakage. To achieve this aim, many works apply the famous privacy framework named differential privacy into deep learning to preserve privacy. In this article, we propose a novel perturbed iterative gradient descent optimization (PIGDO) algorithm and prove that this algorithm satisfies the differential privacy. Besides, we propose a modified moments accountant (MMA) method to conduct the privacy analysis and obtain a tighter bound of privacy loss compared with the original moments accountant method. A number of experiments demonstrate that our optimization algorithm can not only improve the model accuracy and training speed, but also achieve better privacy guarantees over the state-of-the-art algorithm while reaching the equivalent accuracy. We provide codes for all of our experiments in https://github.com/CGCL-codes/DPDLIGDO.git .

Existing FPGA-based graph accelerators, typically designed for static graphs, rarely handle dynamic graphs that often involve substantial graph updates (e.g., edge/node insertion and deletion) over time. In this paper, we aim to fill this gap. The key innovation of this work is to build an FPGA-based dynamic graph accelerator easily from any off-the-shelf static graph accelerator with minimal hardware engineering efforts (rather than from scratch). We observe \em spatial similarity of dynamic graph updates in the sense that most of graph updates get involved with only a small fraction of vertices. We therefore propose an FPGA library, called GraSU, to exploit spatial similarity for fast graph updates. GraSU uses a differential data management, which retains the high-value data (that will be frequently accessed) in the specialized on-chip UltraRAM while the overwhelming majority of low-value ones reside in the off-chip memory. Thus, GraSU can transform most of off-chip communications arising in dynamic graph updates into fast on-chip memory accesses. Our experiences show that GraSU can be easily integrated into existing state-of-the-art static graph accelerators with only 11 lines of code modifications. Our implementation atop AccuGraph using a Xilinx Alveo#8482; \ U250 board outperforms two state-of-the-art CPU-based dynamic graph systems, Stinger and Aspen, by an average of 34.24× and 4.42× in terms of update throughput, improving further overall efficiency by 9.80× and 3.07× on average.

With the rapid growth of real-world graphs, the size of which can easily exceed the on-chip (board) storage capacity of an accelerator, processing large-scale graphs on a single Field Programmable Gate Array (FPGA) becomes difficult. The multi-FPGA acceleration is of great necessity and importance. Many cloud providers (e.g., Amazon, Microsoft, and Baidu) now expose FPGAs to users in their data centers, providing opportunities to accelerate large-scale graph processing. In this paper, we present a communication library, called FDGLib, which can easily scale out any existing single FPGA-based graph accelerator to a distributed version in a data center, with minimal hardware engineering efforts. FDGLib provides six APIs that can be easily used and integrated into any FPGA-based graph accelerator with only a few lines of code modifications. Considering the torus-based FPGA interconnection in data centers, FDGLib also improves communication efficiency using simple yet effective torus-friendly graph partition and placement schemes. We interface FDGLib into AccuGraph, a state-of-the-art graph accelerator. Our results on a 32-node Microsoft Catapult-like data center show that the distributed AccuGraph can be 2.32x and 4.77x faster than a state-of-the-art distributed FPGA-based graph accelerator ForeGraph and a distributed CPU-based graph system Gemini, with better scalability.