Numerical simulation is important in many engineering domains such as aerodynamics and structural mechanics. Researchers have developed message passing (MP)-based multilevel mesh graph networks (MGNs) for numerical simulation with fast prediction time and acceptable accuracy. Unfortunately, when the simulation (e.g., the numerical simulation of air pressure on a car surface within a wind tunnel) involves the complex interactions between simulation objects (car surface) and external conditions (the initial air velocity), existing MP schemes in MGNs face effectiveness and efficiency issues in modeling such simulations. In this paper, we propose a scalable MGN model with a novel edge-centric message passing scheme on large mesh graphs, namely ${\sf E-MGN}$. Firstly, the edge-centric MP scheme (resp. coarse-mesh to fine-mesh refinement) can effectively capture local (resp. global) interactions. Next, for high efficiency, ${\sf E-MGN}$ performs the MP in parallel on distributed mesh partitions with low communication overhead. Evaluation on three datasets (two synthetic and one real) against previous baseline algorithms demonstrates the superiority of ${\sf E-MGN}$. For example, on the Beam dataset, ${\sf E-MGN}$ leads to 6.95% smaller RMSE than the state-of-the-art and leads to 6.93× speedup on 8 GPUs.
Large language models (LLMs) have demonstrated exceptional performance across a variety of online services, which require real-time responses with low time-to-first-token (TTFT). To reduce TTFT, existing works commonly reuse the KV cache for requests sharing identical prefixes and leverage remote KV caches to increase the reuse rate. Existing remote KV-cache reuse policies, which we call resource-insensitive reuse policies, assume the network is stable and ignore real-time resource fluctuations. However, such policies cannot maintain low TTFT, as they neither make dynamic selections among different reuse representations nor leverage hybrid reuse to increase resource utilization, leading to significant waste of compute and transfer resources. We propose that an efficient LLM inference system should adopt a resource-efficient KV-cache reuse policy to reduce TTFT. To do so, we design an LLM inference system, FlexKV, which decreases TTFT while achieving high resource utilization by integrating two novel components: an adaptive Hybrid Adapter that chooses which reuse approaches to utilize, and a layer-wise Pipeline Scheduler that coordinates multiple compute and transfer operations. Evaluation across 14 unique model–workload combinations spanning 6.7B to 65B models, including a multi-node LLaMA-65B-GPTQ deployment, shows that FlexKV achieves up to a 4.90× TTFT speedup over evaluated deployable baselines. Compared with the analytical Optimal reference, FlexKV has a median TTFT gap of 8.10%.
Feature selection plays a pivotal role in both machine learning and data analysis. The primary goal is to improve classification accuracy and reduce computational costs by identifying a pertinent feature subset. Many feature selection approaches exhibit robust performance on small or medium datasets, but their scalability and efficiency for more complex datasets are limited due to the serial methodology and the single-processor environment. Motivated by this, we propose a distributed feature selection method based on granular computing technique. First, an adaptive information granulation method is applied to original data for the extraction of granulation features. Then, the granular computing-based feature selection approach is established with a relevance and redundancy criterion. Furthermore, the distributed feature selection method is developed in the Map-Reduce architecture for distributed computing environments. The relevant experiments illustrate the feasibility, effectiveness and distributed capability on several benchmark datasets for feature selection tasks.
Multitask model merging is essential for enabling a single model to perform multiple tasks simultaneously, ad dressing scalability and efficiency challenges in machine learning systems. While existing methods enhance the performance of traditional approaches like Task Arithmetic, they require substantial GPU memory. For example, AdaMerging enhances performance by autonomously learning merging coefficients through entropy minimization on unlabeled test samples. How ever, it necessitates loading all K models into GPU memory, leading to substantial memory overhead as the number and size of models increase. To overcome these limitations and achieve both high performance and low GPU memory usage, we introduce G-Merging, a decentralized multitask model merging approach. G-Merging distributes models across multiple de vices arranged in a ring topology and performs merging by combining each model only with its adjacent neighbors. Through iterative merging processes, all models converge into a unified multitask learning (MTL) model. Importantly, our method ensures that each device stores no more than four models at any time, regardless of the total number of models being merged. This memory reduction is obtained by trading additional local merging rounds and runtime for a lower per-device GPU memory requirement. Experimental results demonstrate that G-Merging requires about 20% of the post-cleanup resident GPU memory of AdaMerging without compromising accuracy when merging 20 CLIP-ViT-B/32 models, reducing the resident memory cost from 8.2GB to 1.7GB.
High-quality image rendering places significant demands on hardware performance and algorithm implementation. Among the various rendering methods, path tracing is well known for its high accuracy and rendering fidelity, while the heterogeneous many-core architecture of the Sunway supercomputer can provide powerful computational capabilities. In this paper, we propose a novel approach for efficient execution of physically based rendering algorithms on Sunway's many-core platform, specifically targeting task-scheduling strategies and overcoming the algorithmic performance constraints of path tracing. Our optimizations include: 1) a wavefront decomposition approach for complex path tracing processes with multi-layer memory management, 2) effective ray queue structure design and ray packet grouping to mitigate the instruction cache miss issue, and 3) efficient task scheduling for load balancing and profile-guided compilation auto-tuning on the Sunway platform. The experimental results demonstrate the effectiveness of our optimization: our work reaches more than 40x speedup over the baseline version on Sunway, and a peak 1.50x speedup compared with previous deeply optimized Sunway implementations, reducing the instruction cache miss rate by up to 72.7%.
