In this Letter, we introduce a scalable and physics-informed (PI) computational implementation for determinant-based multireference (MR) calculations. The PI philosophy is deeply embedded throughout the methodology and its implementation. At the method level, the orbital entanglement is used to guide an optimized reconstruction of selected CI wave functions, while the MR stage employs a PI kernel optimization (PIKO) strategy that explicitly accounts for configuration grouping patterns and memory access irregularities of integrals. In system hardware codesign, the PI parallel optimization (PIPO) strategy is proposed, with which the most computationally intensive MR module is accelerated via an entropy-based performance model and a hierarchical load-balancing scheme; both of them exploit physical insights into workload irregularity and data locality. This consistent PI-driven approach enables remarkable heterogeneous computing efficiency and parallel scalability. Single-GPU accelerations reach ∼460 times those of full CPU cores, and the strong scaling efficiency exceeds 92.5% on up to 4000 GPUs using HPC clusters. Furthermore, scalar relativistic effects can be incorporated consistently, and spin-orbit coupling (SOC) effects can be included through the present SO treatment.

With the increasing number of computationally intensive applications, heterogeneous systems have become an important solution for improving computing performance. In order to effectively develop and optimize parallel programs running on these systems, performance prediction has become an indispensable part. This article aims to comprehensively review the methods and tools for predicting parallel program performance in heterogeneous systems, analyze the characteristics of existing technologies, explore their development trends, and provide valuable references and guidance for researchers and developers. This article adopts a systematic review method, first sorting out the research process of parallel program performance prediction in heterogeneous systems, and then classifying and summarizing the current mainstream performance prediction methods, including analysis model-based prediction, simulation-based prediction, and machine learning based prediction. This article also summarizes the tools and platforms used to predict parallel program performance in heterogeneous systems. Through review, it was found that various performance prediction methods and tools have their own advantages in feature richness, availability, and accuracy, but they have all improved the efficiency and accuracy of parallel program performance prediction to a certain extent. The review of this article indicates that despite various methods and tools available for performance prediction, there are still many challenges and unresolved issues. Future research should further explore more accurate, efficient and intelligent prediction methods to better support the development and optimization of parallel programs in heterogeneous systems.

Ocean General Circulation Model (OGCM) is a computer simulation tool based on physical principles and mathematical equations, aiming at studying global ocean circulation and its impact on the climate system. Its diverse resolutions and large computational data make parallel optimization a research hotspot. In the present study, we redesign the OGCM under a unified dynamic framework of serial versions (based on an unstructured grid) for the first time with a MPI parallel algorithm. Specifically, our work mainly consists of the following two parts: (1) we first change the model from a raw serial program to a parallel program that supports communication between multiple CPU cores. (2) In response to the problem of a sharp decrease in parallel efficiency as the number of CPU cores increases in the parallel optimized mode, we optimize the grid coding of the original model and proposed a new boundary communication scheme. We conduct single-node and multi-node experiments on a multi-CPU cluster to evaluate the proposed approach. Using the original OGCM—initially limited to serial execution—as a baseline, we achieve a 175.91× speedup on 640 cores. Afterwards, we extend the parallel version to multiple nodes, with strong scaling parallel efficiency of 42.2

Approximate computing is emerging as a promising paradigm of High-Performance Computing (HPC) to increase application performance, with mixed-precision computing standing out as a popular approach within this domain. However, mixed precision tuning is challenging due to the vast search space of mixed precision configurations. The floating-point error sensitivity profile is a technique to quantify the error contribution of each reduced precision operation to the final output error, which can dramatically reduce the search space of mixed precision tuning of applications. To control time complexity, the error sensitivity profile is currently built upon reverse-mode automatic differentiation; however, its high memory demands render this approach impractical for large HPC workloads. This paper proposes a novel error analysis algorithm with much lower space complexity, by leveraging forward-mode automatic differentiation combined with just-in-time error source aggregation. Furthermore, a new error model is introduced to capture dependencies among assignments within the same error source group, addressing a key limitation of existing techniques that assume floating-point assignments to be independent of each other, which is an assumption sufficient only for smaller-scale cases. We implement this approach within the LLVM compiler infrastructure, as an easy-to-use source-code transformation tool, Light-FP. Evaluations conducted on applications from the HPC-MixPBench benchmark suite demonstrate Light-FP’s superiority over the state-of-the-art tool in terms of accuracy, runtime, and memory efficiency.

The sequential task flow (STF) model introduces implicit data dependences to exploit task-based parallelism, simplifying programming but also introducing non-negligible runtime overhead. On emerging cache-less, explicit inter-core message passing (EMP) architectures, the long latency of memory access further amplifies the runtime overhead of the traditional STF model, resulting in unsatisfactory performance.This paper addresses two main components in the STF tasking runtime. We uncover abundant concurrency in the task dependence graph (TDG) building process through three sufficient conditions, put forward PBH, a parallelized TDG building algorithm with helpers which mixes pipeline parallelism and data parallelism to overcome the TDG building bottleneck for fine-grained tasks. We also introduce a centralized, lock-less task scheduler, EMP-C, based on the EMP interface, and propose three optimizations. These two techniques are implemented and evaluated on a product processor with EMP support, i.e. SW26010. Experimental results show that compared to traditional techniques, PBH achieves an average speedup of 1.55 for fine-grained task workloads, and the EMP-C scheduler brings speedups as high as 1.52 and 2.38 for fine-grained and coarse-grained task workloads, respectively. And the combination of these two techniques significantly improves the granularity scalability of the runtime, reducing the minimum effective task granularity (METG) to 0.1 ms and achieving an order of magnitude decrease in some cases.

The Sequential Task Flow (STF) model guides task parallelism by dynamically analyzing data dependencies at runtime, making it well-suited to handle dynamic and irregular parallelism. However, it introduces additional dependency tracking overhead. As task granularity becomes increasingly fine-grained or hardware parallelism increases, the traditional Centralized TDG Building (CB) algorithm progressively becomes a performance bottleneck. The Parallel TDG Building algorithm with Helpers (PBH), which leverages hardware message-passing mechanisms, has achieved significant speedups on the SW26010 platform, but its intensive sub-microsecond irregular synchronizations make it difficult to scale on cache-coherent multicore platforms. This paper proposes Cache-friendly PBH (CPBH), a parallel dependency tracking algorithm optimized for cache-coherent architectures. CPBH introduces a locality-aware lock-free batch synchronization mechanism that reduces the overhead of atomic operation contention and improves data access locality. Additionally, it employs an asynchronous execution strategy to overlap dependency tracking and task graph execution using dynamic reference counting. Experiments on three cache-coherent multicore platforms using 10 HPC benchmarks demonstrate that CPBH achieves an average speedup exceeding 1.4 × compared to CB and over 1.2 × speedup compared to DDAST under fine-grained scenarios.

With the growing demand for users’ flexible use of office desktops and enterprises’ centralized management of information resources, desktop virtualization technologies, represented by remote desktops, have become a prominent approach in current desktop management. Among the mainstream desktop virtualization technologies, Intelligent Desktop Virtualization (IDV) architecture offers significant advantages regarding network dependency and resource utilization. However, the IDV scenario presents challenges on the server, such as the increasing number of centrally managed images, resource consumption due to frequent image pulling, and low efficiency in image synchronization. Moreover, the terminal faces an issue of not fully leveraging hardware resources. Considering the IDV-specific characteristics, we design and implement a set of optimization methods for desktop virtualization, which outperform traditional IDV solutions.

The end host serves as a natural enforcement point for various network functions (NFs), such as network address translators (NATs), firewalls, and load balancers. However, due to the limitations of the Linux networking stack, NFs struggle to achieve high performance when utilizing high-speed network interfaces. The eXpress Data Path (XDP) is a high-performance framework for packet processing within the Linux kernel. It operates as an optimized execution point before the networking stack. In contrast to kernel-bypass solutions like DPDK, XDP offers an appealing alternative by providing comparable performance with lower CPU usage.In this paper, we propose PFC, a novel approach that leverages XDP for Pre-Function table Caching. PFC acts as a packet header processor for incoming packets, it consists of two distinct facets: one involves traffic management within the end host, and the other focuses on processing requests from distributed applications. Experimental results show that PFC can significantly increase throughput and achieve the equivalent performance compared to DPDK. Furthermore, PFC can integrate seamlessly with existing systems without requiring any modifications to the applications.

In this paper, we propose a parallel non-convex approximation framework (NCAQ) for optimization problems whose objective is to minimize a convex function plus the sum of non-convex functions. Based on the structure of the objective function, our framework transforms the non-convex constraints to the logarithmic barrier function and approximates the non-convex problem by a parallel quadratic approximation scheme, which will allow the original problem to be solved by accelerated inexact gradient descent in the parallel environment. Moreover, we give a detailed convergence analysis for the proposed framework. The numerical experiments show that our framework outperforms the state-of-art approaches in terms of accuracy and computation time on the high dimension non-convex Rosenbrock test functions and the risk parity problems. In particular, we implement the proposed framework on CUDA, showing a more than 25 times speed-up ratio and removing the computational bottleneck for non-convex risk-parity portfolio design. Finally, we construct the high dimension risk parity portfolio which can consistently outperform the equal weight portfolio in the application of Chinese stock markets.

NVM provides large memory capacity, long-term data durability, and high memory bandwidth for multi-thread applications on cloud servers. Nowadays, cloud servers often employ NUMA architecture, where the thread scheduling mechanism plays a vital role in overall system performance because of the NUMA property. However, with the increase in server resources' diversity, i.e., hybrid memory systems using DRAM and NVM on NUMA nodes, the exploration space for thread scheduling is expanding rapidly. Unfortunately, the existing thread schedulers, including rule-based algorithms and scheduling domain methods, cannot provide ideal scheduling solutions in such complicated cases. And, those thread schedulers neglect customized heterogeneous memory structures, thus degrading overall system performance. Fortunately, reinforcement learning can choose actions with maximum rewards values in a specific environment, leading the scheduler towards an optimal solution. In this paper, we propose a thread scheduling approach, i.e., Smart Scheduler, by leveraging a reinforcement learning method. Smart Scheduler takes OS event information as input, extends LinUCB to explore the scheduling space, and guides thread-level scheduling. We evaluate Smart Scheduler on the off-the-shelf server equipped with NVM. The experimental results show that the proposed Smart Scheduler can converge faster (usually within 20 actions) than rule-based algorithms and scheduling domain methods and reduce program execution time by up to 59.9%. It also outperforms rule-based algorithms and scheduling domain methods by 4.1% and 19.1% in quality of service latency.

Non-Volatile Memory with byte-addressability invites a new paradigm to access persistent data directly. However, this paradigm brings new challenges to the Non-Uniform Memory Access (NUMA) architecture. Since data accesses cross NUMA node can incur significant performance loss, and, traditionally, OS moves data to the NUMA node where the process accessing it locates to reduce the access latency. However, we find challenges when migrating data on NVM, which motivates us to migrate the process instead. We propose SysMon-N, an OS-level sampling module, to obtain access information about NVM in low overhead. Furthermore, we propose N-Policy to utilize the data collected by SysMon-N to guide process migration. We evaluate SysMon-N and N-Policy on off-the-shelf NVM devices. The experimental results show that they provide 5.9 3.62× bandwidth improvement in the case where cross-node memory accesses happen.

Asynchronous task-based programming models are gaining popularity to address the programmability and performance challenges of contemporary large scale high performance computing systems. In this paper we present AceMesh, a task-based, data-driven language extension targeting legacy MPI applications. Its language features include data-centric parallelizing template, aggregated task dependence for parallel loops. These features not only relieve the programmer from tedious refactoring details but also provide possibility for structured execution of complex task graphs, data locality exploitation upon data tile templates, and reducing system complexity incurred by complex array sections. We present the prototype implementation, including task shifting, data management and communication-related analysis and transformations. The language extension is evaluated on two supercomputing platforms. We compare the performance of AceMesh with existing programming models, and the results show that NPB/MG achieves at most 1.2X and 1.85X speedups on TaihuLight and TH-2, respectively, and the Tend_lin benchmark attains more than 2X speedup on average and attain at most 3.0X and 2.2X speedups on the two platforms, respectively.

多重网格是数值计算领域中一种加速迭代收敛的重要技术,被广泛应用.近年来,大规模并行计算系统向多核化、异构众核化发展,多重网格应用也亟须适应新的并行计算平台.文中采用一种数据驱动的任务并行语言AceMesh将遗产的NAS MG程序移植到"天河二号"和"神威?太湖之光"两种不同架构的国产超算平台上,展示了使用该语言对计算循环、通信代码的任务并行方法,验证了AceMesh语言的跨平台性能可移植性.文中定性地分析了该应用的任务图特征和计算-通信重叠的特点,并分别在两个并行计算平台上将其与现有编程模型MPI/OpenMP和MPI/OpenACC进行性能对比,分析了AceMesh任务图并行程序对访存性能和通信-计算重叠的优化效果.实验数据表明,相比传统的并行编程方法,AceMesh在"神威?太湖之光"和"天河二号"平台上分别最高获得了1.19X和1.85X的性能加速.最后,针对该应用在不同网格层的通信特点以及通信序列化导致大量通信不能隐藏的问题,提出了未来的研究方向.

大气环流模式是研究全球气候变化及其成因的主要工具之一,在大规模异构众核的并行计算系统上高效地并行运行复杂的大气环流模式是一个具有挑战性的课题.Tend_lin是中国科学院大气物理研究所研发的第4代大气环流模式IAP AGCM-4中动力框架的热点过程,具有计算/通信比低的特点.面向国产大规模异构众核超算平台"神威·太湖之光",用OpenACC和AceMesh 2种不同的并行编程接口对Tend_lin进行优化.重点介绍了如何用数据驱动的任务并行编程接口AceMesh对其进行加速,介绍了计算循环和通信代码的任务并行方法,讨论了如何放松通信资源共享,对比了单层任务图和嵌套任务图下的任务映射等优化问题.测试结果表明,相比OpenACC,AceMesh在16～1024进程的不同并行配置下获得了平均2倍左右的性能提升,最后详细分析了性能收益的来源.

Being an extension of classical linear programming, disjunctive programming has the ability to express the problem constraints as combinations of linear equalities and inequalities linked with logic AND and OR operations. All the existing theories such as generalized disjunctive programming, optimization modulo theories, linear optimization over arithmetic constraint formula, and mixed logical linear programming pose one commonality of branching among different solving techniques. However, branching constructs a depth-first search which may traverse a whole bad subtree when the branching makes a mistake ordering a bad successor. In this paper, we propose the interleaved depth-first search with stochastic local optimal increasing (IDFS-SLOI) method for solving the linear optimization problem with disjunctive constraints. Our technique searches depth-first several subtrees in turn, accelerates the search by subtree splitting, and uses efficient backtracking and pruning among the subtrees. Additionally, the local optimal solution is improved iteratively by constructing and solving a stochastic linear programming problem. We evaluate our approach against existing counterparts on the rate-monotonic optimization problem (RM-OPT) and the linear optimization with fuzzy relation inequalities problem (LOFRI). Experimental results show that for the tested instances, the IDFS-SLOI method performs better from performance perspective, especially promising results have been obtained for the larger three groups where the execution time is reduced by 85.6 and 51.6% for RM-OPT and LOFRI, respectively.

Since Balas extended the classical linear programming problem to the disjunctive programming (DP) problem where the constraints are combinations of both logic AND and OR, many researchers explored this optimization problem under various theoretical or application scenarios such as generalized disjunctive programming (GDP), optimization modulo theories (OMT), robot path planning, real-time systems, etc. However, the possibility of combining these differently-described but form-equivalent problems into a single expression remains overlooked. The contribution of this paper is two folded. First, we convert the linear DP/GDP model, linear-arithmetic OMT problem and related application problems into an equivalent form, referred to as the linear optimization over arithmetic constraint formula (LOACF). Second, a tree-search-based algorithm named RS-LPT is proposed to solve LOACF. RS-LPT exploits the techniques of interval analysis and nonparametric estimation for reducing the search tree and lowering the number of visited nodes. Also, RS-LPT alleviates bad construction of search tree by backtracking and pruning dynamically. We evaluate RS-LPT against two most common DP/GDP methods, three state-of-the-art OMT solvers and the disjunctive transformation based method on optimization benchmarks with different types and scales. Our results favor RS-LPT as compared to existing competing methods, especially for large scale cases.

AceMesh is a dataflow-driven'task parallel programming language,which allows programmers to parallelize traditional C/C+ + program by using pragmas marking parallel regions,parallel loops and task regions with data inputs and outputs description.Then the program can be translated to a DAG-based task parallel program,being built dependency graphs at runtime and scheduled to multicore platforms efficiently.This paper analysed typical errors which may exhibit in parallelizing AceMesh programs,and introduced AceMeshCheck,a debug tool for them.The paper presented the implementation details of the tool,and discussed how it reduces the overhead of memory trace collection,and how to rebuild the three dimensional,rectangular access regions from linearized memory access sequences.Experimental results show that the tool can identify typical errors hidden in AceMesh programs with relatively low overhead.

With the recent development on heterogeneous hardware, heterogeneous parallel programming model has been widely used with the intension of simplifying programming and improving efficiency. This paper analyses latest achievements in heterogeneous parallel programming interfaces and runtime supporting systems, and solutions to new problems brought by heterogeneous architectures and various applications. In the end, some future trends in this area are discussed.