The randomized preconditioned CholeskyQR algorithm (rpCholeskyQR) is a state-of-the-art algorithm for tall-and-skinny, ill-conditioned matrices. However, its performance and numerical stability fundamentally depend on the design of the preconditioner, which involves both the sketching operator $\Omega$ and the choice of the sampling factor $d$. In this work, to achieve better performance and adapt to the characteristics of the matrices, we propose a novel XGBoost-driven parallel randomized preconditioner for CholeskyQR Algorithm (XGBPRPr). Firstly, we introduce an XGBoost-driven machine learning model for optimal sampling factor. Secondly, we design a parallel hash-function-driven pseudo-random structured short-axis sparse operator (SSASOs) to enhance numerical stability. Finally, we employ a parallel computation and memory co-optimization strategy, called camco-SpMM, to enhance the performance of sparse matrix-matrix multiplication (SpMM) under the SSASOs structure. Experimental results show that our proposed method achieves up to $2.99 \times$ and $2.23 \times$ speedup compared with the standard QR factorization in the Intel Math Kernel Library (MKL) and the rpCholeskyQR algorithm, respectively, with errors remaining of the same order of magnitude.

The randomized Kaczmarz algorithm is a simple iterative method for solving overdetermined linear systems. However, the classical randomized Kaczmarz algorithm relies on orthogonal projections, and its convergence rate deteriorates significantly when the system exhibits a high linear dependence. This paper describes a novel oblique projection solution, RKOP, a randomized Kaczmarz algorithm with oblique projections based on maximum cosine similarity. Firstly, we select hyperplanes based on maximum cosine similarity and construct oblique projection directions using two hyperplanes to accelerate convergence. Secondly, a dynamic Monte Carlo error estimation method is employed to reduce the computational overhead of error evaluation effectively. Finally, we implement a multi-level parallel framework that achieves effective load balancing through optimized data distribution and uses a delayed update strategy to reduce computational overhead and improve overall efficiency significantly. Experimental results demonstrate that the serial version of RKOP achieves a $64.25 \times$ speedup over the traditional randomized Kaczmarz algorithm. When scaled to 32 cores, RKOP achieves a $57.91 \times$ speedup compared to its single-core version.

As the size of AI models grows dramatically to achieve higher accuracy, more and more models are trained on supercomputers with massive computing and memory. However, it is still challenging to accelerate deep learning applications on supercomputer systems with many-core architectures. In this paper, we propose swDarknet: a Darknet-based heterogeneous parallel deep learning framework on the SW26010 Pro processor. First, We designed three heterogeneous parallel DNN layers: adaptive block-splitting strategy for matrix-multiplication layer, optimized memory access for convolutional layer, and block-based parallel pooling layer. Second, we proposed a model averaging (MA) method to efficiently scale swDarknet across many processors. Finally, we evaluated the performance of swDarknet on a core group. As compared with darknet, the convolutional layers achieved an average speedup of 32x. When training AlexNet and VGG-16, swDarknet can achieve up to 36.79 × and 181.35 × speedup, respectively.

Nowadays, convolutional neural networks are representative of deep learning algorithms. With the development of convolutional neural networks, their network structures become more complex, and the number of parameters for training becomes larger and larger. The parallelization of convolutional neural network algorithms on multicore or many-core processors is essential for training convolutional neural networks. In this paper, we propose a parallel algorithm of LeNet-1 based on the Sunway Bluelight II supercomputer, named SW-LeNet. Moreover, we propose a two-level parallelization scheme, including thread-level optimization and process-level optimization. In thread-level optimization, the following optimization methods are used, including CPEs parallelism, hybrid scheme and DMA optimization, register optimization, and SIMD data parallelism. Data parallelism optimization and parameter packing optimization are used in process-level optimization. Compared with the original LeNet, SW-LeNet can achieve 4.94x speedups in a single core group. Moreover, SW-LeNet can be scaled up to 2,048 processes, with 133,120 cores, and achieves 84.93% parallel efficiency.

The max-min ant system (MMAS) algorithm has found extensive application in tackling combinatorial optimization challenges such as the traveling salesman problem (TSP), production scheduling, and quadratic assignment. Nevertheless, as the scale of the problem increases, the MMAS algorithm gradually encounters performance limitations. To address the performance constraints of MMAS, we propose a parallel max-min ant system (PMMAS) algorithm, where a master subpopulation coordinates multiple subpopulations in parallel search. Furthermore, to facilitate the parallel acceleration of computationally intensive tasks in PMMAS using the CPE array of the SW26010-Pro processor, the selection weight calculation equation in the traditional MMAS algorithm was improved. This improvement led to the introduction of the Sunway parallel max-min ant system (SWPMMAS) algorithm, which implements parallelism using MPI and Athread. The revised selection weight calculation equation is also applicable to the traditional MMAS algorithm and enhances its running speed. Finally, the SWPMMAS algorithm was evaluated using various TSP instances, with city counts ranging from 51 to 11,849. The results demonstrate that the SWPMMAS algorithm provides excellent solutions. For TSP instances with more than 10,000 cities, the SWPMMAS algorithm achieves over 13 × speedup compared to the PMMAS algorithm running on the Sunway architecture and 5.4 × speedup compared to the PMMAS algorithm running on a commercial Shanhe supercomputer. Moreover, testing indicates that the SWPMMAS algorithm exhibits outstanding scalability.

The solution of over-determined equations plays a very important role in fields such as data fitting, signal processing, and machine learning. It is of great significance in predicting natural phenomena, optimizing engineering design, and other fields. However, there is currently no efficient method to solve over-determined equations, either being not accurate enough or consuming a lot of time. In this article, we propose a parallel iterative method for solving over-determined equations, called the PIOD algorithm. By using a sub-convergence condition to terminate the iterative calculation, we have developed a task partitioning strategy for the algorithm and implemented parallelization of the solution of over-determined equations on a distributed memory system. Our proposed algorithm achieves an average speedup of 152 ×times compared to the open-source Eigen solver. Additionally, it also achieves a parallel efficiency of over 30%.

Sparse LU factorization is a critical kernel in scientific computing and engineering applications. A better nonzero pattern of sparse matrixes can accelerate LU factorization by reordering. Traditionally it’s difficult to predict which non-zero pattern is optimal for a sparse matrix. In this paper, we proposed a graph convolutional neural network (GCN) for adaptively selecting the optimal reordering algorithm from five candidate reordering algorithms in the Matrix preprocessing step, referred to as SPM-GCN. After using our SPM-GCN, the average numerical factorization time outperformed the other five algorithms, and the average numerical factorization time was reduced by 17.3% compared with the default method of swSuperLU.

Due to its efficiency and versatility, the preconditioned conjugate gradient algorithm has long been a staple in the realm of iterative linear system solvers. In this paper, we proposed an optimized preconditioned conjugate gradient algorithm tailored for the SW26010-Pro manycore processor, The main work includes: Optimizing data block sizes based on the processor’s storage structure; combining thread-level and data-level parallelism, utilizing manual SIMD to improve efficiency; optimizing the memory access pattern by employing direct memory access and overlapping of computation and communication; employing a shared-memory approach to store long vectors across all cores. Furthermore, we design an accelerated algorithm for reduction operations, to avoid data communication. Experimental results show that the hcaPCG yields up to 28.1× speedups on average compared to the original implementation.

At present, China has reached the world-class level in the research and development of high-performance processors and supercomputers, such as the sunway many-core processor, which is the representative of Chinese high-performance processors. However, the hardware upgrade is difficult to bring about the improvement on the software performance in proportion, and its powerful computing resources cannot be used well at present. In general, sunway many-core processor uses MPE-CPE parallel optimization for hotspot. This paper improves the MPE-CPE parallel optimization, and research a set of new optimization methods, which includes a synchronous mechanism between the MPE (Management Processing Element) and CPE (Computing Processing Element) cluster and an acceleration method of multi-data partition in the CPE cluster. These methods solve the problem of insufficient utilization of CPEs' computing resources and insufficient coordination between MPE and CPEs to a certain extent. By employing the Sunway Taihulight supercomputer, the evaluation measures the execution times and speedups of the comprehensive parallel optimization and MPE-CPE parallel optimization. The result shows that, with different number of processes, the speedups of comprehensive parallel optimization are much higher than those of MPE-CPE parallel optimization, from 23.42% to 36.41%.

Sparse matrix vector multiplication (SpMV) is a critical performance bottleneck for numerical simulation and artificial intelligence training. The new generation of Sunway supercomputer is the advanced exascale supercomputer in China. The SW26010-Pro many-core processor renders itself as a competitive candidate for its attractive computational power in both numerical simulation and artificial intelligence training. In this paper, we propose a heterogeneous and SPM-aggregated SpMV kernel, specifically designed for the SW26010-Pro many-core processor. To fully exploit the computational power of the SW26010-Pro and balance the load of each core group(CG) during computation, we employ asynchronous computation workflow and propose the SPM-aggregated strategy and vector adaptive mapping algorithm. In addition, we propose the two-level data partition scheme to implement computational load balance. In order to improve memory access efficiency, we directly access memory via DMA controller to replace the discrete memory access. Using several optimizations, we achieve a 77.16x speedup compared to the original implementation. Our experimental results show that the hsSpMV yields up to 3.82× speedups on average compared to the SpMV kernel of the state-of-the-art Sunway math library xMath2.0.

With the development of nuclear energy technology, reactor physical calculations have higher requirements for calculation accuracy and speed, and it has become an inevitable trend to use high-performance computers for reactor simulation calculations. The method of characteristics (MOC) is currently recognized as the preferred method for simulating neutron transport in the nuclear reactor core. Based on the architecture of Sunway many-core processor and Sunway Bluelight II supercomputer, this paper proposes a fine grained and universal two-level parallelization, including thread-level parallelization and process-level parallelization. In the thread-level parallelization, the methods such as job pipeline optimization, load balancing across CPEs, and I/O optimization are proposed for acceleration. In the process-level parallelization, a mapping method from software to hardware is proposed. This method can make full use of the hardware of Sunway supercomputers and improve the computing efficiency and data transmission efficiency. For the first time, the OpenMOC program is transplanted and parallelly optimized on the Sunway supercomputers, which enriched the application ecology of Sunway supercomputers. Compared with the original program, the two-level parallelization can achieve up to 18.6x speedup. Moreover, our parallelization is capable to run on more than 3750 processes of Sunway Bluelight II supercomputer with good strong and weak scalability.

The simulation of three-dimensional stress and strain is a research hot spot of computational structural mechanics. As the complexity of the project increasing, the size of the matrix generated increases during the simulation. Therefore, a fast and efficient solver is needed. In this paper, we present swParaFEM, a highly efficient parallel finite element solver on Sunway many-core architecture. It is based on preconditioned conjugate gradient iteration algorithm. We launch a master–slave acceleration model to exploit the computational power of Sunway supercomputer. The kernel aggregation optimization scheme is proposed to deal with the problem that threads’ frequent creation and destruction waste computing resources. Moreover, we improve the data transfer speed from the slave core to the master core through memory access optimization. Using several optimizations, we achieve a speedup of 10.5 $$\times$$ compared to the naive implementation on one compute group of an SW26010-Pro processor and a strong scaling efficiency of 62.8% on 512 compute groups.

The solution of tridiagonal linear systems is used in in various fields and plays a crucial role in numerical simulations. However, there is few efficient solver for tridiagonal linear systems on the new Sunway supercomputer. Based on a three-dimensional heat conduction problem, we propose an adaptive heterogeneous tridiagonal matrix algorithm (AH-TDMA). Our major innovations include: (1) To address computational hotspots within AH-TDMA, a multi-level parallel approach involving MPI+Athread has been adopted. (2) An adaptive data partitioning scheme has been set up to achieve load balance. (3) Employing direct memory access and establishing shared space between registers and main memory, instead of employing discrete memory access, is done to enhance memory access efficiency. (4) The optimization of the loop structure has been made in adjusting the sequencing of the dual-layered loops to reduce communication overhead. The experimental results show that, with one core group, the AH-TDMA achieves a speedup of 99.6 times for hotspot and total time speedup up to 59.4 times compared to the Parallel and Scalable Library for Tridiagonal Matrix Algorithm (PaScal TDMA). The AH-TDMA is scalable up to 2048 core groups, with a parallel efficiency of 69.2%.

The multigrid algorithm is a commonly utilized iterative technique for resolving vast and sparse matrix equations in numerical computation. Hypre is an open-source library of high-performance preconditioners and solvers, incorporates multigrid methods and enables efficient resolution of large, sparse linear systems on parallel computers. We employed a master-slave acceleration model for Algebraic Multigrid method on sunway architecture. Address mapping and intervention strategies for CPE are adopted. Numerical experiments that we proposed optimized algorithm result in speedup of nearly nine times compared to the original version. This showcases the model’s effectiveness in accelerating the resolution of large, sparse linear systems.

Tridiagonal system solver is a basic kernel and has been well-supported in mainstream numerical libraries. The purpose of this paper is to devise an efficient parallel algorithm to solve a large-scale tridiagonal system. Based on the performance analysis of the classic Thomas algorithm and matrix splitting method, we propose a parallel Thomas split (PTS) algorithm. Compared with the matrix splitting method, the PTS algorithm can achieve an acceleration of 10.34 × . Furthermore, we propose a Sunway parallel Thomas split (swPTS) algorithm based on the sw26010pro manycore processor. In the swPTS algorithm, we propose a specific data partitioning scheme to implement MPI+Athread parallelism. In the reduced set of equations, a new reduction approach for the Sunway architecture is proposed. Experiments show that the parallel elimination stage of our swPTS algorithm achieves up to 38.31 × speedup over a PTS algorithm, and overall reaches 5.74 × speedup over a Thomas algorithm.

Over the past three decades, ab initio electronic structure calculations of large, complex and metallic systems are limited to tens of thousands of atoms in computational accuracy and efficiency on leadership supercomputers. We present a massively parallel discontinuous Galerkin density functional theory (DGDFT) implementation, which adopts adaptive local basis functions to discretize the Kohn-Sham equation, resulting in a block-sparse Hamiltonian matrix. A highly efficient pole expansion and selected inversion (PEXSI) sparse direct solver is implemented in DGDFT to achieve O(N1.5) scaling for quasi two-dimensional systems. DGDFT allows us to compute the electronic structures of complex metallic heterostructures with 2.5 million atoms (17.2 million electrons) using 35.9 million cores on the new Sunway supercomputer. The peak performance of PEXSI can achieve 64 PFLOPS (~5% of theoretical peak), which is un-precedented for sparse direct solvers. This accomplishment paves the way for quantum mechanical simulations into mesoscopic scale for designing next-generation electronic devices.

This paper proposes a distributed-memory parallel randomized iterative algorithm for solving linear systems, called the parallel randomized kaczmarz projection (PRKP) algorithm. The algorithm has the property of greedy sampling, alternating projection, and lazy approximation. We derive the alternating projection process from the Randomized Kaczmarz algorithm and develop the greedy sampling and lazy approximation process to improve the convergence rate and reduce the per-process communication volume. Moreover, we develop a sampling residual estimation scheme for our proposed algorithm, which greatly reduces the extra computation cost required to obtain residuals. Our experimental results show that the proposed algorithm significantly outperforms previous parallel iterative algorithms on overdetermined linear systems, and yields up to $\mathbf{159}\times$ and $\mathbf{384}\times$ speedups on average respectively compared to the leading pipelined CG and pipelined GMRES algorithms.