IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum : [proceedings]

The matching problem formulated as Maximum Cardinality Matching in General Graphs (MCMGG) finds the largest matching on graphs without restrictions. The Micali-Vazirani algorithm has the best asymptotic complexity for solving MCMGG when the graphs are sparse. Parallelizing matching in general graphs on the GPU is difficult for multiple reasons. First, the augmenting path procedure is highly recursive, and NVIDIA GPUs use registers to store kernel arguments, which eventually spill into cached device memory, with a performance penalty. Second, extracting parallelism from the matching process requires partitioning the graph to avoid any overlapping augmenting paths. We propose an implementation of the Micali-Vazirani algorithm which identifies bridge edges using thread-parallel breadth-first search, followed by block-parallel path augmentation and blossom contraction. Augmenting path and Union-find methods were implemented as stack-based iterative methods, with a stack allocated in shared memory. Our experimentation shows that compared to the serial implementation, our approach results in up to 15-fold speed-up for very sparse regular graphs, up to 5-fold slowdown for denser regular graphs, and finally a 50-fold slowdown for power-law distributed Kronecker graphs. This implementation has been open-sourced for further research on developing combinatorial graph algorithms on GPUs.

Tracing is a basic approach to analyzing performance and understanding MPI program behavior patterns. However, MPI event trace requires increasingly large storage space as the parallel scale grows. Besides MPI event trace, many performance analysis tasks (e.g., performance variance detection, proxy synthesis) also require detailed runtime performance metrics, which further aggravates the storage issue. In this paper, we propose a scalable tracing tool to effectively record and compress MPI event trace and related runtime performance metrics. The tool analyzes the data redundancy caused by loops and SPMD (single program multiple data) property of MPI programs. According to the analysis, the tool can compactly reorganize and store the data. Compared with existing trace compression methods, our tool can achieve generally higher compression ratio and less time cost.

The majority problem can be stated as follows: there is a collection of n bank cards that belong to different bank accounts. The only way to "read" these cards is through one or more two-input "equivalence testers". Each tester accepts two cards and outputs "yes" if two input cards correspond to the same bank account or "no" if they do not. The objective of the majority problem is to design a solution that uses the minimum number of tests to determine if more than n/2 cards belong to the same bank account and then find these cards. We first review two sequential solutions, including the Boyer-Moore optimal linear solution (i.e., with a time complexity of O(n)). Then, we introduce an optimal elimination solution together with a special structure to support a cost-optimal parallel solution that solves the problem in O(log n) using O(n/ log n) processors (i.e., testers). That is, this solution matches the optimal sequential solution with O(n) operations.

Sparse Matrix-Vector (SpMV) multiplication is a frequently encountered computational kernel, that is notorious for achieving only a small fraction of system peak performance. The optimization of this kernel and the identification of bottlenecks that limit its performance have been the subject of considerable research efforts. Nevertheless, obtaining an unbiased matrix dataset that can expose these bottlenecks remains a recurring challenge. To this end, we propose a feature-based artificial matrix generation method, that associates the most common SpMV bottlenecks with core matrix features, and develop a generator program that can quickly produce a wide variety of matrices on the fly for performance analysis. To evaluate our generator, we compare the performance of artificial and real matrices, across three different platforms: a CPU, a GPU and an FPGA.

The presentation of Peachy Parallel Assignments at parallel and distributed computing education workshops is an effort to promote the reuse of high-quality assignments, both saving precious faculty time and improving the quality of course assignments. These assignments must have been used in class and are selected for being easy to adopt by other instructors and for being “cool and inspirational” so that students spend time on them and talk about them with others. The assignments and their materials are also archived on the Peachy Parallel Assignments website.In this paper, we present two new assignments. The first has students implement the Mandelbrot set in Python, combining an interesting image with Python’s ease of use. The second assignment is a substantial project to implement a programming contest judge. It requires that students use many parallel and distributed computing concepts, with the added benefit of solving a “real problem” and creating software with which students may have personally interacted.

Graphic Processing Units (GPUs) have become ubiquitous in scientific computing. However, writing efficient GPU kernels can be challenging due to the need for careful code tuning. To automatically explore the kernel optimization space, several auto-tuning tools - like Kernel Tuner - have been proposed. Unfortunately, these existing auto-tuning tools often do not concern themselves with integration of tuning results back into applications, which puts a significant implementation and maintenance burden on application developers. In this work, we present Kernel Launcher: an easy-to-use C++ library that simplifies the creation of highly-tuned CUDA applications. With Kernel Launcher, programmers can capture kernel launches, tune the captured kernels for different setups, and integrate the tuning results back into applications using runtime compilation. To showcase the applicability of Kernel Launcher, we consider a real-world computational fluid dynamics code and tune its kernels for different GPUs, input domains, and precisions.

Functiona1, memory-managed parallel languages (FMPLs) are a recent innovative approach to shared-memory parallel programming. Despite their rising prevalence in other areas, FMPLs have yet to gain traction in HPC. In this work, we explore the utility of FMPLs for HPC by re-implementing the NAS Parallel Benchmarks in an FMPL.For this study, we ported the benchmarks into the Parallel ML language. We discuss the advantages and disadvantages of using Parallel ML for HPC applications based on our development experience. We compare the performance of our Parallel ML implementation to the existing C/OpenMP version. The FMPL implementations are $1.02 \times -5.76 \times$ slower compared to OpenMP. Our positive development experience combined with some competitive performance results suggest that FMPLs have the potential to become a viable choice for HPC applications. We conclude by describing our future work to automatically manage distributed memory within an FMPL, creating a compelling new programming model for HPC.

For an undirected graph $G=(V, E)$ and a k-nonnegative integer vector $p=\left(p_{1}, \ldots, p_{k}\right)$, a mapping $l: V \rightarrow$ $\mathbb{N} \cup\{0\}$ is called an $L(p)$-labeling of G if $|l(u)-l(v)| \geq p_{d}$ for any two distinct vertices $u, v \in V$ with distance d, and the maximum value of $\{l(v) \mid v \in V\}$ is called the span of l. Originally, $L(p)$-labeling of G for $p=(2,1)$ is introduced in the context of frequency assignment in radio networks, where 'close' transmitters must receive different frequencies and 'very close' transmitters must receive frequencies that are at least two frequencies apart so that they can avoid interference. $L(p)$ LABELING is the problem of finding the minimum span $\lambda_{p}$ among $L(p)$-labelings of G, which is NP-hard for every non-zero p. $L(p)$-LABELING is well studied for specific p 's; in particular, many (exact or approximation) algorithms for general graphs or restricted classes of graphs are proposed for $p=(2,1)$ or more generally $\boldsymbol{p}=(p, q)$. Unfortunately, most algorithms strongly depend on the values of p, and it is not apparent to extend algorithms for p to ones for another $p^{\prime}$ in general. In this paper, we give a simple polynomial-time reduction of $L(p)$-LABELING on graphs with a small diameter to METRIC (PATH) TSP, which enables us to use numerous results on (METRIC) TSP. On the practical side, we can utilize various high-performance heuristics for TSP, such as Concordo and LKH, to solve our problem. On the theoretical side, we can see that the problem for any p under this framework is 1.5 -approximable, and it can be solved by the Held-Karp algorithm in $O\left(2^{n} n^{2}\right)$ time, where n is the number of vertices, and so on.

A program's architecture-how it organizes the invocation of application-specific logic-influences important program characteristics including its scalability and security. Architecture details are usually expressed in the same programming language as the rest of a program, and can be difficult to distinguish from non-architecture code. And once defined, architecture is difficult and risky to change because it couples tightly with application logic over time. We introduce C-Saw: an approach to express a software's architecture using a new embedded domain-specific language (EDSL) designed for that purpose. It decouples application-specific logic from architecture, making it easier to identify architectural details of software. C-Saw leverages three ideas: (i) introducing a new, formally-specified EDSL to separate an application's architecture description from its programming language; (ii) reducing architecture implementation to the definition and management of distributed key-value tables, and (iii) introducing an expressive state-management abstraction for distributed applications. We describe a prototype implementation of C-Saw for C programs and use it to build end-to-end examples of expressing and changing the architecture of widely-used, third-party software. We evaluate this on Redis, cURL, and Suricata and find that C-Saw provides expressiveness and reusability, requires fewer lines of code when compared to directly using C to express architectural patterns, and imposes low performance overhead on typical workloads.

Direct numerical simulation (DNS) by spectral methods is widely used to reveal turbulence properties because of its high accuracy. DNS with a larger number of grid points than simulations conducted to date, however, requires a large amount of computation time and memory capacity. Therefore, DNS with a large number of grid points is not realistic, even if state-of-the-art supercomputers are used. In this paper, in order to execute a DNS with a larger number of grid points, a memory-saving DNS code has been developed to solve the memory capacity constraint at the expense of increased CPU time. Variables in the code are subdivided and stored in multiple temporary files and are processed sequentially during the calculations. The DNS code has been carried out as an early performance evaluation on SX-Aurora TSUBASA equipped with synchronous and asynchronous input and output (I/O) functions. The results clarified that the synchronous I/O performance outperformed the asynchronous I/O performance. The shortest calculation time was achieved using the synchronous I/O to eight temporary files on solid state drives.

Acquiring significant speedup in gate-level simulation has proven challenging due to limitations such as synchronization and partition overhead. As a result, serial event-driven simulation remains the industry standard despite its slow runtime performance. This paper presents the utilization of Taskflow, a task-graph computing system, to effectively enhance the speedup of gate-level simulation. Taskflow provides solutions to challenges faced in previous attempts at parallelizing gate-level simulation, such as scalable pipelines, conditional tasking, and heterogeneous work stealing. The focus of the paper is on improving speedup within and-inverter graphs, which are used to represent structural implementations of circuits at the gate-level. Experimental results demonstrate significant speedup within and-inverter graph benchmarks.

The field of Quantum Computing (QC) is increasingly becoming applicable for a wide range of computationally intensive problems. Especially Variational Quantum Algorithms (VQA) are promising as they allow dealing with the practical challenges of current quantum devices (also referred to as Noisy Intermediate-Scale Quantum devices). However, developing VQA-based solutions is not a straightforward process but involves several development phases, each with different design decisions and implications. In this paper, we share experiences made in developing VQA-based solutions for two different industrial case studies. Based on the solutions, we extracted five generic development phases quantum engineers must take into consideration when engineering quantum solutions. We have embedded the phases in a model (i.e. a phase model) to show their natural order and to illustrate the phases that require more development iterations. The main objective of this paper is not only to share and discuss our findings and experiences but also to provide the phase model as a starting point or guideline to structure the development process.

Interoperability between libraries is often hindered by incompatible data formats, which can necessitate creating new copies of data when transferring data back and forth between different libraries. This additional data movement incurs additional runtime costs, particularly for sparse applications, where the costs of data movement often dwarf compute costs. In this paper, we investigate interoperability in the context of the C++ GraphBLAS Specification, where C++ concepts allow GraphBLAS algorithms to accept any matrix type as long as it follows the matrix interface defined in the GraphBLAS matrix concept. We first develop non-owning, lazily evaluated adapted views for a number of external data structures, including two categories of graphs defined in the Northwest Graph Library (NWGraph) and traditional pointer-based CSR data structures. These adapted views fulfill the C++ GraphBLAS matrix concept, allowing them to be used inside GraphBLAS algorithms. We then evaluate the performance of these adapted views across two kernels, matrix reduction and sparse times dense matrix multiplication (SpMM), where the performance achieved using a single generic implementation with these views largely matches the performance achieved operating directly on the original data structures, with a slight performance loss in one case. We then propose a mechanism for automatically discovering the availability of these views, allowing algorithms to directly accept external data structures. We also discuss potential extensions to the C++ GraphBLAS specification that might eliminate the small performance dip observed for one of the views.

With the NVIDIA Turing architecture generation, several NVIDIA graphics processing units (GPUs) have introduced ray tracing acceleration hardware (RT cores). Ray tracing processing can be regarded as a simulation of wave and particle propagation, collision, and reflection. Therefore, it is expected to be applied to computational science and high-performance computing. However, few studies have been conducted using RT cores. The purpose of this research is to demonstrate the use of RT cores in the scientific and technical computing fields. We implemented a radio wave propagation loss calculation with the programmable ray tracing application framework OptiX and evaluated its performance. Furthermore, we investigated the challenges of reducing the description of framework-specific settings and the needs of hardware allocation. In the simple two spheres experiment, the RT core implementation showed the highest performance. Moreover, the acceleration was super linear scaling, between (10000, 5000) and (20000, 10000). In the experiment with a sphere and planes, the performance achieved by the RT cores was up to approximately 390 times higher than the parallel execution of the BVH search algorithm. We also proved that a large number of RT cores yielded higher performance. In the open data problem space experiment, we evaluated various GPUs and revealed that a larger number of RT cores is effective. These results show that RT cores are sufficiently effective for radio propagation calculations with an adequate number of ray projections. Through this research, we contributed to the RT core use in computational science by proposing an implementation method for ray tracing applications and revealing the effects of RT cores in radio wave propagation loss calculations.

Quantum computing is emerging as an important (but radical) technology that might take us beyond Moore’s law for certain applications. Today, in parallel with improving quantum computers, computer scientists are relying heavily on quantum circuit simulators to develop algorithms. Most existing quantum circuit simulators run on general-purpose CPUs or GPUs. However, at the same time, quantum circuits themselves offer multiple opportunities for parallelization, some of which could map better to other architecture– architectures such as reconfigurable systems. In this early work, we created a quantum circuit simulator system called Q2Logic. Q2Logic is a coarse-grained reconfigurable architecture (CGRA) implemented as an overlay on Field-Programmable Gate Arrays (FPGAs), but specialized towards quantum simulations. We described how Q2Logic has been created and reveal implementation details, limitations, and opportunities. We end the study by empirically comparing the performance of Q2Logic (running on a Intel Agilex FPGA) against the state-of-the-art framework SVSim (running on a modern processor), showing improvements in three large circuits(#qbit≥27), where Q2Logic can be up-to 7x faster.

Some of the most important categories of performance events count the data traffic between the processing cores and the main memory. However, since these counters are not coreprivate, applications require elevated privileges to access them. PAPI offers a component that can access this information on IBM systems through the Performance Co-Pilot (PCP); however, doing so adds an indirection layer that involves querying the PCP daemon. This paper performs a quantitative study of the accuracy of the measurements obtained through this component on the Summit supercomputer. We use two linear algebra kernelsa generalized matrix multiply, and a modified matrix-vector multiply-as benchmarks and a distributed, GPU-accelerated 3D-FFT mini-app (using cuFFT) to compare the measurements obtained through the PAPI PCP component against the expected values across different problem sizes. We also compare our measurements against an in-house machine with a very similar architecture to Summit, where elevated privileges allow PAPI to access the hardware counters directly (without using PCP) to show that measurements taken via PCP are as accurate as the those taken directly. Finally, using both QMCPACK and the 3DFFT, we demonstrate the diverse hardware activities that can be monitored simultaneously via PAPI hardware components.

We propose, implement and evaluate a data streaming unit (DSU) for a Coarse-Grained Reconfigurable Array (CGRA) that is tightly coupled to its host CPU. The DSU accesses system memory at the granularity of cache lines and streams data to CGRA cells that perform loads, and conversely from CGRA cells that perform stores. The unit supports both direct and indirect accesses, as well as dynamic accesses in which the access location is computed on the CGRA. Further, the DSU can exploit cache line reuse to feed (collect) data on the same cache line to (from) multiple CGRA cells, reducing the number of memory transactions. A prototype DSU is implemented on an Intel Stratix 10MX FPGA that is connected to external DDR4 memory. Evaluation shows that ideal data streaming throughput is achieved for common direct accesses, and for indirect ones when sparsity structure is present. Evaluation also shows that the DSU is effective in exploiting cache line reuse. Thus, we conclude that direct data streaming between memory and CGRA cells is effective.

Diabetes impacts around 8% of the world’s population, with Type 2 diabetes comprising up to 90% of cases. This chronic disease is characterized by a metabolic resistance to insulin which results in a high blood sugar level and increased potential for serious health complications. Preventative medicine and the detection of genetic predisposition play a key part in successful treatment. Although several factors have been identified as possible indicators of underlying diabetes, they are not the same in every patient. There have been different approaches to producing predictive models that could help identify risk of onset diabetes. Models built using Machine Learning algorithms have showed promise in the past in detecting relevant features in sample datasets with data from patients at risk of developing diabetes. However, overall performance has not been consistent across datasets. In this paper we describe a feature extraction approach using Hyperdimensional Computing as a tool for improving already existing classification models. We tested our approach using two public datasets and compare across several state of the art models. Our approach improves poor performing models while fine tuning models with a high classification accuracy.

In this work, we comment on the current practices for benchmarking streaming graph systems. We deduce that present practices can stifle future innovations in storage design, establish wrong precedents, lead to premature optimizations and complexity, and are a recipe for errors. We analyze why such benchmarking errors are prevalent and propose potential solutions. We further deliver a code base, G-Bench, that can be easily integrated with prior and future research prototypes to provide fair performance comparison metrics without compromising innovations. Our evaluations contradict results published in some recent works.

Information-theoretic community discovery method (popularly known as Infomap) is known for delivering better quality results in the Lancichinetti–Fortunat–Radicchi (LFR) benchmark compared to modularity-based algorithms. Parallel algorithms have been developed for Infomap due to the computational challenge of analyzing massive graphs resulting from the tremendous growth of information in bio-sciences, social sciences, business, and other domains. The state-of-the-art techniques on information-theoretic community discovery use hash tables for storing vertex neighborhood flow information, which can be computationally expensive due to collision handling operations and CPU branch mispredictions. The Accelerated Sparse Accumulation (ASA) hardware accelerator for hash accumulation has been developed recently for sparse matrix-matrix multiplication (SpGEMM). We generalize the interface of the ASA accelerator and demonstrate that for state-of-the-art parallel Infomap, the accelerator for hash accumulation with fast on-chip memory can overcome the performance bottlenecks of software hash tables and can achieve a speedup of $5.56\times$ while reducing the number of branch mispredictions by 59%, the CPI rate by 21%, and the total number of instructions by 24%.