The task-based dataflow programming model has emerged as an alternative to the process-centric programming model for extreme-scale applications. However, load balancing is still a challenge in task-based dataflow runtimes. In this paper, we present extensions to the PaRSEC runtime to demonstrate that distributed work stealing is an effective load-balancing method for task-based dataflow runtimes. In contrast to shared-memory work stealing, we find that each process should consider future tasks and the expected waiting time for execution when determining whether to steal. We demonstrate the effectiveness of the proposed work-stealing policies for a sparse Cholesky factorization, which shows a speedup of up to 35% compared to a static division of work.

Summary form only given, as follows. The complete presentation was not made available for publication as part of the conference proceedings. The technological trends in HPC system evolution indicates an increasing burden placed on application developers due to the management of the unprecedented complexity levels of hardware and its associated performance characteristics. Many existing scientific applications codes are unlikely to perform well on future systems without major modifications or even complete rewrites. In the future, it will be necessary to utilize, in concert, many characteristics such as multiple levels of parallelism, many lightweight cores, complex memory hierarchies, novel I/O technology, power capping, system-wide temporal/spatial performance heterogeneity and reliability concerns. The parallel and distributed computing (PDC) community has developed new programming models, algorithms, libraries and tools to meet these challenges in order to accommodate productive code development and effective system use. However, the scientific application community still needs to identify the benefit through practical evaluations. Thus, the focus of this workshop is on methodologies and experiences used in scientific and engineering applications and algorithms to achieve sustainable code development for better productivity, application performance and reliability.

Communication between information processing systems becomes a challenge, especially in the “big data” era. It is a mandatory subject in the topic “Architecture and organization” of the computer science curriculum. However, in our experience, it is a rather complex topic for students. Simulation visualization can be used to graphically illustrate various concepts of computer science. In this paper, we present the application of NICSim-vhd, which is an acronym for VHDL-based Network Interface Card simulation model, as a pedagogical tool for supporting undergraduate computer science students’ education. NICSim-vhd allows simulating the network-to-memory data path at a network node and generating Value Change Dump (VCD) files for simulation visualization of hardware description languages-based models. We provide a taxonomy of learner engagement with simulation visualization. Grounded in Bloom’s well recognized taxonomy of understanding, we suggest how to assess the learning outcomes to which such engagement may lead.

The HYSPLIT air concentration model is an operational Lagrangian trajectory and dispersion model that calculates the concentration or the distribution of pollutants by releasing and tracking particles or puffs. It is widely used for tracking and forecasting the release of pollutants such as radioactive material, dust storms and volcanic ash. Due to the massively parallel nature of the particle tracking system, the HYSPLIT model shows potential to be accelerated by multiple CPUs and GPUs. However, porting such a legacy application to a GPU requires non-trivial work to isolate the parallel code regions and correctly adapt the code to a parallel programming model. This paper presents methods to port the current HYSPLIT code to the shared-memory OpenMP and the CUDA programming models. In the OpenMP approach, the original HYSPLIT concentration model is analyzed, profiled and refactored to remove barriers for parallelizing on multi-core CPUs. A linear speed-up with some serial overhead is achieved for the OpenMP version. In the CUDA approach, we utilize the computing power of NVIDIA GTX960 and Tesla P100 GPUs, providing 4-5 x faster overall performance than the original. With the help of GPU coarse-grained parallelism, a maximum 12.9x speedup has been measured, compared to the original program running on a CPU. The disturbance of different parallelization approaches to the original code base shows that most of the highly difficult work is in the refactoring, and that CUDA requires extensive, but mostly relatively shallow, changes.

Presents the introductory welcome message from the conference proceedings. May include the conference officers' congratulations to all involved with the conference event and publication of the proceedings record.

The Asymmetric Multiprocessor (AMP) architecture brings new opportunities to achieve better trade-offs between performance and operational/financial costs. This paper presents the case of an AMP to address poor I/O performance in a virtualized HPC system, by using small side-cores to offload I/O processing. We use full machine simulations to explore the micro-architectural parameter space in detail and perform an energy-delay-area analysis, taking into account the relationship between size and access delay in the caches. The simulation side-core model has been validated on the Atom processor, with performance counter metrics being within 11%. study focuses on TLBs and caches which our results show to have a remarkable impact on performance. Compared with a previous AMP study considering only performance and limited to existing hardware, our results confirm the broad nature of that design, including the preference for an asymmetric 2-way CPU pipeline. Our improved methodology also boosts the degree of confidence in these results. We however show that the optimal features of an efficient side-core are smaller and simpler L1/L2 caches (16KB 4-way and 16KB 2-way I/D caches and a 128KB 4-way L2 cache) and L1/L2 TLBs (32/48 entry fully associative L1 I/D LBs and 256 entry 4-way L2 I/D TLBs). Meanwhile, our analysis reveals that a processor module consisting of two big cores and a small side-core of our design can reduce average power, energy, and area by 9.2%, 8%, and 24.4%, respectively, compared with a module of three big cores (the AMD K10), while retaining performance (at the cost of 1.3% performance loss).

Scheduling algorithms have a significant impact in the optimal utilization of HPC facilities. Waiting time, response time, slowdown and weighted slowdown are classical metrics used to compare the performance of different scheduling algorithms. This paper investigates the effects of four artefacts, namely non-determinism, shuffling, time shrinking and sampling, on these metrics. We present a scheduling framework based on emulation, that is, using a real scheduler (Slurm) with a sleep program able to take into account periods of suspension. The framework is able to emulate a 50K core cluster using 10 virtualized nodes, with the scheduler running on an isolated node. We find that the non-determinism in repeatedly running a workload has a small but discernible effect of these metrics, and that shuffling job order in a workload increases this by a factor of 5-10. Experiments with shuffled workloads indicate that the average difference of the Backfill and Suspend-Resume strategy performance is within this variation. We also propose methodologies for time shrinking and sampling to decrease the duration of emulations, while aiming to keep these metrics invariant (or linear variant) with the original workload. We find that time shrinking to a factor of up to 90% can have similar effect on the metrics as non-determinism. For sampling, our methodology preserved the distribution of job sizes to a high extent, but had a variation in the metrics somewhat greater than for shuffling. Finally, we use our framework to study in-depth Slurm's scheduling performance, and discover a deficiency in the Suspend-Resume implementation.

MapReduce framework has been one of the most prominent ways for efficient processing large amount of data requiring huge computational capacity. On-demand computing resources of Public Clouds have become a natural host for these MapReduce applications. However, the decision of what type and in what amount computing and storage resources should be rented is still a user's responsibility. This is not a trivial task particularly when users may have performance constraints such as deadline and have several Cloud product types to choose with the intention of not spending much money. Even though there are several existing scheduling systems, however, most of them are not developed to manage the scheduling of MapReduce applications. That is, they do not consider things such as number of map and reduce tasks that are needed to be scheduled and heterogeneity of Virtual Machines (VMs) available. This paper proposes a novel greedy-based MapReduce application scheduling algorithm (MASA) that considers the user's constraints in order to minimize cost of renting Cloud resources while considering Service Level Agreements (SLA) in terms of the user given budget and deadline constraints. The simulation results show that MASA can achieve 25–50% cost reduction in comparison to current SLA agnostic methods and there is only 10% performance disparity between MASA and an exhaustive search algorithm.

Presents the introductory welcome message from the conference proceedings. May include the conference officers' congratulations to all involved with the conference event and publication of the proceedings record.

The proliferation of cloud computing has revolutionized the hosting and delivery of Internet-based application services. However, with the constant launch of new cloud services and capabilities almost every month by both big (e.g., Amazon Web Service and Microsoft Azure) and small companies (e.g., Rackspace and Ninefold), decision makers (e.g., application developers and chief information officers) are likely to be overwhelmed by choices available. The decision-making problem is further complicated due to heterogeneous service configurations and application provisioning QoS constraints. To address this hard challenge, in our previous work, we developed a semiautomated, extensible, and ontology-based approach to infrastructure service discovery and selection only based on design-time constraints (e.g., the renting cost, the data center location, the service feature, etc.). In this paper, we extend our approach to include the real-time (run-time) QoS (the end-to-end message latency and the end-to-end message throughput) in the decision-making process. The hosting of next-generation applications in the domain of online interactive gaming, large-scale sensor analytics, and real-time mobile applications on cloud services necessitates the optimization of such real-time QoS constraints for meeting service-level agreements. To this end, we present a real-time QoS-aware multicriteria decision-making technique that builds over the well-known analytic hierarchy process method. The proposed technique is applicable to selecting Infrastructure as a Service (IaaS) cloud offers, and it allows users to define multiple design-time and real-time QoS constraints or requirements. These requirements are then matched against our knowledge base to compute the possible best fit combinations of cloud services at the IaaS layer. We conducted extensive experiments to prove the feasibility of our approach.

Internet of Things (IoT) has gained substantial attention recently and play a significant role in smart city application deployments. A number of such smart city applications depend on sensor fusion capabilities in the cloud from diverse data sources. The authors introduce the concept of IoT and present in detail ten different parameters that govern our sensor data fusion evaluation framework. They then evaluate the current state-of-the art in sensor data fusion against our sensor data fusion framework. The authors' main goal is to examine and survey different sensor data fusion research efforts based on our evaluation framework. The major open research issues related to sensor data fusion are also presented.

We present a general technique to solve Partial Differential Equations, called robust stencils, which make them tolerant to soft faults, i.e. bit flips arising in memory or CPU calculations. We show how it can be applied to a two-dimensional Lax-Wendroff solver. The resulting 2D robust stencils are derived using an orthogonal application of their 1D counterparts. Combinations of 3 to 5 base stencils can then be created. We describe how these are then implemented in a parallel advection solver. Various robust stencil combinations are explored, representing tradeoff between performance and robustness. The results indicate that the 3-stencil robust combinations are slightly faster on large parallel workloads than Triple Modular Redundancy (TMR). They also have one third of the memory footprint. We expect the improvement to be significant if suitable optimizations are performed. Because faults are avoided each time new points are computed, the proposed stencils are also comparably robust to faults as TMR for a large range of error rates. The technique can be generalized to 3D (or higher dimensions) with similar benefits.

Robust resource share estimation of data-intensive workloads is integral to efficient workload management in a (virtualized) cluster where multiple systems co-exist and share the same infrastructure. However, developing a reliable resource estimator is quite challenging due to (i) heterogeneity of workloads (e.g. stream processing, batch processing, transactional, etc.) in a multi-system shared cluster, (ii) limited (in batch processing) or complete uncertainties (in stream processing) on input data size or arrival rates, and (iii) changing configurations from run to run. To address above challenges, we propose an inclusive framework and related techniques for workload profiling, similar job identification, and resource distribution prediction in a cluster. Our analysis shows that the framework can successfully estimate the whole spectrum of resource usage as probability distribution functions for wide ranges of data-intensive workloads.

Internet of Things IoT has gained substantial attention recently and play a significant role in smart city application deployments. A number of such smart city applications depend on sensor fusion capabilities in the cloud from diverse data sources. The authors introduce the concept of IoT and present in detail ten different parameters that govern our sensor data fusion evaluation framework. They then evaluate the current state-of-the art in sensor data fusion against our sensor data fusion framework. The authors' main goal is to examine and survey different sensor data fusion research efforts based on our evaluation framework. The major open research issues related to sensor data fusion are also presented.

Ultra-large–scale simulations via solving partial differential equations (PDEs) require very large computational systems for their timely solution. Studies shown the rate of failure grows with the system size, and these trends are likely to worsen in future machines. Thus, as systems, and the problems solved on them, continue to grow, the ability to survive failures is becoming a critical aspect of algorithm development. The sparse grid combination technique (SGCT) which is a cost-effective method for solving higher dimensional PDEs can be easily modified to provide algorithm-based fault tolerance. In this article, we describe how the SGCT can produce fault-tolerant versions of the Gyrokinetic Electromagnetic Numerical Experiment plasma application, Taxila Lattice Boltzmann Method application, and Solid Fuel Ignition application. We use an alternate component grid combination formula by adding some redundancy on the SGCT to recover data from lost processes. User-level failure mitigation (ULFM) message passing interface (MPI) is used to recover the processes, and our implementation is robust over multiple failures and recovery (processes and nodes). An acceptable degree of modification of the applications is required. Results using the 2-D SGCT show competitive execution times with acceptable error (within 0.1% to 1.0%), compared to the same simulation with a single full resolution grid. The benefits improve when the 3-D SGCT is used. Experiments show the applications ability to successfully recover from multiple failures, and applying multiple SGCT reduces the computed solution error. Process recovery via ULFM MPI increases from approximately 1.5 sec at 64 cores to approximately 5 sec at 2048 cores for a one-off failure. This compares applications’ built-in checkpointing with job restart in conjunction with the classical SGCT on failure, which have overheads four times as large for a single failure, excluding the recomputation overhead. An analysis for a long-running application considering recomputation times indicates a reduction in overhead of over an order of magnitude.

The need to make large-scale scientific simulations resilient to the shrinking and growing of compute resources arises from exascale computing and adverse operating conditions (fault tolerance). It can also arise from the cloud computing context where the cost of these resources can fluctuate.In this paper, we describe how the Sparse Grid Combination Technique can make such applications resilient to shrinking compute resources. The solution of the non-trivial issues of dealing with data redistribution and on-the-fly malleability of process grid information and ULFM MPI communicators are described. Results on a 2D advection solver indicate that process recovery time is significantly reduced from the alternate strategy where failed resources are replaced, overall execution time is actually improved from this case and for checkpointing and the execution error remains small, even when multiple failures occur.

A disruptive technology that is influencing not only computing paradigm but every other business is the rise of big data. Internet of Things (loT) applications are considered to be a major source of big data. Such loT applications are in general supported through clouds where data is stored and processed by big data processing systems. In order to improve the efficiency of cloud infrastructure so that they can efficiently support loT big data applications, it is important to understand how these applications and the corresponding big data processing systems will perform in cloud computing environments. However, given the scalability and complex requirements of big data processing system5, an empirical evaluation on actual cloud infrastructure can hinder the development of timely and cost effective loT solutions. Therefore, a simulator supporting MT applications in cloud environment is highly demanded, but such work is still in its infancy. To fill this gap, we have designed and implemented IOTSim which supports and enables simulation of loT big data processing using MapReduce model in cloud computing environment. A real case study validates the efficacy of the simulator. (C) 2016 Elsevier B.V. All rights reserved.

Many large scale scientific simulations involve the time evolution of systems modelled as Partial Differential Equations (PDEs). The sparse grid combination technique (SGCT) is a cost-effective method for solve time-evolving PDEs, especially for higher-dimensional problems. It consists of evolving PDE over a set of grids of differing resolution in each dimension, and then combining the results to approximate the solution of the PDE on a grid of high resolution in all dimensions. It can also be extended to support algorithmic-based fault-tolerance, which is also important for computations at this scale.In this paper, we present two new parallel algorithms for the SGCT that supports the full distributed memory parallelization over the dimensions of the component grids, as well as across the grids as well. The direct algorithm is so called because it directly implements a SGCT combination formula. We give details of the design and implementation of a ‘partial’ sparse grid data structure, which is needed for its efficient implementation. The second algorithm converts each component grid into their hierarchical surpluses, and then uses the direct algorithm on each of the hierarchical surpluses. The conversion to/from the hierarchical surpluses is also an important algorithm in its own right. It requires a technique called sub-griding in order to correctly deal with the combination of very small surpluses. An analysis of both indicates the direct algorithm minimizes the number of messages, whereas the hierarchical surplus minimizes memory consumption and offers a reduction in bandwidth by a factor of 1–2−d, where d is the dimensionality of the SGCT. However, this is offset by its incomplete parallelism (70–80%) and a factor of 2d load imbalance in practical scenarios. Our analysis also indicates both are suitable in a bandwidth-limited regime and that the direct algorithm is scalable with respect to d. Experimental results including the strong and weak scalability of the algorithms indicates that, for scenarios of practical interest, both are sufficiently scalable to support large-scale SGCT but the direct algorithm has generally better performance, at least by a factor of 2 in most cases. Hierarchical surplus formation is much less communication intensive, but shows less scalability with increasing core counts. Altering the layout of processes in the process grids and the mapping of processes affects the performance of the 2D SGCT by less than 10%, and affects even less the application part of an SGCT advection application.