Today basically two grid concepts rule the world: service grids and desktop grids. Service grids offer an infrastructure for grid users, thus require notable management to keep the service running. On the other hand, desktop grids aim to utilize free CPU cycles of cheap desktop PCs, are easy to set up, but the availability towards users is limited compared to the service grid. The aim of the EDGeS project is to create ail integrated infrastructure that gathers the advantages of the two grid concepts. A building block of this infrastructure is bridging between the different grid types. In the paper we focus on bridging from desktop grids towards service grids, i.e. making desktop grids able to utilize free service grid resources.
Most existing Grid workflow modeling tools are based on user-defined notations. Lack of standards hinders the collaboration among different Grid-related projects. The work presented in this paper introduces a graphical workflow editor Teuta, which has been implemented based on the latest standard UML 2.0 notations and tailored for specifying Grid workflows based on our Abstract Grid Workflow Language (AGWL). In Teuta, Grid workflows are composed by combining predefined UML modeling elements or user-defined constructs in a hierarchical fashion. Teuta can generates the corresponding AGWL representations and submit them to the ASKALON Grid runtime system for execution. We validate our approach for a real world hydrological application.
The optimization of data access will largely influence the performance of the current and next generation Data Grid systems. Job finishing times are an important factor in the characterization of Grid performance, as the earlier a newly Submitted job is finished, the more the submitter is satisfied.Our focus in this paper is mainly on how the introduction of agents can improve job finishing time on the Data Grid systems. The strategy takes into account the way applications access their data and extend the Grid middleware so that earlier job finishing times can be achieved. The success of this strategy is due mainly to the following two reasons: (i) the scheduler takes into account the job's behaviour, (ii) the proposed agents deliver the necessary files earlier, so the job has to wait less before the execution.
Recent research into the optimization of collective MPI operations has resulted in a wide variety of algorithms and corresponding implementations, each typically only applicable in a relatively narrow scope: on a specific architecture, on a specific network, with a specific number of processes, with a specific data size and/or data-type - or any combination of these (or other) factors. This situation presents an enormous challenge to portable MPI implementations which are expected to provide optimized collective operation performance on all platforms. Many portable implementations have attempted to provide a token number of algorithms that are intended to realize good performance on most systems. However, many platform configurations are still left without well-tuned collective operations. This paper presents a proposal for a framework that will allow a wide variety of collective algorithm implementations and a flexible, multi-tiered selection process for choosing which implementation to use when an application invokes an MPI collective function.
Development of tools [hat assist efficient computing in a distributed environment is a challenging problem in modem and future Grid systems. A registry that aids searching for information about existing components of distributed applications belongs to this class of tools. This work presents the Grid Registry, which is a distributed, scalable and fault-tolerant facility that stores information about structure and semantics of Grid/Web services. A set of performance tests that present the behavior of the registry are also described.
Web Service Resource Framework (WSRF) is a, recent effort of the grid community to facilitate modeling of the stateful services [11]. Design and development of the WSRF service based systems is quite common and there are several emerging WS initiatives, which tries to automate the process of discovery, composition and invocation of such services. The semantic web services are a typical example, showing the potential of how ontological modeling can improve the shortcomings of the service oriented computing. One of the major obstacles in the process is the development of the ontologies, which describe web and grid services. Although, there are numerous standards for modeling semantic services, there are very few frameworks and tools, which can help automate the process of generating the semantic descriptions of services. This article presents a tool, which can semi-automatically generate the OWL-S descriptions for both stateful and Stateless Services based on the Web Service Description Language (WSDL) and corresponding annotations. Such functionality is inevitable in the grid environment hosting a vast number of services, which have to be semantically described in order to enable automated discovery, composition and invocation.
Matlab is a popular interactive computing environment that gives scientists a powerful and easy to use set of analysis and visualization tools. However, Matlab has been criticized for performance limitations on computationally and memory intensive applications. In addition, because of Matlab's single-threaded processing model, long-running processes prevent a user from performing other tasks while computation occurs. We discuss a framework for remote, concurrent execution of tasks in the Matlab environment that address these shortcomings by asynchronously distributing tasks to dedicated high-performance compute servers. We employ a client-server model based on, web-standards and Matlab's built in scripting and Java environment.
The Desktop Grid model harvests the unused CPU cycles of any computer connected. In this paper vie present a concept how the separated Desktop Grids can be used as building blocks for larger scale grids by organizing them in a hierarchical tree. We present a prototype implementation and show the challenges and security considerations we discovered. We describe methods and give solutions how security can be enhanced to satisfy the requirements for real-world deployment.
Grid computing has gone through some generations and as a result only a few widely used middleware architectures remain. Using the tools of these middlewares, various resource brokers have been developed to automate job submission over diffierent grids. Most of the present brokers operate only on a single grid infrastructure, where they have been developed. This taxonomy helps identifying and categorizing the most important properties of brokers within different Resource Management Systems. The result of this work reveals the differences of the examined Resource Brokers, which can enhance a more efficient grid usage and future development.
Grid Infrastructure Monitoring is distinguished as important technique for diagnosing, addressing and preventing problems which may occur in the grids. It is also source of valuable information suitable for scheduling, load balancing and other applications. Currently, infrastructure monitoring is done using specialised toolkits which provide complete solution for single infrastructure and middleware. In large-scale grids, where it is necessary to comprise resources from multiple virtual organisations, it may be necessary to integrate various platforms into single ubiquitous monitored fabric. C-GMA provides tools for such integrations. In this paper, we address the scalability of the C-GMA by proposing a distributed design of C-GMA mediator component. In particular, we examine the properties of Publish/SLibscribe systems for this purpose.
This paper introduces a combination of the existing parallel checkpointing techniques for software heterogeneous ClusterGrid infrastructures. Most of the existing solutions are aiming at supporting application transparency (no checkpoint related code development in application), but some others build middleware transparent (no service modification) solutions. The main contribution of this paper is to introduce a solution providing both application and middleware transparency at the same time. Compatibility and integrity requirements are identified and corresponding conditions are established using Abstract State Machines. The most relevant checkpointing systems are checked against the conditions in order to examine their conformity. Based on the conditions, a novel checkpointing method is defined and a proof of concept checkpointing tool, called TotalCheckpoint (TCKPT) is introduced.
In order to improve reliability and to deal with the high complexity of existing middleware solutions, today's production Grid systems restrict the services to be deployed on their resources. On the other hand end-users require a wide range of value added services to fully utilize these resources. This paper describes a solution how legacy code support is offered as third party service for production Grids. The introduced solution, based on the Grid Execution Management for Legacy Code Architecture (GEMLCA), do not require the deployment of additional applications on the Grid resources, or any extra effort from Grid system administrators. The implemented solution was successfully connected to and demonstrated on the UK National Grid Service.
Bulk Synchronous Parallel ML is a high-level language for programming parallel algorithms. Built upon OCaml and using the BSP model, it provides a safe setting for their implementation, avoiding concurrency related problems (deadlocks, indeterminism). Only a limited set of the features of OCaml can be used in BSML to respect its properties of safety: this paper describes a way to add exception handling to this set by extending and adapting OCaml’s exceptions. The behaviour of these new exceptions and the syntactic constructs to handle them, together with their implementation, are described in detail, and results over an example are given.
In a distributed system where scalability is an issue, the problem of enforcing mutual exclusion often arises in a soft form: the infrequent failure of the mutual exclusion predicate is tolerated, without compromising the consistent operation of the overall system. For instance this occurs when the operation subject to mutual exclusion requires massive use of a shared resource. We introduce a scalable soft mutual exclusion algorithm, based on token passing: one distinguished feature of our algorithm is that instead of introducing an overlay topology we adopt a random walk approach. The consistency of our proposal is evaluated by simulation, and we exemplify its use in the coordination of large data transfers in a backbone based network. This algorithm is studied in the frame of the CoreGRID Institute of Grid Information, Resource and Workflow Monitoring Services, in cooperation with the FORTH Institute, in Greece.
The number of processors embedded on high performance computing platforms is growing daily to satisfy the user desire for solving larger and more complex problems. Scalable and fault-tolerant runtime environments are needed to support and adapt to the underlying libraries and hardware which require a high degree of scalability in dynamic large-scale environments. This paper presents a self-healing network (SHN) for supporting scalable and fault-tolerant runtime environments. The SHN is designed to support transmission of messages across multiple nodes while also protecting against recursive node and process failures. It will automatically recover itself after a failure occurs. SHN is implemented on top of a scalable fault-tolerant protocol (SFTP). The experimental results show that both the latest multicast and broadcast routing algorithms used in SHN are faster and more reliable than the original SFTP routing algorithms.
In this paper, we explore a novel locality optimizing algorithm for developing stream programs in Imagine to sustain high computational ability. Our specific contributions include that we formulate the relationship between streams and kernels as a Data&Computation Matrix (D&C Matrix), and present the key techniques for locality enhancement based on this matrix. The experimental results on five representative scientific applications show that our algorithm can effectively improve the computational intensiveness and avoid the utilization of index streams to achieve high locality in LRF and SRF.
The paper presents a comparative analysis of parallel implementation of the preconditioned conjugate gradient method with the symmetric-successive over-relaxation preconditioner. Two parallel implementations of the matrix solver are compared. The first one is a message-passing version executed on a cluster of workstations. The other one is an efficient version simulated on a novel architecture of dynamically reconfigurable shared memory clusters with a new paradigm of inter-processor communication called communication on the fly. The presented example shows high suitability of the proposed architecture for fine grain numerical computations. It can be very useful in the simulation of physical phenomena described as numerical problems suitable for fine grain parallel execution.
In scientific applications, netCDF was proposed for storing datasets created and used by them to provide portable I/O operations on a wide variety of platforms. Besides, its parallel I/O interface, parallel netCDF, was developed with the help of an MPI-I/O library. To realize the same operations among computers which have different MPI libraries, a remote I/O mechanism of a Stampi library, which is a flexible intermediate library to realize seamless MPI operations both inside a computer and among Computers, has been introduced in some of parallel netCDF functions. This newly implemented mechanism has been evaluated on interconnected PC clusters, and sufficient performance has been achieved with huge amount of data.
Grid infrastructures are reaching their production phase but still we miss a well defined and complete infrastructure testing system for such Grids to fulfill the requirements of both end-users. In the beginning of the paper a short overview of Grid infrastructure testing methods is given followed by a discussion about the key issues of these approaches. Later on we are introducing a General Grid Testing Model to show what sort of Grid tests would be very much required by the grid end-users. In the second half of the paper we give an overview about P-GRADE Grid Portal and we describe how can solve some user oriented Grid testing services offered by the P-GRADE Portal.
