How to effectively handle heterogeneous data sources is one of the main challenges in the design of large-scale research computing platforms to collect, analyze and integrate data from IoT sensors. The platform must seamlessly support the integration of myriads of data formats and communication protocols, many being introduced after the platform has been deployed. Edge gateways, devices deployed at the edge of the network near the sensors, communicate with measurement stations using their proper protocol, receive and translate the messages to a standardized format, forward the data to the processing platform and provide local data buffering and preprocessing. In this work we present the TDM Edge Gateway architecture, which we have developed to be used in research contexts to meet the requirements of being self-built, low-cost, and compatible with current or future connected sensors. The architecture is based on a microservice-oriented design implemented with software containerization and leverages publish/subscribe Inter Process Communication to ensure modularity and resiliency. Costs and construction simplicity are ensured by adopting the popular Raspberry Pi Single Board Computer. The resulting platform is lean, flexible and easy to expand and integrate. It does not pose constraints on programming languages to use and relies on standard protocols and data models.

HL7 version 2 is the most popular messaging standard for clinical systems interoperability. Most of the tools for messaging management are Java or .NET based, while Python programming language lacks of comparable solutions. This paper describes HL7apy, an open-source HL7 v2 compliant messaging library, written in Python. The library offers means to create, parse, navigate and validate messages. As an example application, we present a full implementation of the IHE Patient Demographics Query ITI-21 transaction. The resulting module has been integrated in GNU Health, a popular open-source Hospital Information System.

The increasing diusion of data acquisition systems paves the way to traceability based process management and de nition. In the clinical context, IHE formalizes the reference guidelines, periodically enhanced to re ect processes evolution. In this work we describe how we have modeled the phlebotomy process following the IHE references and best-practices to obtain a fully traceable work ow. The work has resulted in two new transactions for the IHE LBL pro le, describing samples containers production and samples collection. The complete work ow has been im- plemented and successfully tested in real clinical environ- ments. The traceability data acquired have then been stud- ied using Process Mining techniques to compare the pro- duction model with idealized work ow and guide further developments.

The increasing diffusion of data acquisition systems paves the way to traceability based process management and definition.In the clinical context, IHE formalizes the reference guidelines, periodically enhanced to reflect processes evolution.In this work we describe how we have modeled the phlebotomy process following the IHE references and best-practices to obtain a fully traceable workflow.The work has resulted in two new transactions for the IHE LBL profile, describing samples containers production and samples collection.The complete workflow has been implemented and successfully tested in real clinical environments.The traceability data acquired have then been studied using Process Mining techniques to compare the production model with idealized workflow and guide further developments.

HL7 version 2 is the most popular messaging standard for clinical systems interoperability. Most of the tools for messaging management are Java or .NET based, while Python programming language lacks of comparable solutions. This paper describes HL7apy, an open-source HL7 v2 compliant messaging library, written in Python. The library offers means to create, parse, navigate and validate messages. As an example application, we present a full implementation of the IHE Patient Demographics Query ITI-21 transaction. The resulting module has been integrated in GNU Health, a popular open-source Hospital Information System.

Although the use of virtual environments provided by cloud computing infrastructures is gaining consensus from the scientific community, running applications in these environments is still far from reaching the maturity of more usual computing facilities such as clusters or grids. Indeed, current solutions for managing virtual environments are mostly based on centralized approaches that barter large-scale concerns such as scalability, reliability and reactivity for simplicity. However, considering current trends about cloud infrastructures in terms of size (larger and larger) and in terms of usage (cross-federation), every large-scale concerns must be addressed as soon as possible to efficiently manage next generation of cloud computing platforms.In this work, we propose to investigate an alternative approach lever-aging DIStributed and COoperative mechanisms to manage Virtual EnviRonments autonomicallY (DISCOVERY). This initiative aims at overcoming the main limitations of the traditional server-centric solutions while integrating all mandatory mechanisms into a unified distributed framework. The system we propose to implement, relies on a peer-to-peer model where each agent can efficiently deploy, dynamically schedule and periodically checkpoint the virtual environments they manage. The article introduces the global design of the DISCOVERY proposal and gives a preliminary description of its internals.

Although the use of virtual environments provided by cloud computing infrastructures is gaining consensus from the scientific community, running applications in these environments is still far from reaching the maturity of more usual computing facilities such as clusters or grids. Indeed, current solutions for managing virtual environments are mostly based on centralized approaches that barter large-scale concerns such as scalability, reliability and reactivity for simplicity. However, considering current trends about cloud infrastructures in terms of size (larger and larger) and in terms of usage (cross-federation), every large-scale concerns must be addressed as soon as possible to efficiently manage next generation of cloud computing platforms. In this work, we propose to investigate an alternative approach leveraging DIStributed and COoperative mechanisms to manage Virtual EnviRonments autonomicallY (DISCOVERY). This initiative aims at overcoming the main limitations of the traditional server-centric solutions while integrating all mandatory mechanisms into a unified distributed framework. The system we propose to implement, relies on a peer-to-peer model where each agent can efficiently deploy, dynamically schedule and periodically checkpoint the virtual environments they manage. The article introduces the global design of the DISCOVERY proposal and gives a preliminary description of its internals.

For a large class of scientific data analysis applications it is becoming important, due to the sheer size of datasets, to have the option to perform the analysis directly where the data are stored, rather than on remote computational clusters. A possible strategy is the use of virtual clusters, thus guaranteeing a high degree of isolation from the underlying physical computational structure, and a very compact initial description. Deploying, saving and restoring HPC dedicated virtual clusters introduces, however, a different class of requirements on the virtual machines managing infrastructure, in particular for what concerns storage I/O requirements, whose scalability boundaries are easily reached. Here we discuss an alternative approach based on a storage model that leverages the WORM (write once, read many) character of the data used by VM management to increase, in a scalable way, the aggregate data bandwidth available to virtual cluster level operations and provide preliminary results indicating that it is a viable solution.

High Performance Computational Clusters are, in general, rather rigid objects that present to their user a limited number of degrees of freedom related, usually, only to the specification of the resources requested and to the selection of specific applications and libraries. While in standard production environments this is reasonable and actually desirable, it can become an hindrance when one needs a dynamic and flexible computational environment, for instance for experiments and evaluation, where very different computational approaches, e.g., map-reduce, standard parallel jobs and virtual HPC clusters need to coexist on the same physical facility. In this paper we will present our efforts to address some of these challenges while maintaining a unified cluster management environment.

The rapid growth in the throughput to cost ratio of experimental data production technologies is generating vast amounts of scientific data, often organized into "large" objects (genomes, bio-images) exhibiting complex internal structures. Frequently, datasets must be shared between multiple research groups interested not only in the final results, but also in how they are produced. The practical difficulties of moving terabytes or more of data across the network, as well as the need to maintain a clear separation between software stack and storage infrastructure, are thus raising interest in the use of virtual clusters for HPC and data intensive applications. In this paper we employ a MapReduce implementation of an image analysis pipeline used by deep sequencing platforms to analyse different virtual cluster scenarios and their impact on system performance.

A systematic study of issues related to suspending, migrating and resuming virtual clusters for data-driven HPC applications is presented. The interest is focused on nontrivial virtual clusters, that is where the running computation is expected to be coordinated and strongly coupled. It is shown that this requires that all cluster level operations, such as start and save, should be performed as synchronously as possible on all nodes, introducing the need of barriers at the virtual cluster computing meta-level. Once a synchronization mechanism is provided, and appropriate transport strategies have been setup, it is possible to suspend, migrate and resume whole virtual clusters composed of "heavy" (4 GB RAM, 6 GB disk images) virtual machines in times of the order of few minutes without disrupting parallel computation - albeit of the MapReduce type - running inside them. The approach is intrinsically parallel, and should scale without problems to larger size virtual clusters. (c) 2010 Elsevier B.V. All rights reserved.

Virtualization is an essential enabling technology for building and controlling computing frameworks that can dynamically adapt available physical resources to transient tasks such as the temporary creation of a virtual computing center tailored to the needs of a virtual organization. In this paper we will report on our strategy for the creation of virtual computer clusters based on standard Service-Oriented Architecture (SOA) and hosts virtualization technologies. We will describe our infrastructure designed for dynamical allocation of resources to applications via a general control plane based on workflows of coordinated web services. The control plane is based on logically independent services that are responsible for physical resource management and virtual nodes deployment. The control plane is also responsible for operations on virtual clusters such as their creation, startup and control.

Building large computational facilities requires scalable and flexible deployment tools that can cope with massive loads. Classical installation methods are not very flexible, since they are usually limited in the number of OS supported, rely on transfer solutions that impose constraints on network topology, and do not scale very well. Here we describe HaDeS (Hardware Deployment System), a new deployment system for large scale installation designed to be agnostic with respect to the network topology and the OS deployed and to scale with the number of nodes being deployed.

Virtualization is an essential enabling technology for the construction and control of computing facilities that can dynamically adapt available physical resources to transient tasks such as the temporary creation of a virtual computing center tailored to the needs of a virtual organization. In this paper we will describe our strategy for the creation of virtual computer clusters based on standard SOA and hosts virtualization technologies and we will report on our ongoing work on the application of the latter to the deployment and management of a research cluster with 140 dual core cpu. Our deployment mechanism, as well as the system management, is delegated to a control plane based on workflows of coordinated web services. The control plane is based on two logically independent modules, the first is responsible of the physical resources and the deployment on the hardware of virtual Xen hypervisor images, while the second manages operations on virtual clusters such as their creation, startup and control. Low level operations e.g., the control of a running image on a given computational host - are directly provided by atomic web services, in this specific case a WSRF service running in the dom0 of each participating physical Xen host, while all logic above that level is implemented as BPEL scripts.

Current bioinformatics applications require both management of huge amounts of data and heavy computation: fulfilling these requirements calls for simple ways to implement parallel computing. MapReduce is a general-purpose parallelization technology that appears to be particularly well adapted to this task. Here we report on its application, using its open source implementation Hadoop, to two relevant algorithms: BLAST and GSEA. The first is characterized by streaming computation on large data sets, while the second requires a multi-pass computational strategy on relatively small data sets. The analysis of these algorithms is relevant to a wide class of complex applications, e.g., structural genomics and genome-wide association studies, since they typically contain a mixture of these two computational flavors. Our results are very promising and indicate that the framework could have a wide range of bioinformatics applications while maintaining good computational efficiency, scalability and ease of maintenance.