Infectious diseases remain a major threat to human societies. During the recent COVID-19 pandemic, mathematical modeling and extensive computer simulations proved highly effective in supporting public health experts and decision makers. Despite these advances, the full potential of modern modeling approaches and digital technologies has not yet been realized. Many critical tasks – including expert consultations, model execution, scenario analyses, report preparation, and result communication – still relied heavily on manual, human-driven processes with each manual interaction introducing avoidable delays and limiting responsiveness during rapidly evolving outbreaks. Pandemic preparedness should opt for automated workflows and seamlessly integrated software modules that can improve pandemic mitigation capabilities by substantially reducing response times. For this step, we require robust and flexible computational infrastructure capable of supporting heterogeneous hardware and continuously evolving infectious-disease models. In addition, data sources need to be dynamically integrated. Managing such demands needs infrastructure that supports automated high-performance computing (HPC) workflows. Beyond computational performance, software infrastructure must ensure secure user and data management to comply with data-protection regulations and provide clear, transparent presentation of results to both decision makers and the public. Meeting the aforementioned challenges requires tight integration of state-of-the-art scientific software with modern, scalable infrastructure that can leverage supercomputing resources when necessary. For rapid deployment in future epidemic or pandemic scenarios, adherence to the FAIR principles for research software is critical to ensure reusability and sustainability.

When faced with the challenge of now- and forecasting infectious diseases, multiple data sources and state-of-the-art models have to be considered. Automatic aggregation, processing, and publishing to relevant data sinks is paramount to achieving consistent, reproducible, and timely results given daily-reported data. To facilitate scientific collaboration and reproducibility of workflows, open and extensible architectures for compute pipelines are required.In this research, we devise an architecture realizing the seamless management and processing of reproducible pipelines. Our case-study is a daily pipeline for nowcasting the state of SARS-CoV-2 in Germany based on public data and state-of-the-art models implemented in the simulation software MEmilio. The results of our pipeline are pushed to ESID (Epidemiological Scenarios for Infectious Diseases), a user interface to epidemiological simulations.To realize the given pipeline, a workflow management system is required to ensure pipeline processing and secure access to multiple heterogeneous data storages. For this purpose, we based our work on an open-source workflow management system - Apache Airflow, which provides the orchestration, coordination and management of complex connected tasks. S3 is utilized as an intermediate data storage service for sharing data between workflow steps and persisting experiment output. We provide a comprehensive view on our work on automated, end-to-end and reproducible pipelines, with detailed commentary on use case, and its realization.

In Remote Sensing (RS) classification, generalization ability is one of the measure that characterizes the success of Machine Learning (ML) models, but is often impeded by the scarse availability of annotated training data. Annotated RS samples are expensive to obtain and can present large disparities when produced by different annotators. In this paper, we utilize Few-Shot Learning (FSL) with meta-learning to address the challenge of generalization using limited amount of training information. The data used in this paper is leveraged from different datasets that have diverse distributions, that means distinct feature spaces. We tested our approach on publicly available RS benchmark datasets to perform few-shot RS image classification using meta-learning. The results of the experiments suggest that our approach is able to generalize well on the unseen data even with limited number of training samples and reasonable training time.

Computer-aided diagnostics is helping the medical experts for fast diagnostics, using machine learning and representation learning techniques. Various types of diagnostics are using the assistance of machine learning approaches including endoscopy. In this paper, a transfer learning based bagging approach is investigated for endoscopy images analysis. Bagging is used to fine-tune several instances of the deep learning model with 70% of data in each bag. These all models of deep learning are combined to generate a single prediction using majority voting and neural-network-based decision approach. The best approach resulted in an F1-score of 0.60 on the EndoTech 2020 dataset having 23 abnormalities in the GI-Tract.

Abstract. Scientific computing applications involving complex simulations and data-intensive processing are often composed of multiple tasks forming a workflow of computing jobs. Scientific communities running such applications on computing resources often find it cumbersome to manage and monitor the execution of these tasks and their associated data. These workflow implementations usually add overhead by introducing unnecessary input/output (I/O) for coupling the models and can lead to sub-optimal CPU utilization. Furthermore, running these workflow implementations in different environments requires significant adaptation efforts, which can hinder the reproducibility of the underlying science. High-level scientific workflow management systems (WMS) can be used to automate and simplify complex task structures by providing tooling for the composition and execution of workflows – even across distributed and heterogeneous computing environments. The WMS approach allows users to focus on the underlying high-level workflow and avoid low-level pitfalls that would lead to non-optimal resource usage while still allowing the workflow to remain portable between different computing environments. As a case study, we apply the UNICORE workflow management system to enable the coupling of a glacier flow model and calving model which contain many tasks and dependencies, ranging from pre-processing and data management to repetitive executions in heterogeneous high-performance computing (HPC) resource environments. Using the UNICORE workflow management system, the composition, management, and execution of the glacier modelling workflow becomes easier with respect to usage, monitoring, maintenance, reusability, portability, and reproducibility in different environments and by different user groups. Last but not least, the workflow helps to speed the runs up by reducing model coupling I/O overhead and it optimizes CPU utilization by avoiding idle CPU cores and running the models in a distributed way on the HPC cluster that best fits the characteristics of each model.

The progress of remote sensing technologies leads to increased supply of high-resolution image data. However, solutions for processing large volumes of data are lagging behind: desktop computers cannot cope anymore with the requirements of macro-scale remote sensing applications; therefore, parallel methods running in High-Performance Computing (HPC) environments are essential. Managing an HPC processing pipeline is non-trivial for a scientist, especially when the computing environment is heterogeneous and the set of tasks has complex dependencies. This paper proposes an end-to-end scientific workflow approach based on the UNICORE workflow management system for automating the full chain of Support Vector Machine (SVM)-based classification of remotely sensed images. The high-level nature of UNICORE workflows allows to deal with heterogeneity of HPC computing environments and offers powerful workflow operations such as needed for parameter sweeps. As a result, the remote sensing workflow of SVM-based classification becomes re-usable across different computing environments, thus increasing usability and reducing efforts for a scientist.

This manuscript describes a new workflow implementations based on the state-of-theart UNICORE middleware suite to automate and simplify complex task structure across distributed and heterogeneous computing environments. The authors take as a case study an example of coupling two numerical models, one for continuum dynamics of a glacier (Elmer Ice) and a discrete model for calving processes (HiDEM), to demonstrate the progress made by this new implementation. This is an interesting paper that describes a new coupling method that will make the lives of scientists that want to couple different models, even on different platforms, much easier, if the implementation is

Scientific computing applications involving complex simulations and data-intensive processing are often composed of multiple tasks forming a workflow of computing jobs. Scientific communities running such applications on distributed and heterogeneous computing resources find it cumbersome to manage and monitor the execution of these tasks. Scientific workflow management systems (WMS) can be used to automate and simplify complex task structures by providing tooling for the composition and execution of workflows across distributed and heterogeneous computing environments. As a case study, we 5 apply the UNICORE workflow management system to a formerly hard-coded coupling of a glacier sliding and calving simulation that contains many tasks and dependencies, ranging from pre-processing and data management to repetitive executions in heterogeneous high-performance computing (HPC) resource environments. Using the UNICORE workflow management system, the composition, management, and execution of the glacier modelling workflow becomes easier with respect to usage, monitoring, maintenance, re-usability, portability, and reproducibility in different environments and by different user groups. 10

The progress of remote sensing technologies leads to increased supply of high-resolution image data. However, solutions for processing large volumes of data are lagging behind: desktop computers cannot cope anymore with the requirements of macro-scale remote sensing applications; therefore, parallel methods running in High-Performance Computing (HPC) environments are essential. Managing an HPC processing pipeline is non-trivial for a scientist, especially when the computing environment is heterogeneous and the set of tasks has complex dependencies. This paper proposes an end-to-end scientific workflow approach based on the UNICORE workflow management system for automating the full chain of Support Vector Machine (SVM)-based classification of remotely sensed images. The high-level nature of UNICORE workflows allows to deal with heterogeneity of HPC computing environments and offers powerful workflow operations such as needed for parameter sweeps. As a result, the remote sensing workflow of SVM-based classification becomes re-usable across different computing environments, thus increasing usability and reducing efforts for a scientist.

Classification of remote sensing images often use Support Vector Machines (SVMs) that require an n-fold cross-validation phase in order to do model selection. This phase is characterized by sweeping through a wide set of parameter combinations of SVM kernel and cost parameters. As a consequence this process is computationally expensive but represents a principled way of tuning a model for better accuracy and to prevent overfitting together with regularization that is in SVMs inherently solved in the optimization. Since the cross-validation technique is done in a principled way also known as 'gridsearch', we aim at supporting remote sensing scientists in two ways. Firstly by reducing the time-to-solution of the cross-validation by applying state-of-the-art parallel processing methods because the sweep of parameters and cross-validation runs itself can be nicely parallelized. Secondly by reducing manual labour by automating the parallel submission processes since manually performing cross-validation is very time consuming, unintuitive, and error-prone especially in large-scale cluster or super-computing environments (e.g., batch job scripts, node/core/task parameters, etc.).

The XSEDE project seeks to provide “a single virtual system that scientists can use to interactively share computing resources, data and experience.” The potential compute resources in XSEDE are diverse in many dimensions, node architectures, interconnects, memory, local queue management systems, and authentication policies to name a few. The diversity is particularly rich when one considers the NSF funded service providers and the many campuses that wish to participate via campus bridging activities. Resource diversity presents challenges to both application developers and application platform developers (e.g., developers of gateways, portals, and workflow engines). The XSEDE Execution Management Services (EMS) architecture is an instance of the Open Grid Services Architecture EMS and is used by higher level services such as gateways and workflow engines to provide end users with execution services that meet their needs. The contribution of this paper is to provide a concise explanation and concrete examples of how the EMS works, how it can be used to support scientific gateways and workflow engines, and how the XSEDE EMS and other OGSA EMS architectures can be used by applications developers to securely access heterogeneous distributed computing and data resources.

Emerging challenges for scientific communities are to efficiently process big data obtained by experimentation and computational simulations. Supercomputing architectures are available to support scalable and high performant processing environment, but many of the existing algorithm implementations are still unable to cope with its architectural complexity. One approach is to have innovative technologies that effectively use these resources and also deal with geographically dispersed large datasets. Those technologies should be accessible in a way that data scientists who are running data intensive computations do not have to deal with technical intricacies of the underling execution system. Our work primarily focuses on providing data scientists with transparent access to these resources in order to easily analyze data. Impact of our work is given by describing how we enabled access to multiple high performance computing resources through an open standards-based middleware that takes advantage of a unified data management provided by the the Global Federated File System. Our architectural design and its associated implementation is validated by a usecase that requires massivley parallel DBSCAN outlier detection on a 3D point clouds dataset.

Computing middlewares play a vital role for abstracting complexities of backend resources by providing a seamless access to heterogeneous execution management services. Scientific communities are taking advantage of such technologies to focus on science rather than dealing with technical intricacies of accessing resources. Multi-disciplinary communities often bring dynamic requirements which are not trivial to realize. Specifically, to attain massivley parallel data processing on supercomputing resources which require an access to large data sets from widely distributed and dynamic sources located across organizational boundaries. In order to support this abstract scenario, we bring a combination that integrates UNICORE middleware and the Global Federated File System. Furthermore, the paper gives architectural and implementation perspective of UNICORE extension and its interaction with Global Federated File System space through computing, data and security standards.

In this paper we give a brief overview of the three projects that were chosen for XSEDE-PRACE collaboration in 2014. We begin this paper with an introduction of the XSEDE and PRACE organizations and the motivation for a collaborative effort between these two organizations. We then talk about the three projects that were involved in this collaboration. We provide an overview of the projects themselves and what was in scope for this collaboration. We also outline the hurdles and issues faced during this unique collaborative effort and also discuss the benefits the projects derived from this collaboration. We finally outline the future steps envisioned for XSEDE-PRACE collaborative efforts going forward.

The UltraScan data analysis application is a software package that is able to take advantage of computational resources in order to support the interpretation of analytical ultracentrifugation (AUC) experiments. Since 2006, the UltraScan scientific gateway has been used with ordinary Web browsers in TeraGrid by scientists studying the solution properties of biological and synthetic molecules. Unlike other applications, UltraScan is implemented on a gateway architecture and leverages the power of supercomputing to extract very high resolution information from the experimental data. In this contribution, we will focus on several improvements of the UltraScan scientific gateway that enable a standardized job submission and management to computational resources while retaining its lightweight design in order to not disturb the established workflows of its end-users. This paper further presents a walkthrough of the architectural design including one real installation deployment of UltraScan in Europe. The aim is to provide evidence for the added value of open standards and resulting interoperability enabling not only UltraScan application submissions to resources offered in the US cyber infrastructure Extreme Science and Engineering Discovery Environment (XSEDE), but also submissions to similar infrastructures in Europe and around the world. The use of the Apache Airavata framework for scientific gateways within our approach bears the potential to have an impact on several other scientific gateways too.

In this paper we discuss the implementation of UNICORE in XSEDE. UNICORE is a Grid middleware tool that was identified by XSEDE to further the areas of remote job submission, campus bridging and workflows. We talk about the overall architecture of UNICORE, a typical HPC environment at XSEDE and why UNICORE is a good fit for this environment. We also discuss the initial efforts made by the UNICORE development team as well as XSEDE's Software development team to integrate UNICORE into the XSEDE landscape. We detail how UNICORE went through the XSEDE engineering process and highlight deployment details at XSEDE. We touch upon how UNICORE is beneficial to the HPC user community. In our final section we talk about future efforts to better integrate UNICORE within XSEDE.

Parameter sweeps are used by researchers with scientific domain-specific tools or workflows to submit a large collection of computational jobs whereby each single job of it only varies in certain parts. They require a more fine-grained distribution of jobs across resources, which also raise a significant challenge for efficient resource management in middleware environments that have been not specifically designed to perform parameter sweeps. This paper offers insights into parameter sweep solutions that support multi-disciplinary science environments via abstraction from resource management complexities using middleware. The solutions are based on use case requirements, enable efficient submission, enhanced usability, and standard compliance. We also apply a use case taken from the life science domain to demonstrate usefulness and efficiency of the solutions.

Certain scientific use cases possess complex requirements to have Grid jobs executed in collections where the jobs' request contains only some variation in different parts. These scenarios can easily be tackled by a single job request which abstract this variation and can represent the same collection. The Open Grid Forum (OGF) standards community modeled this requirement through the Job Submission and Description Language (JSDL) Parameter Sweep specification, which takes a modular approach to handle different variations of parameter sweeps (e.g. document and file sweep). In this paper we present the UNICORE server environment implementing this specification build upon its existing JSDL implementation. We also demonstrate the application of UNICORE's parameter sweep extension for optimizing job executions, which are submitted as a sub-activity of a Taverna based scientific workflow. Further we validate our approach by analyzing performance of the workflow with and without using the parameter sweep extension.