This paper uses accounting concepts—particularly the concept of Return on Investment (ROI)—to reveal the quantitative value of scientific research pertaining to a major US cyberinfrastructure project (XSEDE—the eXtreme Science and Engineering Discovery Environment). XSEDE provides operational and support services for advanced information technology systems, cloud systems, and supercomputers supporting non-classified US research, with an average budget for XSEDE of US$20M+ per year over the period studied (2014–2021). To assess the financial effectiveness of these services, we calculated a proxy for ROI, and converted quantitative measures of XSEDE service delivery into financial values using costs for service from the US marketplace. We calculated two estimates of ROI: a Conservative Estimate, functioning as a lower bound and using publicly available data for a lower valuation of XSEDE services; and a Best Available Estimate, functioning as a more accurate estimate, but using some unpublished valuation data. Using the largest dataset assembled for analysis of ROI for a cyberinfrastructure project, we found a Conservative Estimate of ROI of 1.87, and a Best Available Estimate of ROI of 3.24. Through accounting methods, we show that XSEDE services offer excellent value to the US government, that the services offered uniquely by XSEDE (that is, not otherwise available for purchase) were the most valuable to the facilitation of US research activities, and that accounting-based concepts hold great value for understanding the mechanisms of scientific research generally.

This paper explores the financial effectiveness of a national advanced computing support organization within the United States (US) called the eXtreme Science and Engineering Discovery Environment (XSEDE). XSEDE was funded by the National Science Foundation (NSF) in 2011 to manage delivery of advanced computing support to researchers in the US working on non-classified research. In this paper, we describe the methodologies employed to calculate the return on investment (ROI) for governmental expenditures on XSEDE and present a lower bound on the US government’s ROI for XSEDE from 2014 to 2020. For each year of the XSEDE project considered, XSEDE delivered measurable value to the US that exceeded the cost incurred by the Federal Government to fund XSEDE. That is, the US Federal Government’s ROI for XSEDE is at least 1 each year. Over the course of the study period, the ROI for XSEDE rose from 0.99 to 1.78. This increase was due partly to our ability to assign a value to more and more of XSEDE’s services over time and partly to the value of certain XSEDE services increasing over time. From 2014 to 2020, XSEDE offered an ROI of more than $1.5 in value for every $1.0 invested by the US Federal Government. Because our estimations were very conservative, this figure represents the lower bound of the value created by XSEDE. The most important part of ”returns” created by XSEDE are the actual outcomes it enables in terms of education, enabling new discoveries, and supporting the creation of new inventions that improve quality of life. In future work we will use newly developed accounting methodologies to begin assessing the value of the outcomes of XSEDE.

Scientific applications built on wide-area distributed systems such as emerging cloud based architectures and the legacy grid computing infrastructure often struggle with user adoption even though they succeed from a systems research perspective. This paper examines the coupling of user-centered design processes with modern distributed systems. Further in this paper, we describe approaches for conceptualizing a product that solves a recognized need: to develop a data gateway to serve the data management and research needs of experimentalists of electron microscopes and similar shared scientific instruments in the context of a research service laboratory. The purpose of the data gateway is to provide secure, controlled access to data generated from a wide range of scientific instruments. From the functional perspective, we focus on the basic processing of raw data that underlies the lab’s "business" processes, the movement of data from the laboratory to central access and archival storage points, and the distribution of data to respective authorized users. Through the gateway interface, users will be able to share the instrument data with collaborators or copy it to remote storage servers. Basic pipelines for extracting additional metadata (through a pluggable parser framework) will be enabled. The core contribution described in this paper, building on the aforementioned distributed data management capabilities, is the adoption of user-centered design processes for developing the scientific user interface. We describe the user-centered design methodology for exploring user needs, iteratively testing the design, learning from user experiences, and adapting what we learn to improve design and capabilities. We further conclude that user-centered design is, in turn, best enabled by an adaptable distributed systems framework. A key challenge to implementing a user-centered design is to have design tools closely linked with a software system architecture that can evolve over time while providing a highly available data gateway. A key contribution of this paper is to share the insights from crafting such an evolvable design-build-evaluate-deploy architecture and plans for iterative development and deployment.

In recent years, considerable attention has been given to assessing the value of investments in cyberinfrastructure (CI). This paper includes a survey of current methods for the assessment of financial returns on investment (ROI) in CI. Applying the financial concept of ROI proves challenging with regard to a service that, in most academic environments, does not generate a "sold amount" such as one would find in the buying and selling of stocks. The paper concludes with a discussion of future research directions and challenges in the assessment of financial ROI in CI. This work is intended less as a definitive guide than as a starting point for further exploration in the assessment of CI's value for scientific research.

The importance of physical security in a data center cannot be emphasized enough. Poor physical security would not only be out of compliance with government regulations, but also present an incredible risk to the integrity of the machines in question. Physical security means a set of policies, precautions, and practices must be adopted to avoid unauthorized access and manipulation of a data center’s resources. Physical security practices in data centers cover a great number of steps to follow, such as locking up the server room, setting up surveillance, using rack mount servers, and many other security protocols. Existing physical security programs use traditional learning methods such as online classes, books and onsite training. However, these materials have limitations to provide trainees practical and hands-on experiences that are based on procedural tasks. We developed CiSE-ProS (Cyberinfrastructure Security Education for Professionals and Students) VR, a virtual reality training application that supports students in learning physical security principles through various procedural tasks in a virtual data center environment. Since physical security training consists of various procedural tasks, CiSE-ProS VR could be an effective environment which can support people through embodied scenario-based activities. We investigate how a virtual reality training tool would benefit a trainee in remembering sequential tasks of physical security in a data center in comparison to a training video. The pilot study shows that CiSE-ProS VR enhances trainees’ procedural memory formation of physical security concepts in the data center by allowing them to access the data center facility and dynamically interact with individual items in the data center.

Whether your interests lie in scientific arenas, the corporate world, or in government, you have certainly heard the praises of big data: Big data will give you new insights, allow you to become more efficient, and/or will solve your problems. While big data has had some outstanding successes, many are now beginning to see that it is not the Silver Bullet that it has been touted to be. Here our main concern is the overall impact of big data; the current manifestation of big data is constructing a Maginot Line in science in the 21st century. Big data is not "lots of data" as a phenomena anymore; The big data paradigm is putting the spirit of the Maginot Line into lots of data. Big data overall is disconnecting researchers and science challenges. We propose No-Boundary Thinking (NBT), applying no-boundary thinking in problem defining to address science challenges.

Web 2.0-style services and capabilities collectively define a comprehensive\break distributed computing environment that may be considered an alternative or supplement to existing Grid computing approaches for e-Science. Web 2.0 is briefly summarized as building upon network-enabled, stateless services with simple message formats and message exchange patterns to build rich client interfaces, mash-ups (custom, composite, Web applications), and online communities. In this chapter, we review several of our activities in these areas: service architectures for chemical informatics; Web 2.0 approaches for managing real-time data from online experiments; management and federation of digital entities and their metadata obtained from multiple services; and the use of tagging and social bookmarking to foster scientific networking at minority serving institutions. We conclude with a discussion of further research opportunities in the application of Web 2.0 to e-Science.

The financial, functional and educational returns from investing in the development of remote access services are well known and widely recognised.State of the art high performance laboratory instrumentation, such as high flux X-ray diffraction systems and powerful electron microscopes, is increasingly expensive and too costly to replicate in multiple locations.Not only is there the high initial capital cost, there is the on-going burden of technical staffing and specialised maintenance costs.Remote access services would maximise returns on the high construction and operating costs of landmark national research facilities, such as synchrotrons and neutron sources, as well as more conventional laboratory facilities.A significant disincentive to developing 'custom-built' remote access systems, is that there is a high coding overhead that may well reproduce functionality already provided by an instrument manufacturer.Recognition of this disincentive is reflected, at least in part, in the cur-rent preference for the use of the remote desktop [1-4] approach to providing remote access to instruments at major facilities [5].A significant advantage of the custom built interface approach however, is that the actions of the remote instrument user may be tightly controlled, while at the same time services outside the desktop environment can be provided to offer a richer operating environment.Communication between the user and the facility may be adjusted to suit prevailing bandwidth constraints.Labour invested in constructing a remote access system may be subsequently leveraged if a framework approach is adopted, and used to support modules and plug-ins.The Common Instrument Middleware Architecture [6-9] (CIMA) project is a pioneering NSF Middleware Initiative project to research and define a consistent and re-usable middleware framework to enable and embed instruments as addressable Web and Grid resources.The framework provides a capability for reuse, and plugins provide service specificity.

The notions of ubiquitous computing and networking, and the increasing availability of compact, low power sensing technologies naturally lead to the idea that we can expect to see large numbers of sensors embedded in the fabric of our everyday lives and that these could form a "Real World Web" of real-time data sources about our world.Implications for e-Research include sharing of real-time data from scientific instruments and aggregation of many types of information to answer complex questions as they arise.The World Wide Web was made possible through a combination of the increasing availability of the Internet, simple network protocols, and open content and delivery standards.The Real World Web can only grow and become self-sustaining if we pay attention to these same core design values of simplicity and openness.Recent developments in cloud computing and in Web 2.0 technologies and design stances provide enablers from which to build the Real World Web, but in addition, require a shift in thinking away from a classical Web services and layered standards model.This paper explores these issues and the role of the Real World Web as a paradigm for sharing instruments, sensors and other real-time data sources in e-Research collaborations.

The Common Instrument Middleware Architecture (CIMA) model for Web services based monitoring of remote scientific instruments is being extended and enhanced to provide a capability for remote instrument control. X-ray diffraction has been selected as an ideal domain for prototype development, with the goal being a comprehensive and feature rich portal system for access to remote instruments and their data. The system has two principle components, one of which serves the instrument and data, and the second serves the client user. Plugin modules are used to provide flexibility and re-use, and the notion of plugin control is being developed. The architecture supports remote access to multiple instruments from a single portal. The use of Web 2.0 Pushlet and AJAX technologies has been introduced for push based portlet refresh and updating. An X3D based 3D virtual representation of the instrument provides data collection simulation and (pseudo) real time instrument representation.

The Common Instrument Middleware Architecture (CIMA) aims at integrating instruments and sensors into computing and storage Grids through a standard interface methodology that existing and future instrument facilities can provide. This instrument interface provides a base which data acquisition and reduction applications can rely on as the design of the underlying facilities evolves and changes. After connecting instruments and sensors in a reliable and secure manner to applications, the next step is to integrate instruments, data acquisition, reduction and analysis applications with community provided resources such as application servers, temporary and archival storage. Practices and workflows at the instrument facilities also need to be captured and presented to users. Portals provide a natural approach to a ‘community’ interface to instrument and sensor facilities. Here we describe the CIMA Crystallography portal, a portal based on GridSphere that provides a user interface to equipment and workflows for a global federation of crystallography laboratories. Copyright © 2007 John Wiley & Sons, Ltd.

A two component portal system is being developed for collaborative remote instrument and data control and monitoring. The system builds on and enhances the Common Instrument Middleware Architecture (CIMA) model for Web services based monitoring of remote scientific instruments and sensors. The architecture supports remote access to multiple instruments from a single portal.. Plugin modules are used to provide flexibility and re-use, and the notion of plugin control is being developed. The use of Web 2.0 Pushlet and AJAX technologies has been introduced for push based portlet refresh and updating. An X3D based 3D virtual representation of the instrument provides data collection simulation and (pseudo) real time instrument representation. An important component of the system is a Webs services driven portlet for collaborative image viewing

An infrastructure for remote instrument access, data acquisition and data management is being developed for e-Research. The Common Instrument Middleware Architecture (CIMA) is being used to provide a scalable and extensible basis for the cyberinfrastructure, and X-ray diffraction is targeted as an ideal development domain. Australian research is enhancing the CIMA model to enable federated Grid storage via SRB, and the use of the Kepler workflow system. Kepler has been introduced to enable automated data management, and the facile extraction and generation of instrument and experimental metadata. The system permits real-time deposition of experimental data into an SRB data store using a schema that allows for searching on the basis of metadata or user supplied annotations, image file previewing and data management and download. The CCLRC scientific metadata model is being adopted for metadata definition. In addition to monitoring, CIMA is being further extended to support instrument control, and is being embedded as a component in a feature rich portal for remote instrument access. The architecture supports remote access to multiple instruments from a single portal. The use of Pushlet and AJAX technologies has been introduced for push based portlet refresh and updating. An X3D based 3D virtual representation of the instrument provides data collection simulation and (pseudo) real time instrument representation. A tool for multi-user collaborative image evaluation is being developed for the infrastructure system.

The Common Instrument Middleware Architecture (CIMA) project, supported by the NSF Middleware Initiative, aims at making scientific instruments and sensors remotely accessible by providing a general solution for services and user interfaces to remotely access data from instruments and to remotely monitor experiments. X-ray crystallography is one of several motivating applications for the development of CIMA. Data such as CCD frames and sensor readings may be accessed by portals through middleware services as they are being acquired or through persistent archives. CIMA software may be used to federate online instruments in multiple labs, so this project must also address problems in data management and data sharing. This paper describes a collaboration between the CIMA and the Open Grid Computing Environments (OGCE) project to enable remote users to monitor instruments and interact with data gathered from CIMA-enabled crystallography laboratories through various Web portal components (portlets) running within a standards-compliant portal container. We also discuss an approach taken to develop portlets that use Web services for data management and solutions for managing distributed identity and access control

The Common Instrument Middleware Architecture (CIMA) model is being adapted and extended for a remote instrument access system being developed as an Australian eScience project. Enhancements include the use of SRB federated Grid storage infrastructure, and the introduction of the Kepler workflow system to automate data management and provide facile extraction and generation of instrument and experimental metadata. The SRB infrastructure will in effect underpin a Grid enabled network of X-ray diffraction instruments. The CIMA model is being further extended to include instrument control, and is being embedded as a component in a feature rich GridSphere portal based system for remote access.

A portal is a Web-based application that acts as an entry point to distributed resources. Individual portlets in a portal can be used to integrate information from a variety of back-end Web services. However, when Web services are deployed, they are available to unintended clients not related to the portal so a general solution for authorizing access to them is needed that is integrated with the portal’s own authentication and authorization mechanisms. This paper investigates the feasibility of an implementation of a general purpose solution for authorization between portlets and their back end Web services based on Privilege and Role Management Infrastructure Standards (PERMIS) which uses Web services security standards such as WSSecurity and SAML. This solution is also appropriate for authorization across organizational boundaries supporting the inclusion of service resources to a portal which are contributed by many different organizations. A motivating example of instrument sharing based on the CIMA remote instrument access protocol is presented.

Instruments and sensors are the primary sources of data driving science and the development and refinement of theory. A critical component of eresearch yet to be clarified is the role of instruments in cyberinfrastructure. Are instruments, sensors and other real-time data sources to be mediated by file systems, or can they be fully integrated into computing and storage grids with appropriate protocol standards? Our position is that they can and must become regular grid resources and that standards for doing so represent an important research topic. Our approach, the Common Instrument Middleware Architecture, and a model application, X-ray crystallography, are described here in order to stimulate a broader discussion of requirements for grid-enabling instruments and sensors.