It is clear from the number of e-science projects involving remote access to instruments that remote access and facility sharing is an emerging area of concern. In this chapter we discuss the Common Instrument Middleware Architecture (CIMA) project funded by the US National Science Foundation (NSF) through the NSF Middleware Initiative. CIMA aims at providing a middleware set to facilitate remoting instruments and sensors as community resources and building collaborations around shared facilities. We provide an overview of CIMA, discuss its relationship to some other instrument remote access projects, examine our experiences in developing and applying the architecture and look forward to future development directions for CIMA and its applications.

The Common Instrument Middleware Architecture (CIMA) model for Web services based monitoring of remote scientific instruments is being extended and enhanced to provide a capability for remote instrument control. X-ray diffraction has been selected as an ideal domain for prototype development, with the goal being a comprehensive and feature rich portal system for access to remote instruments and their data. The system has two principle components, one of which serves the instrument and data, and the second serves the client user. Plugin modules are used to provide flexibility and re-use, and the notion of plugin control is being developed. The architecture supports remote access to multiple instruments from a single portal. The use of Web 2.0 Pushlet and AJAX technologies has been introduced for push based portlet refresh and updating. An X3D based 3D virtual representation of the instrument provides data collection simulation and (pseudo) real time instrument representation.

Predefining structures for data preclude the ability to describe certain details effectively, as data is dynamically structured by nature and can be highly diverse and multifaceted. Conventional datastores, such as relational databases, do not conveniently accommodate dynamically varying structures, as frequently modifying database schemas is not feasible. Although, XML databases have been proposed as suitable for such dynamic structured data, these databases suffer from update anomalies. RDF triplestores offer a flexible solution for handling such data, where any property about an entity can be described by a triple consisting of a subject, a predicate, and an object. Data is inherently distributed due to origination points, ownership, and many other reasons. In this dissertation, we develop a distributed triplestore while investigating different approaches for improving the efficiency of query processing. We implement several index structures at a Mediator where each index structure helps to minimize unproductive communications. We show how the addition of each index structure reduces the query response time. We make use of graph-based caches at the Mediator and at individual triplestores to store triples that correspond to the most frequent set of queries. The Mediator cache enables the Mediator to respond to certain queries, while the triplestore caches avoid the need to perform joins between partial results from different triplestores. Sub-graph isomorphism is used to determine whether a cache has sufficient triples to answer a given query. We show how these caches significantly improve querying efficiency.

Data is dynamically structured by nature and can be highly diverse and multifaceted. Often, such diverse and complex information needs to be linked. Conventional datastores, such as relational databases, do not conveniently accommodate dynamically varying structures, as frequently modifying database schemas is not feasible. RDF triplestores offer a flexible solution for handling such data, where any property about an entity can be described by a triple having a subject, a predicate, and an object. Also, data is inherently distributed due to origination points, ownership and many other reasons. Furthermore, storing data in triplestores gives rise to the need to distribute data due to the large number of triples that would result by migrating existing data from a database, for example. In this paper, we present our work on designing index structures in order to facilitate efficient querying of a distributed triplestore (DTS). The distributed querying algorithm in DTS makes use of a sub-graph isomorphism approach to eliminate traversing edges between triplestores that does not have the potential to produce any results. We show that our triplestore has equivalent performance as 3Store when used in a non-distributed mode. Our performance tests in the distributed mode show that the indexes improve efficiency of querying.

XML has become the de facto standard for semi-structured data across a variety of domains. XML is generally considered to be slow for scientific data and therefore only used for control information. This approach puts a burden on software developers as they have to be familiar with two type systems; one for control and the other for data. Also, this approach prevents scientific data from being "first class members" in XML, especially in a web services framework. A type system, such as binary XML targeted towards scientific data will be flexible enough to represent the control information and would efficiently encode such scientific data. In order to get wide acceptance for such a format, it should be beneficial to real-life scientific applications and also should be usable across many different applications and programming languages. In this paper we extend our previous work on Binary XML for Scientific Applications (BXSA) by (1) applying BXSA to the Gyrokinetic Toroidal Code fusion application, and showing that performance is better than HDF5 in our test cases; (2) demonstrating an interoperable Java implementation that is faster than Xerces and Sun's Fast Infoset on common document types; and (3) showing that BXSA is also applicable to business data in addition to scientific data by evaluating its performance on a variety of XML documents against libxml2 and expat. These results show that BXSA is applicable for many scenarios.

Problem Solving Environments (PSE) in scientific computing domains require the ability to couple High Performance Computing (HPC) components. A PSE facilitates coupling of tasks or computations in order to aid a scientist in finding a solution to a problem or at least getting closer to a solution. Reflection capabilities are required in order to effectively dynamically couple these components. Reflection facilitates adaptive behavior such as rebinding calls to different functions at run-time, or integrating flexible interpreted languages with compiled languages such as C++ or Fortran. Currently, however, reflection is not available in languages commonly used in high performance computing. While there have been several attempts to incorporate reflection into C++, all of them are either intrusive or are not fully compliant with the C++ standard. In this paper, we present a number of use cases for reflective programming, and show how it can be efficiently and robustly implemented in languages such as C++. Our implementation uses code generation to add metadata, and is fully compliant with the standard C++ specification. We compare the overhead of reflection with languages such as Java, and show that our overhead is acceptable for many scenarios. Our reflection library is open-source, and is available at http://www.extreme.indiana.edu/reflcpp.

The efficient serialization and deserialization of data is a fundamental operation in many grid systems. Some serializers are message-based, while others are stream-based with no inherent message boundaries. Streaming serializers can be more scalable and flexible than message-oriented serializers, by promoting free form conversations not fixed to any particular static structure. This paper presents the design and implementation of the XBS binary serializer, focusing on three important features: efficient serialization of large and small arrays, efficient pass-through of opaque data by grid intermediaries, and efficient representation of numbers with a large dynamic range. The first feature is important because such arrays dominate scientific computing. The second feature is useful for grid intermediaries such as gateways and proxies, which are becoming ever more important as grid systems become more complex, and focus support on legacy applications. The third feature is important for efficiently supporting systems without arbitrary size limits. XBS is a freely-available C++ library and provides an object-oriented API, based on generic programming techniques.

The emerging cyberinfrastructure holds the promise of providing on-demand access to high performance network, compute and data resources. Image guided neurosurgery is one of many applications that requires such on-demand access to resources. In this paper we have studied the feasibility of accessing such resources on-demand. An experiment was designed and carried out across five TeraGrid clusters for this study. This paper provides an analysis of the results and draws some conclusion regarding feasibility of on-demand access to high performance resources.

The Common Instrument Middleware Architecture (CIMA) aims at Grid-enabling a wide range of scientific instruments and sensors to enable easy access to and sharing and storage of data produced by these instruments and sensors. This paper describes the implementation of CIMA applied to the field of single-crystal X-ray crystallography. To allow the researchers to easily view the current and past data streams from the instruments or sensors in a laboratory, a crystallography portal and associated portlets were developed for this application. The CIMA-based crystallography system provides an opportunity for anyone with Web access to observe and use crystallographic and other data from laboratories that previously had only limited access.

The purpose of the course was to develop a middleware for sensor networks that can be used in homes. The ultimate goal is to support a care network (family and friends) in trying to keep a loved one out of an assisted living facility. The middleware will allow the sensors in a home to be viewed and managed as a collection. This middleware will be used to specify what data should be gathered and stored, so that “privacy” can be tuned.

— As network attached instruments and sensors become available new opportunities for expanding their accessibility, usefulness and throughput present themselves. One of the keys to integrating instruments into computing and storage grids is to provide a standard methodology for developing interfaces that existing and future instruments can provide and data acquisition and reduction applications can rely on. In this paper we present an overview of the Common Instrument Middleware Architecture (CIMA), a Web Services based approach to making instruments and sensors network accessible in a standards-based, uniform way, and for interacting remotely with instruments and the data they produce. Some of the issues CIMA addresses include: standardization of the network protocol for interacting with instruments and sensors, flexibility in the underlying network transport, efficient and high throughput data transport, the availability (or lack of) computational, storage and networking resources at the instrument or sensor platform, evolution of instrument design, and reuse of data acquisition and processing codes.

Instruments and sensors and their accompanying actuators are essential to the conduct of scientific research. In many cases they provide observations in electronic format and can be connected to computer networks with varying degrees of remote interactivity. These devices vary in their architectures and type of data they capture and may generate data at various rates. In this paper we present an overview of the design goals and initial implementation of the common instrument middleware architecture (CIMA), a framework for making instruments and sensors network accessible in a standards-based, uniform way, and for interacting remotely with instruments and the data they produce. Some of the issues CIMA addresses include: flexibility in network transport, efficient and high throughput data transport, the availability (or lack of) computational, storage and networking resources at the instrument or sensor platform, evolution of instrument design, and reuse of data acquisition and processing codes

XML provides flexible, extensible data models and type systems for structured data, and has found wide-acceptance in many domains. XML processing can be slow, however, especially for scientific data, thus leading to the conventional wisdom that XML is not appropriate for such data. Instead, data is stored in specialized binary formats, and is transmitted via work-arounds such as attachments and base64 encoding. Though these work-arounds can be useful, they nonetheless relegate scientific data to second-class status within the Web services framework; and they generally require yet another API, data model, and type system. An alternative solution is to use more efficient encodings of XML, often known as "binary XML". Using XML uniformly throughout an application simplifies and unifies design and development. In this paper we present a binary XML format and implementation for scientific data called Binary XML for Scientific Applications (BXSA). We show that performance is comparable to that of commonly used scientific data formats such as netCDF. These results challenge the prevailing practice of handling control and data separately in scientific applications, with Web services for control and specialized binary formats for data