Data provenance is a valuable tool for protecting and troubleshooting distributed systems. Careful design of the provenance components reduces the impact on the design, implementation, and operation of the distributed system. In this paper, we present Curator, a provenance management toolkit that can be easily integrated with microservice-based systems and other modern distributed systems. This paper describes the design of Curator and discusses how we have used Curator to add provenance to distributed systems. We find that our approach results in no changes to the design of these distributed systems and minimal additional code and dependencies to manage. In addition, Curator uses the same scalable infrastructure as the distributed system and can therefore scale with the distributed system.
Ground personnel at the tactical edge often lack data and analytics that would increase their effectiveness. To address this problem, this work investigates methods to deploy cloud computing capabilities in tactical environments. Our approach is to identify representative applications and to design a system that spans the software/hardware stack to support such applications while optimizing the use of scarce resources. This paper presents our high-level design and the results of initial experiments that indicate the validity of our approach.
Provenance collection techniques have been carefully studied in the literature, and there are now several systems to automatically capture provenance data. However, the analysis of provenance data is often left "as an exercise for the reader". The provenance community needs tools that allow users to quickly sort through large volumes of provenance data and identify records that require further investigation. By detecting anomalies in provenance data that deviate from established patterns, we hope to actively thwart security threats. In this paper, we discuss issues with current graph analysis techniques as applied to data provenance, particularly Frequent Subgraph Mining (FSM). Then we introduce Directed Acyclic Graph regular grammars (DAGr) as a model for provenance data and show how they can detect anomalies. These DAGr provide an expressive characterization of DAGs, and by using regular grammars as a formalism, we can apply results from formal language theory to learn the difference between "good" and "bad" provenance. We propose a restricted subclass of DAGr called deterministic Directed Acyclic Graph automata (dDAGa) that guarantees parsing in linear time. Finally, we propose a learning algorithm for dDAGa, inspired by Minimum Description Length for Grammar Induction [1].
Building secure systems used to mean ensuring a secure perimeter, but that is no longer the case. Today's systems are ill-equipped to deal with attackers that are able to pierce perimeter defenses. Data provenance is a critical technology in building resilient systems that will allow systems to recover from attackers that manage to overcome the "hard-shell" defenses. In this paper, we provide background information on data provenance, details on provenance collection, analysis, and storage techniques and challenges. Data provenance is situated to address the challenging problem of allowing a system to "fightthrough" an attack, and we help to identify necessary work to ensure that future systems are resilient.
The following is a description of an inexpensive and simple student experiment for measuring the differences between the three types of spacetime topology—Euclidean (flat), Riemann (spherical), and Lobachevskian (saddle) curvatures. It makes use of commonly available tools and materials, and requires only a small amount of construction. The experiment applies to astronomical topics such as gravity, spacetime, general relativity, as well as geometry and mathematics.
XSEDE users, science gateways, and services need a variety of accurate information about XSEDE resources so that they can use those resources effectively. They need information to decide which resources to use, to track their usage of resources, and to provide services to their users. To support this, XSEDE is deploying a new system to gather and publish static and dynamic resource information. This paper gives an overview of the resource information available with this new system, describes the design and performance of the software and services that make up this system, and finally provides examples of how to use this new resource information.
The Texas Advanced Computing Center provides a variety of high-end resources to local, state, national, and international computational scientists and engineers. Many of these users obtain information about these resources via the TACC User Portal, which is supported by an information gathering and distribution infrastructure. After a number of years of use, we had a need to improve our presentation mechanisms by both redesigning the user portal and adding mobile interfaces. To support this, we also needed to update our information infrastructure. This paper describes our new design, including the new mobile interfaces, and details the improvements it makes over the previous design.
This paper describes the process of incorporating predictions of job queue wait times and run times into a Science Gateway. Science Gateways that integrate multiple resources can use predictions of queue wait times and run times to advice users when they choose where a job is executed or in an automated resource selection process. These predictions are also critical in executing workflows were it isn't feasible to have users specify where each task executes and the workflow management system therefore has to perform resource selection programmatically. SEAGrid science gateway has partly integrated the estimation of wait time prediction based on Karnak prediction service and is in the process of extending this to run time prediction.
This paper describes an information system designed to support the large volume of monitoring information generated by a distributed testbed. This monitoring information is produced by several subsystems and consists of status and performance data that needs to be federated, distributed, and stored in a timely and easy to use manner. Our approach differs from existing approaches because it federates and distributes information at a low architectural level via messaging; a natural match to many of the producers and consumers of information. In addition, a database is easily layered atop the messaging layer for consumers that want to query and search the information. Finally, a common language to represent information in all layers of the information system makes it significantly easier for users to consume information. Performance data shows that this approach meets the significant needs of FutureGrid and would meet the needs of an experimental infrastructure twice the size of FutureGrid. In addition, this design also meets the needs of existing distributed scientific infrastructures.
The ability to conduct consistent, controlled, and repeatable large-scale experiments in all areas of computer science related to parallel, large-scale, or distributed computing and networking is critical to the future and development of computer science. Yet conducting such experiments is still too often a challenge for researchers, students, and practitioners because of the unavailability of dedicated resources, inability to create controlled experimental conditions, and variability in software. Availability, repeatability, and open sharing of electronic products are all still difficult to achieve. To discuss those challenges and share experiences in their solution, the Workshop on Experimental Support for Computer Science brought together scientists involved in building and operating infrastructures dedicated to sup- porting computer science experiments to discuss challenges and solutions in this space. The workshop was held in November 2011 and was collocated with the SC11 conference in Seattle, Washington. Our objec- tives were to share experiences and knowledge related to supporting large-scale experiments conducted on experimental infrastructures, understand user requirements, and discuss methodologies and opportunities created by emerging technologies. This report ties together the workshop presentations and discussion and the consensus that emerged on the state of the field and directions for moving forward.
Cyberinfrastructures such as the TeraGrid often have information systems based on querying. While this pull-style information system is appropriate in some circumstances, there are many other circumstances where a push-style is more appropriate. This paper describes an information system based on push-style publish/subscribe messaging and evaluates the suitability of this approach.
This paper is intended to explain how the Tera Grid would like to be able to measure "usage modalities." We would like to (and are beginning to) measure these modalities to understand what objectives our users are pursuing, how they go about achieving them, and why, so that we can make changes in the Tera Grid to better support them.
Science Gateways often execute computations on clusters managed by batch schedulers. These schedulers queue computations until resources are available and then execute them. Clusters are typically scarce resources so the amount of time a computation waits before it begins to execute can be significant. Gateways can potentially reduce the turn around time of their computations and improve their user experience if they use estimates of how long computations will wait in a queue before beginning to execute. This paper describes a service that provides such estimates as well as current and historical job information. An instance of this service has been deployed on TeraGrid and is available to TeraGrid science gateways.
The NSF TeraGrid project has designed and constructed a federated integrated information service (IIS) to serve its capability publishing and discovery needs. This service has also proven helpful in automating TeraGrid's operational activities. We describe the requirements that motivated this work; IIS's system architecture, information architecture, and information content; processes that IIS currently supports; and how various layers of the system architecture are being used. We also review motivating use cases that have not yet been satisfied by IIS and outline approaches for future work.
Users of distributed systems such as the TeraGrid and Open Science Grid can execute their applications on many different systems. We wish to help such users, or the grid schedulers they use, select where to run applications by providing predictions of when tasks will complete if sent to different systems. We make predictions of file transfer times, batch scheduler queue wait times, and application execution times using historical information and instance-based learning techniques. Our prediction errors for data from the TACC lonestar system are 37 percent of mean file transfer time, 115 percent for mean queue wait time, and 72 percent of mean execution time. Our approach achieves significantly lower prediction error on other workloads. We have wrapped these prediction techniques with Web services, making predictions available to users of distributed systems as well as tools such as resource brokers and metaschedulers.
In the last several years, the electronics industry has released hand generator-powered flashlights, which are advertised as the end of battery-powered flashlights. This has become possible because of recent advances in capacitor, magnet, and LED technology. Nevertheless, the physics behind these flashlights is fairly simple.
Darcy Quesnel合作论文数Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, IL5
Shava Smallen合作论文数San Diego Supercomputer Center4
Geoffrey Fox合作论文数Department of Physics, College of Arts and Sciences, Indiana University;Department of Intelligent Systems Engineering, Indiana University;Community Grid Laboratory, Indiana University;Digital Science Center of Pervasive Technology Institute;School of Engineering and Applied Science, University of Virginia2
V. Welch合作论文数Globus Alliance board member;Illinois Trust Institute;NCSA: Security R&D group manager2
Nicholas T. Karonis合作论文数Department of Computer Science;Northern Illinois University2