The ACCESS Software Documentation Service (SDS) is designed to help ACCESS HPC users identify the software available on various ACCESS Resource Providers’ (RPs) resources, an important step toward selecting an RP. Not only does it provide a single webpage with information about all RP software, but it also provides extensive information about each piece of software, enabling researchers to discover software by research domain and type, learn its intended purpose and use, and read documentation and examples showing how to use the software. The SDS is designed to periodically obtain information about software at each RP to keep information current. For each software package, SDS shows the software name, RPs that have the software, available versions, RP specific documentation for the software, descriptions, core features, documentation links, and various other useful information. In the past eight months, the SDS website has received over thirteen thousand visits, demonstrating significant adoption among ACCESS users. A standalone version with support for containers, website customization, and other enhanced features is also available for any HPC center to use. This paper describes the SDS implementation, use within ACCESS, its benefits to both researchers and resource providers, and the stand-alone version of SDS available to HPC centers.
FABRIC is a national and international scale research infrastructure built to enable cutting-edge research in a wide range of networking and computing areas and domain sciences that depend on advanced networking and computing capabilities. Since its full operation in October 2023, FABRIC has seen rapidly emerging experiments in different research topics. This paper provides a description of FABRIC’s hardware resources, software services, and an analysis of the characteristics of the emerging experiments that are leveraging FABRIC’s unique capabilities. The analysis showed a wide spectrum of topic areas, frequent emphasis on at-scale and high performance experiments that interact with real facilities and real people, and growing complexity and rigor in tools, data and measurement. Researchers are seen sharing well-developed artifacts with others to enable larger experiments and reproducible research.
The NSF FABRIC network is an advanced user-programmable national network infrastructure that enables the deployment of leadingedge network applications and services not possible in today's Internet. A key feature of the FABRIC network is that it makes detailed infrastructure measurement and monitoring data available to users of the infrastructure and external researchers - not just the operators of the infrastructure as is typically done. This information can be invaluable to users as they try to understand, debug, and optimize their cutting-edge experiments, applications, and services - often significantly reducing the time-to-scientific-discovery. This paper presents the FABRIC Measurement Framework and its hierarchical data access architecture which makes large portions of FABRIC's measurement data available to not only operators but also the general public, infrastructure users, and external researchers. Key features of the FABRIC Measurement Framework include the ability to share measurement data with multiple user groups having differing levels of authorization, simple interfaces for viewing or downloading measurement data, selectable timeframes for comparative and longitudinal studies, and isolation features that make it easier to secure and protect the measurement data and the FABRIC infrastructure.
ACCESS provides many essential High Performance Computing (HPC) resources to researchers across diverse scientific disciplines. However, selecting the most appropriate resource from the numerous available options presents significant challenges, especially for researchers without extensive technical expertise. This paper introduces the ACCESS Resource Advisor (ARA), a web-based tool designed to guide researchers toward suitable HPC resources by using accessible language, only asking about basic HPC concepts that users would understand, identifying workflow patterns, and providing recommendations with minimal required input. We detail the design philosophy, implementation approach, and rules-based recommendation used by the ARA. Usage data collected over ten months demonstrates consistent adoption, averaging 99.2 recommendations per month, with recent increases coinciding with the tool’s integration into the ACCESS Resource page. By simplifying the resource selection process, ARA helps researchers from over a hundred scientific fields make more informed decisions about computational resources, ultimately improving research efficiency and resource utilization across the ACCESS ecosystem.
Distributed cyberinfrastructures (CI) pose opportunities and challenges for the execution of scientific workflows, especially in the context of Earth science applications. They provide heterogeneous resources that can meet the needs of the applications that are part of the scientific workflows and provide the necessary performance and scalability to achieve scientific goals. However, the challenge with distributed CI is that it is difficult to find the right resources for the applications and to orchestrate the workflow execution from resource provisioning to job execution to delivering the final results. In some cases, poor choice of resources may result in slow execution or outright failure. In this paper, we present Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS) Pegasus, a CI solution built as part of the U.S. National Science Foundation ACCESS program that provides automated execution of scientific applications. We demonstrate Pegasus's capabilities with SOil MOisture SPatial Inference Engine (SOMOSPIE), an earth science multi-component application for fine-grained soil moisture predictions. We identify a roadmap to migrate applications such as SOMOSPIE on ACCESS resources with the support of ACCESS Pegasus, outlining both strengths and weaknesses of this approach.
NSF-supported cyberinfrastructure (CI) has been highly successful in advancing science and engineering over the last few decades.During that time, there have been significant changes in the size and composition of the participating community, the architecture and capacity of compute, storage, and networking platforms, and the methods by which researchers and CI professionals communicate.These changes require rethinking the role of research support services and how they are delivered.To address these changes and support an expanding community, MATCH is implementing a model for research support services in ACCESS that comprises
Network security policies provide high-level directives regarding acceptable and unacceptable use of the network. Organizations specify these high-level directives in policy documents written using human-readable natural language. The challenge is to convert these natural language policies to the network configurations/specifications needed to enforce the policy. Network administrators, who are responsible for enforcing the policies, typically translate the policies manually, which is a challenging and error-prone process. As a result, network operators (as well as the policy authors) often want to verify that network policies are being correctly enforced. In this paper, we propose Network Policy Conversation Engine (NPCE), a system designed to help network operators (or policy writers) interact with the network using natural language (similar to the language used in the network policy statements themselves) to understand whether policies are being correctly enforced. The system leverages emerging big data collection and analysis techniques to record flow and packet level activity throughout the network that can be used to answer users policy questions. The system also takes advantage of recent advances in Natural Language Processing (NLP) to translate natural language policy questions into the corresponding network queries. To evaluate our system, we demonstrate a wide range of policy questions – inspired by actual networks policies posted on university websites – that can be asked of the system to determine if a policy violation has occurred.
Network policies govern the use of an institution's networks, and are usually written in a high-level human-readable natural language. Normally these policies are enforced by low-level, technically detailed network configurations. The translation from network policies into network configurations is a tedious, manual and error-prone process. To address this issue, we propose a new intermediate language called POlicy LANguage for Campus Operations (POLANCO), which is a human-readable network policy definition language intended to approximate natural language. Because POLANCO is a high-level language, the translation from natural language policies to POLANCO is straightforward. Despite being a high-level human readable language, POLANCO can be used to express network policies in a technically precise way so that policies written in POLANCO can be automatically translated into a set of software defined networking (SDN) rules and actions that enforce the policies. Moreover, POLANCO is capable of incorporating information about the current network state, reacting to changes in the network and adjusting SDN rules to ensure network policies continue to be enforced correctly. We present policy examples found on various public university websites and show how they can be written as simplified human-readable statements using POLANCO and how they can be automatically translated into SDN rules that correctly enforce these policies.
Network security devices intercept, analyze and act on the traffic moving through the network to enforce security policies. They can have adverse impact on the performance, functionality, and privacy provided by the network. To address this issue, we propose a new approach to network security based on the concept of short-term on-demand security exceptions. The basic idea is to bring network providers and (trusted) users together by (1) implementing coarse-grained security policies in the traditional way using conventional in-band security approaches, and (2) handling special cases policy exceptions in the control plane using user/application-supplied information. By divulging their intent to network providers, trusted users can receive better service. By allowing security exceptions, network providers can focus inspections on general (untrusted) traffic. We describe the design of an on-demand security exception mechanism and demonstrate its utility using a prototype implementation that enables high-speed big-data transfer across campus networks. Our experiments show that the security exception mechanism can improve the throughput of flows by trusted users significantly.
Traditional high performance computing (HPC) centers that operate a single large supercomputer cluster have not required sophisticated mechanisms to manage and enforce network policies. Recently, HPC centers have expanded to support a wide range of computational infrastructure, such as OpenStack-based private clouds and Ceph object stores, each with its own unique characteristics and network security requirements. Network security policies are becoming more complex and harder to manage. To address the challenge, this paper explores ways to define and manage the new network policies required by emerging HPC systems. As the first step, we identify the new types of policies that are required and the technical capabilities needed to support them. We present example policies and discuss ways to implement those policies using emerging programmable networks and intent-based networks. We describe our initial work toward automatically converting human readable network policies into network configurations and programmable network controllers that implement those policies using business rule management systems.
In this paper we consider an alternative possibility for routing money in the Internet ecosystem based on a spot market for interconnection service that operates alongside, or in addition to, the traditional contract service model. Recent work by others showed that under certain assumptions, enabling transit providers (i.e., networks that carry packets between networks) to sell their excess capacity on a best-effort basis improves both provider profit and consumer surplus. While prior work only focused on pricing strategies, we explore the technical feasibility of such a market in this paper. We consider what is needed to make such a spot market possible and focus on the interaction between technical and economic considerations. In particular, we describe two approaches that demonstrate how economic software defined exchanges (ESDXs) can be used as trusted intermediaries to tie the forwarding service to the flow of money.
A key concept of software-defined networking (SDN) is separation of the control and data plane. This idea provides several benefits, including fine-grained network control and monitoring, and the ability to deploy new services in a limited scope. Unfortunately, it is often cost-prohibitive for enterprises (and universities in particular) to upgrade their existing networks to wholly SDN-capable networks all at once. A compromise solution is to deploy SDN capabilities incrementally in the network. The challenge then is to take full advantage of SDN-based services throughout the network, in an integrated fashion rather than in a few "islands" of SDN support. At the University of Kentucky, SDN has been integrated into the campus network for several years. In this paper, we describe two aspects of this challenge, along with our solution approaches. One is the general reluctance of campus network administrations to allow novel or experimental (SDN-based) services in the production network. The other is how to extend such services throughout the legacy part of the network. For the former, we lay out a set of principles designed to ensure that the production service is not harmed. For the latter, we use policy based routing and a graph database to extend our previously-described VIP Lanes service. Our simulation results in a campus-like topology testbed show that we can provide a host with custom path service even if it is connected to a legacy router.
Current pricing mechanisms for end-to-end Internet transport involve heavyweight service-level agreements (SLAs) between providers, which are enforced through policy configurations in the Border Gateway Protocol. One drawback of this practice is that BGP gives preference to a single path, and thus inherently limits the richness of the network topology available to users at any time. Moreover, the coarse granularity and long time scales of interdomain SLAs limit the effect of competition and market forces. In this paper, we show how the use of software-defined infrastructure, in the form of an Economic Software-Defined Exchange Point (ESDX), can help enable a more dynamic interdomain ecosystem. In particular, providers can use the ESDX to establish short-term contracts and thus access a much richer set of path alternatives and contracts. We show that transparent pricing mechanisms are possible and that a pricing network equilibrium can be achieved.
Shared research infrastructure that is globally distributed and widely accessible has been a hallmark of the networking community. We present a vision for a future mid-scale distributed research infrastructure aimed at enabling new types of discoveries. The "lessons learned" from constructing and operating the Global Environment for Network Innovations (GENT) infrastructure are the basis for our attempt to project future concepts and solutions. Our aim is to engage the community to contribute new ideas and to inform funding agencies about future research directions.
Campus networks and enterprise networks increasingly depend on middleboxes (e.g., firewalls, NAT, load balancers, IDS/IDP) to provide essential services or enforce network policies. These middleboxes often limit the performance of network applications, especially those involved in big data transfer. To address this problem, we propose a Software Defined Networking (SDN) campus network architecture, called VIP Lanes, that provides the ability for pre-authorized, trusted users to create flows that bypass middleboxes, thereby enabling those users to achieve substantially better performance while maintaining security and policy compliance for other network traffic. In this paper, we present the VIP Lanes abstraction and describe an authorization and policy-enforcement service used to establish trusted VIP Lanes. We describe an initial prototype implementation that not only demonstrates the viability of the VIP Lanes approach, but also gives an indication of the types of performance improvements that are possible - in some cases approaching a two order of magnitude reduction in transmission times.
Deployment of innovative new networking services requires support by network providers. Since economic motivation plays an important role for network providers, it is critical that a network architecture intrinsically considers economic relationships. We present the design of a protocol that associates access to network services with economic contracts. We show how this protocol can be realized in fundamentally different ways, using out-of-band signaling and in-band signaling, based on two different prototype implementations. We present results that show the effectiveness of the proposed protocol and thus demonstrate a first step toward realizing an economy plane for the Internet.
Unlike the current Internet where applications have little control over the path that their data travels, emerging network architectures enable applications to control the path their packets/flows take. Although an application can select its paths, providing an application with the up-to-date information needed to make an informed decision is a difficult problem. In particular, monitoring (i.e., collecting performance information about) a network the size of the Internet presents serious scalability challenges. In this paper, we present a network monitoring service that can provide applications with up-to-date network information, yet scale to the size of the Internet. Our network monitoring service dynamically selects and enables/disables monitoring points to focus on paths currently needed by applications. We present experimental results based on real-world traffic traces that show our system can provide up-to-date information for 80% of the paths used by applications by monitoring as few as 100 strategically identified locations in the Internet.
Experimentation with new network architectures and protocols is one of the primary motivations for building future Internet testbeds such as the Global Environment for Network Innovations (GENI) testbed. A key part of experimentation is the ability to observe, measure, evaluate, and compare these new architectures and protocols. Observing an experiment's network performance requires setting up the measurement infrastructure needed to monitor and record the behavior of the network. It also requires a full set of tools and user interfaces that enable access to the measurement data both while the experiment is running and later during post-analysis.To simplify the task of measuring experiments in future Internet testbeds like GENI, we developed an instrumentation and measurement system called INSTOOLS. It automates the process of setting up the measurement infrastructure, tailoring the measurement infrastructure and the data capture to the experimental network's topology and configuration. In addition, INSTOOLS provides a suite of tools via its "portal" service that make it easy for users to observe, measure, format, and archive data from their experiments. This paper describes the INSTOOLS system and the set of interfaces/tools it offers to users. INSTOOLS has been in use for several years, and we provide performance results that illustrate its scalability. We also present our second-generation portal, the GENI One Stop Portal, that offers a comprehensive. interface to a wide range of tools. (c) 2013 Elsevier B.V. All rights reserved.
The Internet has been a key enabling technology for many new distributed applications and services. However, the deployment of new protocols and services in the Internet infrastructure itself has been sluggish, especially where economic incentives for network providers are unclear. In our work, we seek to develop an "economy plane" for the Internet that enables network providers to offer new network-based services (QoS, storage, etc.) for sale to customers. The explicit connection between economic relationships and network services across various time scales enables users to select among service alternatives. The resulting competition among network service providers will lead to overall better technological solutions and more competitive prices. In this paper, we present the architectural aspects of our ChoiceNet economy plane as well as some of the technological problems that need to be addressed in a practical deployment.
W. Brent Seales合作论文数Computer Science Department5