AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
Qualitative classification of resource use, we have developed a suite of trained artificial neural networks and hidden Markov models

The Autopilot performance-directed adaptive control system

Future Generation Comp. Syst., no. 1 (2001): 175-187

Cited: 184|Views86
EI

Abstract

High-performance computing is rapidly expanding to include distributed collections of heterogeneous sequential and parallel systems and irregular applications with complex, data dependent execution behavior and time-varying resource demands. To provide adaptive resource management for dynamic applications, we are developing the Autopilot ...More

Code:

Data:

0
Introduction
  • The scope of high-performance computing is rapidly expanding from single parallel systems to distributed collections of heterogeneous sequential and parallel systems.
  • Autopilot provides a flexible set of performance sensors, decision procedures, and policy actuators to realize adaptive control of applications and resource management policies on both parallel and wide area distributed systems.
  • In Section 7, the authors describe flexible decision procedure mechanisms based on fuzzy logic, followed in Section 8 by a unified example of Autopilot’s application to adaptive input/output systems.
Highlights
  • The scope of high-performance computing is rapidly expanding from single parallel systems to distributed collections of heterogeneous sequential and parallel systems
  • The performance sensitivity of current parallel and distributed systems is a direct consequence of resource interaction complexity and the failure to recognize that resource allocation and management must evolve with applications, becoming more flexible and resilient to changing resource availability and resource demands
  • By integrating dynamic performance instrumentation and on-the-fly performance data reduction with configurable, malleable resource management algorithms and a real-time adaptive control mechanism, flexible runtime systems could automatically choose and configure resource management algorithms based on application request patterns and observed system performance
  • Qualitative classification of resource use, we have developed a suite of trained artificial neural networks (ANNs) and hidden Markov models (HMMs) [10,12]
  • Using the global access pattern classification, a file policy selection and configuration mechanism based on fuzzy logic might select a write back policy that merges file writes from each processor, forming larger, contiguous blocks for sequential write back to secondary storage devices
Results
  • As a complement to quantitative performance data, knowledge of qualitative application behavior, obtained from either user-written assertions or automatic classification techniques, is useful when dynamically choosing resource management policies.
  • A set of fuzzy logic decision mechanisms that exploit real-time sensor inputs and behavioral classifications to dynamically select resource management policies;
  • One can use the sensor infrastructure to capture real-time performance data streams and drive displays of system activity without use of either the decision procedure or actuator components.
  • As an example of the latter, an attached function could process a sequence of file read/write offsets captured by a file input/output sensor and produce a qualitative classification of the file access pattern.
  • Sensors provide the quantitative data needed to make resource policy management decisions, the experience with adaptive file system policy selection [9] has shown that qualitative data on current and future resource demands is an effective complement.
  • New application programming interfaces (APIs) for high-performance input/output include interfaces for user specification of future file access patterns.
  • Based on Nexus global pointers, sensors and actuators provide the mechanism needed to monitor and control both local and remote tasks.
  • Fig. 7 illustrates a simple set of fuzzy logic rules that might be used to control file prefetching in an adaptive input/output system.
Conclusion
  • As an example of the potential power of an adaptive parallel file system using the Autopilot infrastructure, consider a parallel application on P processors that first writes an N block data file using a strided access pattern.
  • Using the global access pattern classification, a file policy selection and configuration mechanism based on fuzzy logic might select a write back policy that merges file writes from each processor, forming larger, contiguous blocks for sequential write back to secondary storage devices.
  • Using performance data on disk queue lengths and response times, captured by real-time performance sensors, the fuzzy logic controls would use policy actuators to choose the write back policy and optimally choose the size of the write back units.
Funding
  • ଝ This work was supported in part by the Defense Advanced Research Projects Agency under DARPA contracts DABT6394-C0049 (SIO Initiative), DAVT63-91-C-0029 and DABT63-93C-0040, F30602-96-C-0161, and DABT63-96-C-0027 by the National Science Foundation under grants NSF ASC 92-12369, NSF CDA 94-01124, and ASC 97-20202, and by the Department of Energy under contracts DOE B-341494, W-7405ENG-48, and 1-B-333164
Reference
  • P.M. Chen, D.A. Patterson, Maximizing performance in a striped disk array, in: Proceedings of the 17th Annual International Symposium on Computer Architecture, 1990, pp. 322–331.
    Google ScholarLocate open access versionFindings
  • P.F. Corbett, J.-P. Prost, C. Demetriou, G. Gibson, E. Riedel, J. Zelenka, Y. Chen, E. Felten, K. Li, J. Hartman, L. Peterson, B. Bershad, A. Wolman, R. Aydt, Proposal for a common parallel file system programming interface, Version 1.0, November 1997. http://www.cs.arizona.edu/sio/.[3] P.E. Crandall, R.A. Aydt, A.A. Chien, D.A. Reed, Input/output characteristics of scalable parallel applications, in: Proceedings of Supercomputing’95, San Diego, CA, December 1995, IEEE Computer Society Press.
    Locate open access versionFindings
  • [4] US DOE, United States Department of Energy Accelerated Strategic Computing Initiative (ASCI), January 1997. http://www.llnl.gov/asci.
    Findings
  • [5] I. Foster, C. Kesselman, S. Tuecke, The Nexus approach to integrating multithreading and communication, J. Parall. Distrib. Comput. 37 (1996) 70–82.
    Google ScholarLocate open access versionFindings
  • [6] I. Foster, C. Kesselman, Globus: a metacomputing infrasturcture toolkit, Int. J. Supercomput. Appl. 11 (1997) 115–128.
    Google ScholarLocate open access versionFindings
  • [7] J.V. Huber, C.L. Elford, D.A. Reed, A.A. Chien, D.S. Blumenthal, PPFS: a high-performance portable parallel file system, in: Proceedings of the Ninth ACM International Conference on Supercomputing, July 1995, pp. 385–394.
    Google ScholarLocate open access versionFindings
  • [8] N.K. Kasabov, Foundations of Neural Networks, Fuzzy Systems, and Knowledge Engineering, The MIT Press, Cambridge, MA, 1996.
    Google ScholarFindings
  • [9] T.M. Madhyastha, C.L. Elford, D.A. Reed, Optimizing input/output using adaptive file system policies, in: Proceedings of the Fifth Goddard Conference on Mass Storage Systems and Technologies, Vol. II, September 1996, pp. 493–514.
    Google ScholarLocate open access versionFindings
  • [10] T.M. Madhyastha, D.A. Reed, Exploiting global input/output access pattern classification, in: Proceedings of Supercomputing’97, November 1997.
    Google ScholarLocate open access versionFindings
  • [11] T.M. Madhyastha, D.A. Reed, Intelligent, adaptive file system policy selection, in: Proceedings of Frontiers’96, October 1996, pp. 172–179.
    Google ScholarLocate open access versionFindings
  • [12] T.M. Madhyastha, D.A. Reed, Input/output access pattern classification using hidden Markov models, in: Proceedings of IOPADS’97, November 1997, pp. 57–67.
    Google ScholarLocate open access versionFindings
  • [13] B.P. Miller, M.D. Callaghan, J.M. Cargille, J.K. Hollingsworth, R. Bruce Irwin, K.L. Karavanic, K. Kunchitkapadam, T. Newhall, The Paradyn parallel performance measurement tools, IEEE Comput. 28 (11) (1995) 37–46.
    Google ScholarLocate open access versionFindings
  • [14] B.P. Miller, M. Clark, J. Hollingsworth, S. Kierstead, S.-S. Lim, T. Torzewski, IPS-2: the second generation of a parallel program measurement system, IEEE Trans. Comput. 1 (2) (1990) 206–217.
    Google ScholarLocate open access versionFindings
  • [15] D.A. Reed, Experimental performance analysis of parallel systems: techniques and open problems, in: Proceedings of the Seventh International Conference on Modelling Techniques and Tools for Computer Performance Evaluation, May 1994, pp. 25–51.
    Google ScholarLocate open access versionFindings
  • [16] D.A. Reed, R.A. Aydt, R.J. Noe, P.C. Roth, K.A. Shields, B.W. Schwartz, L.F. Tavera, Scalable performance analysis: the pablo performance analysis environment, in: A. Skjellum (Ed.), Proceedings of the Scalable Parallel Libraries Conference, IEEE Computer Society Press, 1993, pp. 104–113.
    Google ScholarLocate open access versionFindings
  • [17] D.A. Reed, C.L. Elford, T. Madhyastha, W.H. Scullin, R.A. Aydt, E. Smirni, I/O: performance analysis, and performance data immersion, in: Proceedings of MASCOTS’96, February 1996, pp. 1–12.
    Google ScholarLocate open access versionFindings
  • [18] D.A. Reed, C.L. Elford, T. Madhyastha, E. Smirni, S.L. Lamm, The next frontier: interactive and closed loop performance steering, in: Proceedings of the 1996 International Conference on Parallel Processing Workshop, August 1996, pp. 20–31.
    Google ScholarLocate open access versionFindings
  • [19] H. Simitci, D.A. Reed, Adaptive Disk striping for parallel input/output, Proc. 7th Goddard Conf. On Mass Storage Systems and Technologies, March 15–19, 1999, San Diego, CA, USA, pp. 88–102.
    Google ScholarFindings
  • [20] E. Smirni, C.L. Elford, D.A. Reed, Performance modeling of a parallel I/O system: an application driven approach, in: Proceedings of the Eighth SIAM Conference on Parallel Processing for Scientific Computing, March 1997.
    Google ScholarLocate open access versionFindings
  • [21] E. Smirni, D.A. Reed, I/O requirements of scientific applications: an evolutionary view, in: Proceedings of the Fifth IEEE International Symposium on High-performance Distributed Computing, August 1996, pp. 49–59.
    Google ScholarLocate open access versionFindings
  • [22] E. Smirni, D.A. Reed, Workload characterization of input/output intensive parallel applications, in: Proceedings of the Ninth International Conference on Modelling Techniques and Tools for Computer Performance Evaluation, June 1997.
    Google ScholarLocate open access versionFindings
  • [23] J.C. Yan, Performance tuning with AIMS: an automated instrumentation and monitoring system for multicomputers, in: Proceedings of the 27th Hawaii International Conference on System Sciences, January 1994, pp. 625–633.
    Google ScholarLocate open access versionFindings
  • [24] L. Zadeh, Common sense knowledge representation based on fuzzy logic, IEEE Comput. 16 (10) (1983) 61.
    Google ScholarLocate open access versionFindings
  • [25] L.A. Zadeh, Fuzzy sets, Inform. Contr. 8 (3) (1965) 338–353. Randy L. Ribler is an Assistant Professor and the Program Coordinator for the Computer Science program at Lynchburg College. He holds a PhD in Computer Science from Virginia Tech, an MS in Computer Science from George Mason University, and a BS in Computer Science from the University of Maryland at College Park. After graduation from Virginia Tech, Dr. Ribler joined the Pablo Group at the University of Illinois at Urbana-Champaign, where he worked as a postdoctoral research associate for two years. Prior to his return to academics in 1991, Dr. Ribler worked extensively in the high-performance computer industry. He was a principal software designer for the Star Technologies 910/VP, a SPARC-based mini-super computer. He also optimized software for high-speed medical imaging computers used in computer tomography.
    Google ScholarLocate open access versionFindings
  • Huseyin Simitci is an engineer at XIOtech Corporation, specializing in performance analysis of networked storage technologies. He received his PhD in Computer Science from the University of Illinois at Urbana-Champaign in 2000. He obtained his MS and BS from Bilkent University, Ankara, Turkey in 1994 and 1992, respectively. His research interests include intelligent storage systems, high performance computing, and system performance analysis.
    Google ScholarFindings
0
Your rating :

No Ratings

Tags
Comments
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn