Resource sharing is a common technique to improve the resource utilization of a computer system. If poorly managed, however, resource sharing can lead to unpredictable performance, which can violate applications' Quality of Service requirements. Traditionally, computer applications share resources at coarse granularities (e.g., complete processor(s) and physical memory pages), and conventional instruction set architecture features allow OS policies to manage sharing in order to ensure applications' performance requirements are satisfied. However, with the evolution toward single chip multicore hardware, threads can share fine-grained, microarchitecture hardware resources. To efficiently satisfy the demands of future platforms, multicore systems should employ coordinated hardware/software microarchitecture resource management. Coordinated microarchitecture resource management requires a well-structured framework for building solutions that can be tailored to applications' specific requirements. Such a framework should be consistent with established system abstractions and provide a distinct separation between mechanisms and policies. A system's mechanisms should provide a universal set of workload independent resource management primitives, and a system's policies should provide workload dependent resource management solutions. To this end, I introduce Virtual Private Machines (VPMs). The VPM abstraction acts as the interface between policies and mechanisms. A VPM consists of a complete set of virtual microarchitecture resources, where a virtual resource has both a microarchitectural component and a temporal component (time slices). By design, a VPM has the same performance characteristics as a real machine with an equivalent set of resources. VPM policies, implemented primarily in software, translate application performance objectives into VPM resource assignments, thereby managing system resources. Then, VPM mechanisms securely multiplex and arbitrate hardware resources to satisfy the VPM assignments.

Current resource management mechanisms and policies are inadequate for future multicore systems. Instead, a hardware/software interface based on the virtual private machine abstraction would allow software policies to explicitly manage microarchitecture resources. VPM policies, implemented primarily in software, translate application and system objectives into VPM resource assignments. Then, VPM mechanisms securely multiplex, arbitrate, or distribute hardware resources to satisfy the VPM assignments.

ions can be used to abstract away irrelevant implementationspecific VPM details. ......................................................................................................................................................................................................................... ARCHITECTURE-OS INTERACTION .......................................................................

Virtual Private Machines (VPM) provide a framework for Quality of Service (QoS) in CMP-based computer systems. VPMs incorporate microarchitecture mechanisms that allow shares of hardware resources to be allocated to executing threads, thus providing applications with an upper bound on execution time regardless of other thread activity. Virtual Private Caches (VPCs) are an important element of VPMs. VPC hardware consists of two major components: the VPC Arbiter, which manages shared cache bandwidth, and the VPC Capacity Manager, which manages the cache storage. Both the VPC Arbiter and VPC Capacity Manager provide minimum service guarantees that, when combined, achieve QoS for the cache subsystem. Simulation-based evaluation shows that conventional cache bandwidth management policies allow concurrently executing threads to affect each other significantly in an uncontrollable manner. The evaluation targets cache bandwidth because the effects of cache capacity sharing have been studied elsewhere. In contrast with the conventional policies, the VPC Arbiter meets its QoS performance objectives on all workloads studied and over a range of allocated bandwidth levels. The VPC Arbiter’s fairness policy, which distributes leftover bandwidth, mitigates the effects of cache preemption latencies, thus ensuring threads a high-degree of performance isolation. Furthermore, the VPC Arbiter eliminates negative bandwidth interference which can improve aggregate throughput and resource utilization.

We propose and evaluate a multi-thread memory scheduler that targets high performance CMPs. The proposed memory scheduler is based on concepts originally developed for network fair queuing scheduling algorithms. The memory scheduler is fair and provides quality of service (QoS) while improving system performance. On a four processor CMP running workloads containing a mix of applications with a range of memory bandwidth demands, the proposed memory scheduler provides QoS to all of the threads in all of the workloads, improves system performance by an average of 14% (41% in the best case), and reduces the variance in the threads' target memory bandwidth utilization from .2 to .0058

1. Introduction Over the past two decades, advances in semiconductor process technology and microarchitecture have led to significant reduction in processor clock periods. Meanwhile, advances in memory technology have led to ever increasing memory densities, but relatively minor reductions in memory access time. Consequently, memory latencies measured in processor clock cycles are continually increasing and are now on the order of hundreds of clock cycles in duration. Cache memories help bridge the processor-memory latency gap, but, of course, caches are not always effective. Cache misses to main memory still occur, and when they do, the penalty is very high. Probably the most basic technique for enhancing cache performance is to incorporate prefetching. As the processor-memory latency gap continues to increase, there is a need for continued development and refinement of prefetch methods. Most existing prefetching research has focuses on uniprocessor prefetching. In this paper, we investigate cache prefetching, aimed specifically at prefetching in a Chip Multiprocessor (CMP). Prefetching in a CMP system has very different constraints than uniprocessor prefetching. In a CMP, pin bandwidth and the number of transaction buffer entries (TBEs, the maximum number of outstanding memory requests) are much more important. Multiple processors are competing for off-chip bandwidth and TBEs, reducing the systems tolerance to inaccurate prefetches, where prefetch accuracy is the percent of prefetches that are accessed by demand fetches before they are evicted from the cache. Inaccurate prefetches waste system resources, increase bus contention, and can degrade overall system performance. Furthermore, in a directory-based system with multiple CMPs, memory latency is extremely important. Often these systems store the directory in memory, which may require a memory access to retrieve, which effectively doubles the latency of the request (assuming memory access times is much larger than the bus transaction time [5]). The CMP prefetching method we study is based on " stride stream buffer prefetching concentration zones " (CZones) [14]. This method, as originally proposed, divides memory into fixed size zones and looks for stride patterns in sequences of cache misses directed toward the individual zones. When it finds a stride pattern, it launches prefetch requests. This method has the desirable property of not needing the program counter values of the load instructions that cause misses, which may not be readily available at lower levels of the memory hierarchy. Throughout the rest of this paper we support using CZone prefetching in a CMP system. In section 2, …