Throughput architectures such as GPUs require substantial hardware resources to hold the state of a massive number of simultaneously executing threads. While GPU register files are already enormous, reaching capacities of 256KB per streaming multiprocessor (SM), we find that nearly half of real-world applications we examined are register-bound and would benefit from a larger register file to enable more concurrent threads. This article seeks to increase the thread occupancy and improve performance of these register-bound applications by making more efficient use of the existing register file capacity. Our first technique eagerly deallocates register resources during execution. We show that releasing register resources based on value liveness as proposed in prior states of the art leads to unreliable performance and undue design complexity. To address these deficiencies, our article presents a novel compiler-driven approach that identifies and exploits last use of a register name (instead of the value contained within) to eagerly release register resources. Furthermore, while previous works have leveraged "scalar" and "narrow" operand properties of a program for various optimizations, their impact on thread occupancy has been relatively unexplored. Our article evaluates the effectiveness of these techniques in improving thread occupancy and demonstrates that while any one approach may fail to free very many registers, together they synergistically free enough registers to launch additional parallel work. An in-depth evaluation on a large suite of applications shows that just our early register technique outperforms previous work on dynamic register allocation, and together these approaches, on average, provide 12% performance speedup (23% higher thread occupancy) on register bound applications not already saturating other GPU resources.

Modern throughput processors such as GPUs employ thousands of threads to drive high-bandwidth, long-latency memory systems. These threads require substantial on-chip storage for registers, cache, and scratchpad memory. Existing designs hard-partition this local storage, fixing the capacities of these structures at design time. We evaluate modern GPU workloads and find that they have widely varying capacity needs across these different functions. Therefore, we propose a unified local memory which can dynamically change the partitioning among registers, cache, and scratchpad on a perapplication basis. The tuning that this flexibility enables improves both performance and energy consumption, and broadens the scope of applications that can be efficiently executed on GPUs. Compared to a hard-partitioned design, we show that unified local memory provides a performance benefit as high as 71% along with an energy reduction up to 33%.

Modern graphics processing units (GPUs) employ a large number of hardware threads to hide both function unit and memory access latency. Extreme multithreading requires a complex thread scheduler as well as a large register file, which is expensive to access both in terms of energy and latency. We present two complementary techniques for reducing energy on massively-threaded processors such as GPUs. First, we investigate a two-level thread scheduler that maintains a small set of active threads to hide ALU and local memory access latency and a larger set of pending threads to hide main memory latency. Reducing the number of threads that the scheduler must consider each cycle improves the scheduler’s energy efficiency. Second, we propose replacing the monolithic register file found on modern designs with a hierarchical register file. We explore various trade-offs for the hierarchy including the number of levels in the hierarchy and the number of entries at each level. We consider both a hardware-managed caching scheme and a software-managed scheme, where the compiler is responsible for orchestrating all data movement within the register file hierarchy. Combined with a hierarchical register file, our two-level thread scheduler provides a further reduction in energy by only allocating entries in the upper levels of the register file hierarchy for active threads. Averaging across a variety of real world graphics and compute workloads, the active thread count can be reduced by a factor of 4 with minimal impact on performance and our most efficient three-level software-managed register file hierarchy reduces register file energy by 54%.

Modern graphics processing units (GPUs) use a large number of hardware threads to hide both function unit and memory access latency. Extreme multithreading requires a complicated thread scheduler as well as a large register file, which is expensive to access both in terms of energy and latency. We present two complementary techniques for reducing energy on massively-threaded processors such as GPUs. First, we examine register file caching to replace accesses to the large main register file with accesses to a smaller structure containing the immediate register working set of active threads. Second, we investigate a two-level thread scheduler that maintains a small set of active threads to hide ALU and local memory access latency and a larger set of pending threads to hide main memory latency. Combined with register file caching, a two-level thread scheduler provides a further reduction in energy by limiting the allocation of temporary register cache resources to only the currently active subset of threads. We show that on average, across a variety of real world graphics and compute workloads, a 6-entry per-thread register file cache reduces the number of reads and writes to the main register file by 50% and 59% respectively. We further show that the active thread count can be reduced by a factor of 4 with minimal impact on performance, resulting in a 36% reduction of register file energy.

As processors increasingly become power limited, performance improvements will be achieved by rearchitecting systems with energy efficiency as the primary design constraint. While some of these optimizations will be hardware based, combined hardware and software techniques likely will be the most productive. This work redesigns the register file system of a modern throughput processor with a combined hardware and software solution that reduces register file energy without harming system performance. Throughput processors utilize a large number of threads to tolerate latency, requiring a large, energy-intensive register file to store thread context. Our results show that a compiler controlled register file hierarchy can reduce register file energy by up to 54%, compared to a hardware only caching approach that reduces register file energy by 34%. We explore register allocation algorithms that are specifically targeted to improve energy efficiency by sharing temporary register file resources across concurrently running threads and conduct a detailed limit study on the further potential to optimize operand delivery for throughput processors. Our efficiency gains represent a direct performance gain for power limited systems, such as GPUs.

Modern multicore chips target thread-level parallelism at the expense of increasing instruction-level parallelism from single threaded programs. While recent work has attempted to construct a wide-ILP machine from multiple simple cores, these approaches suffer from ISA overheads or scalability challenges. In this paper, we describe an architecture that is inspired by the scalability and flexibility of the TFLEX architecture, yet elides the unorthodox ISA and the overheads that stem from its dataflow execution model. Our results focus on the tradeoff between near out-of-order execution (small out-of-order window within a block of instructions) and far out-of-order execution across blocks. Experiments indicate that a small out-of-order window combined with block-level speculation enables our proposed ReFLEX architecture to achieve comparable performance and flexibility as TFLEX yet with simpler cores and a more conventional ISA.

Modern multicore chips target thread-level parallelism at the expense of increasing instruction-level parallelism from single threaded programs. While recent work has attempted to construct a wide-ILP machine from multiple simple cores, these approaches suffer from ISA overheads or scalability challenges. In this paper, we describe an architecture that is inspired by the scalability and flexibility of the TFLEX architecture, yet elides the unorthodox ISA and the overheads that stem from its dataflow execution model. Our results focus on the tradeoff between near out-of-order execution (small out-of-order window within a block of instructions) and far out-of-order execution across blocks. Experiments indicate that a small out-of-order window combined with block-level speculation enables our proposed ReFLEX architecture to achieve comparable performance and flexibility as TFLEX yet with simpler cores and a more conventional ISA.

The TRIPS system employs a new instruction set architecture (ISA) called Explicit Data Graph Execution (EDGE) that renegotiates the boundary between hardware and software to expose and exploit concurrency. EDGE ISAs use a block-atomic execution model in which blocks are composed of dataflow instructions. The goal of the TRIPS design is to mine concurrency for high performance while tolerating emerging technology scaling challenges, such as increasing wire delays and power consumption. This paper evaluates how well TRIPS meets this goal through a detailed ISA and performance analysis. We compare performance, using cycles counts, to commercial processors. On SPEC CPU2000, the Intel Core 2 outperforms compiled TRIPS code in most cases, although TRIPS matches a Pentium 4. On simple benchmarks, compiled TRIPS code outperforms the Core 2 by 10% and hand-optimized TRIPS code outperforms it by factor of 3. Compared to conventional ISAs, the block-atomic model provides a larger instruction window, increases concurrency at a cost of more instructions executed, and replaces register and memory accesses with more efficient direct instruction-to-instruction communication. Our analysis suggests ISA, microarchitecture, and compiler enhancements for addressing weaknesses in TRIPS and indicates that EDGE architectures have the potential to exploit greater concurrency in future technologies.

The TRIPS hardware prototype is the first instantiation of anExplicit Data Graph Execution (EDGE) architecture. Building the com piler, toolset, and system software for the prototype required sup porting the system's unique dataflow construction, its banked registerand memory configurations, and its novel Instruction Set Architecture . In particu- lar, the TRIPS ISA includes (i) a block atomic execution mode l, (ii) ex- plicit mappings of instructions to execution units, and (ii i) predicated instructions which may or may not fire, depending on the outco me of preceding instructions. Our primary goal has been to cons truct tools to consume standard C and Fortran source code and gener ate binaries both for the TRIPS software simulators and hardwar e proto- type. A secondary goal has been to build the software infrast ructure on standard platforms using standard tools. These goals have b een met through a combination of off-the-shelf and custom tools. Wepresent a number of design issues and their resolution in enabling en d users to exercise the prototype ISA using familiar tools and progr amming interfaces. Finally, we offer download instructions for th ose who wish to test-drive the TRIPS tools.

Diminishing performance gains in conventional architectu res are driving modern architectures to exploit parallelism mo re effectively. Next-generation architectures hold promise in the Digital Signal Processing (DSP) arena where high performance and power efficiency are equally important. To better identify optimization techniques on these emerging new arc hitectures, we optimized and evaluated a suite of benchmarks representative of high performance DSP applications. Usin g these benchmarks, this paper analyzes the performance effects of several code optimizations on a next-generation ge neral purpose processor.

Acknowledgments I would like to thank the following people: • My supervisor Dr. Stephen Keckler for his guidance over the last two years • Saurabh Drolia, Sadia Sharif, Paul Gratz, and the TRIPS team members for their help throughout this project. Mark,Gebhart The University of Texas at Austin May 2006 iv Implementation of a Streaming Parallel Model for the TRIPS Architecture Mark Gebhart, B.S. The University of Texas at Austin, 2006 Supervisor: Stephen W. Keckler One common,method,of improving performance is to use a collection of

Diminishing performance gains in conventional architectures are fueling novel designs which more effectively extract parallelism and have the potential to change the nature of architectural bottlenecks. Consequently, workload characterization is of a growing importance in the design of modern high performance computing architectures. However, the accurate performance evaluation necessary for workload characterization can be prohibitively constrained by immature compilers. In this paper, we present a workload characterization for High Performance Digital Signal Processing (HP-DSP) applications on the TRIPS architecture. Included is a bottleneck analysis of this novel next-generation architecture and a discussion of our evaluation methodology. Using a combination of hand and machine optimization techniques we succesfuly characterize the workload of the TRIPS architecture on HP-DSP applications under the constraint of a developing compiler. This detailed performance characterization illustrates the potential of HP-DSP applications to successfully map to highly concurrent hardware and discusses bottlenecks unique to the TRIPS architecture.

The TRIPS hardware prototype is the first instantiation of an Explicit Data Graph Execution (EDGE) architecture. Building the compiler, toolset, and system software for the prototype required supporting the system’s unique dataflow construction, its banked register and memory configurations, and its novel Instruction Set Architecture. In particular, the TRIPS ISA includes (i) a block atomic execution model, (ii) explicit mappings of instructions to execution units, and (iii) predicated instructions which may or may not fire, depending on the outcome of preceding instructions. Our primary goal has been to construct tools to consume standard C and Fortran source code and generate binaries both for the TRIPS software simulators and hardware prototype. A secondary goal has been to build the software infrastructure on standard platforms using standard tools. These goals have been met through a combination of off-the-shelf and custom tools. We present a number of design issues and their resolution in enabling end users to exercise the prototype ISA using familiar tools and programming interfaces. Finally, we offer download instructions for those who wish to test-drive the TRIPS tools.

: The TRIPS system employs a new instruction set architecture (ISA) called Explicit Data Graph Execution(EDGE) that renegotiates the boundary between hardware and software to expose and exploitconcurrency. EDGE ISAs use a block-atomic execution model in which blocks are composed of dataflowinstructions. The goal of the TRIPS design is to mine concurrency for high performance while toleratingemerging technology scaling challenges, such as increasing wire delays and power consumption. Thispaper evaluates how well TRIPS meets this goal through a detailed ISA and performance analysis. Wecompare performance, using cycles counts, to commercial processors. On SPEC CPU2000, the Intel Core2 outperforms compiled TRIPS code in most cases, although TRIPS matches a Pentium 4. On simplebenchmarks, compiled TRIPS code outperforms the Core 2 by 10 and hand-optimized TRIPS code outperformsit by factor of 3. Compared to conventional ISAs, the block-atomic model provides a largerinstruction window, increases concurrency at a cost of more instructions executed, and replaces registerand memory accesses with more efficient direct instruction-to-instruction communication. Our analysissuggests ISA, microarchitecture, and compiler enhancements for addressing weaknesses in TRIPS and indicatesthat EDGE architectures have the potential to exploit greater concurrency in future technologies.