This article describes a comprehensive approach for silicon debug of a server chipset that includes a high-performance, third-generation chip-multithreaded (CMT) Sparc microprocessor. Efficiently debugging the chipset required a combination of debug features in silicon and system platforms, firmware support for debug, test generation tools, and debug data interpretation tools. Several useful lessons were learned in the process.

Major microprocessor vendors have integrated functional software-based self-testing in their manufacturing test flows during the last decade. Functional self-testing is performed by test programs that the processor executes at-speed from on-chip memory. Multiprocessors and multithreaded architectures are constantly becoming the typical general-purpose computing paradigm, and thus the various existing uniprocessor functional self-testing schemes must be adopted and adjusted to meet the testing requirements of complex multiprocessors. A major challenge in porting a functional self-testing approach from the uniprocessor to the multiprocessor case is to take advantage of the inherent execution parallelism offered by the multiple cores and the multiple threads in order to reduce test execution time. In this paper, we study the application of functional self-testing to chip multithreaded (CMT) processors. We propose a method that exploits thread-level parallelism (TLP) to speed up the execution of self-test routines in every physical core of a multiprocessor chip. The proposed method effectively splits the self-test routines into shorter ones, assigns the new routines to the hardware threads of the core and schedules their execution in order to minimize the core idle intervals due to cache misses or long latency operations and maximize the utilization of core computing resources. We demonstrate our method in the open-source CMT multiprocessor model, Sun's OpenSPARC T1, which contains eight CPU cores, each one supporting four hardware threads. Our experimental results show a self-test execution speedup of more than three times compared to the single thread execution.

This third-generation Chip-Multithreading (CMT) SPARC processor consists of 16 cores with shared memory architecture and supports a total of 32 main threads plus 32 scout threads. It is targeted for high-performance servers, and is optimized for both single- and multi-threaded applications. The 396 mm2 chip is fabricated in an 11 metal layer 65-nm CMOS process and operates at a nominal frequency of 2.3 GHz, consuming a maximum power of 250 W at 1.2 V. This paper provides an overview of the architectural highlights and describes the physical implementation challenges and solutions including circuit innovations in memory arrays, register files, and floating-point hardware that boost the performance and circuit robustness with low area overhead.

The higher complexity of the hardware and software employed by modern computing systems, as well as semiconductor technology scaling, are increasing the likelihood of Silent Data Corruption (SDC). SDC occurs when incorrect data is provided to the user, e.g., written to the memory or I/O system, and no error is triggered. Such events may have catastrophic effects, in the case of life critical applications, and represent a significant cost penalty for businesses. The purpose of this panel is to provide real examples of silent corruption, and discuss solutions for avoiding it. The presentations address SDC generated at the semiconductor device level, as well as the virtualization software level. Techniques for reducing SDC, from the circuit to system level, will be covered. Results of an extensive SDC study, carried out at Los Alamos National Laboratory (LANL) on high-performance computing (HPC) platforms are also given.

OpenSPARC is an open source community based around hardware design and experimentation aids for UltraSPARC Tl and T2 chip multi-threaded (CMT) microprocessors. It is the genesis of a vision to create a larger community where open conversations and collaborative development projects spawn dramatic innovations around CMT processor architectures and chip design. Individual programmers as well as representatives from Universities, industry associations, supporting software companies, foundries, entrepreneurs, large corporations and visionaries have already begun to participate in this expanded community. The URL for OpenSPARC is http ://www. opensparc.net.

TM T1 and T2 chip multi-threaded (CMT) microprocessors[1]. The UltraSPARC TM T2 processor is the industry's first "server on a chip", with 8 cores, 64 threads and on-chip networking and security. The richness of the RTL source code, tools and information in OpenSPARC has made it a comprehensive, practical and relevant platform for research in several areas of computing. This paper highlights the potential of using OpenSPARC for research in hardware reliability. Examples of university research projects, results achieved, benefits gained and lessons learned using OpenSPARC are described. Future research directions in reliability based on OpenSPARC are proposed. I.INTRODUCTION istorically, microprocessors have been designed to improve the execution performance of single thread programs by exploiting instruction level parallelism (ILP). Common techniques used to improve single thread performance are deep pipelines, multiple instruction issue, speculation, and out-of-order instruction execution. Recently, these techniques have reached a point of diminishing returns because of inherently low or hard to exploit application ILP [2]. Techniques used to improve single thread performance often give rise to complex processor designs with poor pipeline efficiencies and high power consumption H OpenSPARC is based on Sun's UltraSPARC TM T1 [3],[4] and T2 [5],[6],[7] microprocessors, which are designed for commercial workloads that exhibit large amounts of thread level parallelism (TLP). UltraSPARC TM T1 and T2 employ chip multi-threading (CMT) technology to achieve high throughput on commercial workloads by taking advantage of the TLP inherent to such workloads. OpenSPARC is an open source community based around hardware design and experimentation aids for UltraSPARC TM T1 and T2. OpenSPARC provides open source availability of  The author is also affiliated with the School of Computer and Communication Sciences, École Polytechnique Fédérale de Lausanne. complete micro-architecture specifications, Verilog RTL code, a full suite of RTL and architectural simulations and infrastructure, FPGA implementations of the microprocessors, reference boards with microprocessors, hypervisor code and multiple operating system ports [1]. This paper briefly describes the UltraSPARC TM T1 and T2 architecture upon which OpenSPARC is based. The reliability and error management features of the architecture are discussed, especially the benefits of lower temperature operation achieved through reduced power consumption. Some ongoing research projects in hardware reliability at universities designated as OpenSPARC Centers of Excellence – are described along with their experiences in using OpenSPARC for the research. Future research directions in hardware reliability, error management, fault tolerance and on-line test based on OpenSPARC are proposed in …

The third generation CMT (chip multithreaded) microprocessor from Sun Microsystems has 16 cores and is optimized for high throughput without compromising high single thread performance. This paper describes the unique challenges faced in DFX of this complex CMT processor and the DFX solutions deployed. Some of the notable new DFX features include a highly configurable scan architecture, a memory test network that leverages functional access paths, BIST of special memories, and a test mode for running functional tests in the presence of non-deterministic serdes interfaces. Identification of chips with partially good cores and caches is supported in manufacturing for yield and in the field for availability.

Built-in self-test (BIST) techniques have evolved as cost-eeective techniques for testing digital circuits. These techniques add test circuitry to the chip such that the chip has the capability to test itself. A prime concern in using BIST is the area overhead due to the modiication of normal registers to BIST registers. This paper proposes a high-level synthesis methodology that addresses this concern at an early stage in the design cycle. Data path allocation algorithms are presented that 1) maximize the sharing of registers as BIST resources resulting in a small number of registers being modiied for BIST, and 2) minimize the number of CBILBO registers required in the BIST version. The designs synthesized by our algorithms have the same number of functional modules and registers as those synthesized using traditional approaches but with a much lower testability overhead.

Throughput computing requires chip I/O bandwidth of the order of Tbits/sec which can be met by high speed, large scale implementation of SerDes I/Os (serial/deserial differential I/Os with clock embedded in data stream). The traditional test philosophy and existing ATE do not meet the challenges of testing chip interfaces with few hundreds of I/Os operating at multi-Gbps. In this paper, we present the test challenges and describe on-chip DFT modes and new ATE directions for chip level characterization and test of such interfaces used in throughput computing chip sets

The next generation of throughput computing systems designed by Sun Microsystems require interconnect with bandwidth of the order of Tbits/sec. Interconnect topologies for such high bandwidth are based on a few hundred SerDes I/Os on chips operating at multi-Gbps. Testing of these I/Os at only the component level is inadequate. In this paper, we describe the design-for-testability features for system manufacturing and on-line test of such I/Os and interconnect

Silicon technology trends of 65nm technology and architectural trends of the next generation of processors, chip-sets and systems are driving new design paradigms and shifts in the approaches towards robust system design. This paper addresses the convergence of these trends in designing the next generation of highly reliable systems at Sun Microsystems.

This paper presents an approach for analysis of system state differences observable through the scan chain for the debug of functional failures. A novel methodology for Latch Divergence Analysis (LDA) is proposed for creating stable failure signatures and reducing system noise. The methodology and processing flow have been integrated into the normal debug flow for the UltraSPARCTM family processors and have been successfully applied in numerous debugs in the bring-up of new products.

The need for considering BIST requirements during the scheduling and assignment stages of behavioral synthesis has been demonstrated in previous research and techniques for reducing BIST resources of a data path during these stages of synthesis have been developed. However, the degree of freedom that can be exploited during scheduling and assignment to minimize these resources is often limited by the data and control dependencies of a behavior. In this paper, we propose transformation of a behavior before scheduling and assignment, namely introducing redundant computations such that the resulting data path is testable using few BIST resources. The transformation makes use of spare capacity of modules to add redundancy that enables test paths to be shared among the modules. A technique for identifying potential BIST resource sharing problems in a behavior and resolving them by redundant computations is presented. Introduiction of redundant computations is performed without compromising the latency and functional resource requirement of the behavior.

Lower bound estimations of functionalresources at various stages of high-level synthesis have been developed to guide synthesis algorithms toward optimal solutions. In this paper we present lower bounds on the number of test resources (i.e., registers that generate pseudo-random test patterns and/or compress test responses) required to test a synthesized data path using built-in self-test(BIST). The bounds on different types of test resources are proved tobe individually achievable and experiments show that in most cases thebounds can be achieved simultaneously and with minimum number offunctional registers. Efficient ways of computing the lower bounds aredeveloped. The estimations are performed on scheduled data flow graphswith a given module assignment and provide a practical way ofselecting or modifying module assignments and schedules such that theresulting synthesized data path requires a small number of BISTresources to test itself.

Built-in self-test (BIST) techniques modify functional hardware so that a chip has the capability to test itself. A prime concern in using BIST is the area overhead due to the modification of normal registers to BIST registers. This paper proposes register and interconnect assignment techniques that address the BIST area overhead issue during high-level synthesis. A minimal intrusion BIST methodology is employed where a subset of the functional registers are modified to be BIST registers. Depending on the BIST functions performed (test pattern generation and/or test response compression) and the concurrency of the functions, four types of BIST registers with varying costs are used. Data path allocation techniques are presented that (1) maximize the sharing of BIST registers between modules, and (2) minimize the number of expensive BIST registers that are essential for minimal intrusion BIST of a data path. The designs synthesized by our techniques have the same number of functional modules and registers as those synthesized using traditional approaches but require significantly lower BIST area overhead.

Built-in self-test (BIST) techniques modify functional hardware to give a data path the capability to test itself. The modification of data path registers into registers (BIST resources) that can generate pseudo-random test patterns and/or compress test responses, incurs an area overhead penalty. We show how scheduling and module assignment in high-level synthesis affect BIST resource requirements of a data path. A scheduling and module assignment procedure is presented that produces schedules which, when used to synthesize data paths, result in a significant reduction in BIST area overhead and hence total area.

The degree of freedom that can be exploited during scheduling and assignment to minimize BIST resources is often limited by the data dependencies of a behavior. W e propose transformation of a behavior by introducing redundant computations such that the resulting data path requires few BIST resources. The transformation makes use of spare capacity of modules to add redundancy that enables test paths to be shared among the modules. A technique is presented for introducing redundant computations that reduce the BIST resource requirements of a data path without compromising the latency and functional resource constraints.

Lower bound estimations of resources at various stages of high-level synthesis are essential to guide synthesis algorithms towards optimal solutions. In this paper we present lower bounds on the number of test resources (i.e. test pattern generators, signature analyzers and CBILBO registers) required to test a synthesized data path using built-in self-test (BIST). The estimations are performed on scheduled data flow graphs and provide a practical way of selecting or modifying module assignments and schedules such that the resulting synthesized data path requires a small number of test resources to test itself

Estimation of resources at various stages of the high-level synthesis process is essential to guide high-level synthesis algorithms towards optimal solutions. Lower bound es timation bounds the design space and gives an indication of the quality of the design synthesized. Previous work in high-level synthesis focused on bounds on functional resources. In this paper, we present lower bounds on the number of test resources (i.e. test pattern generators, signature analyzers and CBILBO registers) required to test the synthesized data path by the partial intrusion built-in self-test (BIST) method ology. The estimation is performed on scheduled data flow graphs and provides a practical way of selecting or modifying module assignments and schedules such that the synthesized data path requires a small number of test resources to test it. "This work was supported by the Advanced Research Projects Agency and monitored by the Department of the Army, Ft.Huachuca, under Contract No. DABT63-95-C-0042. The information reported here does not necessarily reflect the position or the policy of the Government and no official endorsement should be inferred.