Compliant direct attach implemented with single phase coolant has been shown to be an effective means of achieving reliable cooling of high-power components. Primary targets of such cooling systems are high power CPUs which have highly nonuniform power densities. While single phase water cooling is effective thermally, it comes with potential issues associated with water leaks. A desirable solution to this problem would be an approach capable of at least similar performance at high local power (core) densities utilizing a non-conductive fluid. Two phase cooling is a candidate but can have instability and dry-out concerns with very high local power densities when implemented in conventional parallel channel form. Expanding channels in a radial configuration have been demonstrated to effectively mitigate these concerns but are difficult to implement in a compliant direct attach configuration due to the central inlet and are best implemented when the device to be cooled is close to square rather than rectangular. In this work we present a cold plate approach implementing multilayer counter-flow expanding channels with a load-supporting outlet plenum which allows compliant direct attach in a small footprint. The design process included multiple rounds of modeling and revision to bring the single-phase and two-phase pressure drops at the flow rate associated with a $50 \%$ full power vapor quality into the desired range. A thermal test vehicle comprised of an array of 4.2 mm by 4.2 mm heaters of total size 25 mm by 42 mm was used to generate heat distributions of the type seen in large CPU’s, with core “rows” implemented at high power densities and remaining chip areas at low power densities. Effective local cold plate thermal resistance (measured local temperature increase relative to inlet fluid temperature over local power density) as low as $18^{\circ} \mathrm{C}-\mathrm{mm}^{2} / \mathrm{W}$ was measured which includes both the silicon die resistance and the pad thermal interface material thermal resistance. When the coolant pre-heat impact on the downstream core temperature is included in evaluation of a similar format high-performance single-phase water cold plate, the thermal performance is nearly identical. Overall, this approach appears very promising for water-free high-performance cooling of high local power density devices.

Increasing device power and the desire to operate systems with higher ambient temperature for reduced cooling facility power are driving a need for improved electronic system thermal solutions. Liquid cooling is one area of significant effort toward this goal. To create a reliable system, however, most high-power device thermal solutions are implemented in a lidded configuration. This configuration requires that the generated heat transit two thermal interface materials (TIM) and the lid. A lower thermal resistance approach is direct attach, where the liquid cooled heat sink is coupled with a single TIM to the device silicon. Generally, such an approach makes significant demands of the TIM, requiring substantial mechanical compliance and robustness. One solution to reducing these demands is to move at least some of the compliance requirements into the heat sink by making the heat sink conformable. Prior work has shown one very effective but expensive demonstration of this approach. In this work we present a different compliant direct attach heat sink approach that shows promise to be high performance, reliable, and cost competitive. A thin, flexible cold plate with distributed pressure loading is combined with a pad TIM resulting in overall junction to water thermal resistance values as low as 19 C-mm 2 /W. The cold plates are constructed in a mesh configuration utilizing two different stacked sheet assembly processes with total cold plate thickness ranging from 2-5 mm. Some versions are post-processed to reduce pressure drop. Thermal resistance for these cold plates was measured utilizing a uniform power test vehicle. Thermal performance and pressure drop results were obtained across a range of coolant flows. Initial power-cycling data with up to 7000 full-power cycles showed no negative change in thermal performance, indicating the potential for good reliability. Model results for the base mesh structure were matched against the experimental results and used to predict junction temperature improvement relative to a lidded structure for power maps more representative of high-performance product. These results showed potential peak temperature improvement of 14 °C or more for one representative power map with a peak power density of ~3 W/mm 2 . Overall, this technology shows substantial promise for significant product temperature reduction at competitive thermal solution cost.

Increasing power density in high performance microprocessors has driven research into high performance and energy efficient methods of cooling including both single-phase and two-phase liquid cooling. These approaches employ coldplates with complex high-density cooling structures such as fine-pitch microstructures, linked-pin-fins, metal foams, etc., to achieve high heat transfer coefficients. The development of coldplates, especially for two-phase cooling, with such complex cooling structures requires computationally efficient high fidelity thermal models to evaluate the device and system performance under different cooling configurations and operating environments. In this work, a novel reduced physics model based on porous media approximation was developed with homogeneous equilibrium model assumptions for the two-phase flow in the coolant flow domain. For an exemplar complex cooling structure, this reduced physics model produced pressure drop values close to those obtained from full-physics simulations while utilizing significantly less computational nodes and >100X less computational time.

We report on the successful design and fabrication of optical modules using a 50 micron pitch polymer waveguide interface, integrated for low loss, high density optical data transfer with very low space requirements on a Si photonics die. This prototype module meets JEDEC reliability standards and promises to increase the number of optical fibers that can be connected at the edge of a chip, a measure known as beachfront density, by six times compared to state of the art technology. Scalability of the polymer waveguide to less than 20 micron pitch stands to improve the bandwidth density upwards of 10 Tbps/mm.

WSPC Series in Advanced Integration and PackagingEmbedded Cooling of Electronic Devices, pp. 155-206 (2024) No AccessChapter 4: Chip-Embedded Two-Phase CoolingTimothy Chainer, Pritish Parida, and Mark SchultzTimothy ChainerIBM T. J. Watson Research Center, NY, USA, Pritish ParidaIBM T. J. Watson Research Center, NY, USA, and Mark SchultzIBM T. J. Watson Research Center, NY, USAhttps://doi.org/10.1142/9789811279379_0004Cited by:0 (Source: Crossref) PreviousNext AboutSectionsPDF/EPUB ToolsAdd to favoritesDownload CitationsTrack CitationsRecommend to Library ShareShare onFacebookTwitterLinked InRedditEmail Abstract: This chapter describes the experimental and modeling studies of two-phase chip-embedded cooling. The experimental investigation included chip-embedded cooling of a thermal test vehicle (TTV) and an embedded-cooled microprocessor (ECM). The modeling study included the development of both full- and reduced-physics two-phase flow boiling models, which were validated against the experimental data. The TTV was designed with chip-embedded radially expanding microchannels and a power map to mimic an eight-core microprocessor. The TTV enabled an experimental investigation of the limits of two-phase cooling, demonstrating cooling capability well beyond that of current microprocessor heat dissipation. Thermal modeling of the TTV chip-embedded cooling showed good agreement with the experimental data. The ECM module provided a test vehicle to directly compare chip-embedded cooling to a commercial air-cooled system. The air-cooled microprocessors in an IBM server were fully characterized in an environmental chamber and then modified by creating chip-embedded microchannels on the backside of the microprocessor die. A set of workloads was used to compare the performance of air- and embedded-cooled microprocessors. The ECM showed a significant reduction in chip junction temperature compared to air cooling, with the model in good agreement with experiments. The results showed that two-phase dielectric flow boiling provided highly effective cooling in microchannels with dimensions compatible with 3D chip stack cooling. Keywords: Electronics coolingtwo-phase coolingtwo-phase flowchip-embedded cooling FiguresReferencesRelatedDetails Recommended Embedded Cooling of Electronic DevicesMetrics History KeywordsElectronics coolingtwo-phase coolingtwo-phase flowchip-embedded coolingPDF download

Removal of heat from semiconductor at power densities of 1 W/mm 2 and higher has traditionally been the arena of liquid cooled cold plates in combination with a thermal interface material (TIM) coupling the cold plates to the heat generating devices. In standard rigid liquid cooled cold plate applications, the TIM is responsible to both conduct heat and absorb both static and dynamic mechanical surface mismatches between the cold plate and the device. This dual mission generally constrains the thermal performance of the TIM. In some applications like high power device test, the materials allowed for the TIM are further constrained, resulting in difficulty obtaining desired thermal performance. This work describes an innovative approach to conducting heat while providing mechanical compliance, allowing higher performance from most TIM’s and allowing a wider range of TIM’s for a given required thermal performance. The development of this approach from a heat conducting compliant interposer, to an interposer with an integrated high performance heat sink, to the final very high performance heat sink which flows liquid coolant through the compliant interposer structure incorporating high heat transfer fins is described. The heat sink provides junction to water thermal resistances as low as 16 Cmm 2 /W (including the full semiconductor die resistance) while greatly reducing variability in thermal resistance across the device being cooled when compared with rigid heat sinks. The high thermal performance is accomplished at reasonable pressure drop through the use of multiple manifolding of short 55 mm width channels and fins. Both mechanical and thermal response data are presented in detail.

The standard cooling approach for medium to high power microprocessor devices remains air cooling. However, increasing power density has driven overall interest in alternative, more efficient methods of cooling microprocessors. In this work we present and compare performance results for three different cooling approaches: air cooling, single-phase liquid cooling and chip-embedded two- phase liquid cooling each applied to the exact same processor devices. The devices were operated over a wide range of frequencies and workload, creating a range of operating powers that varied with the utilized cooling technology. The single- and two-phase liquid cooling approaches provide substantial reductions in system power usage and operating temperatures, with both approaches well-suited to chiller-less facility level implementation. The two-phase cooling approach has the advantage of extendibility to 3D stack cooling at the same level of cooling effectiveness. Overall, the improved thermal performance and efficiency of the described alternatives to air cooling make them prime candidates for implementation in future systems.

A new era of computing has begun with the development of high-performance computing (HPC), artificial intelligence (AI), machine learning (ML), and cognitive systems. Dramatic increases in the power density of the electronic components have led to the design and architecture of efficient thermal management technologies on these systems. IBM designed and delivered in 2018 the most powerful and fastest supercomputers of the world known as Summit and Sierra having 200 petaflops peak computing performance through LINPACK benchmarks. These systems which are called as IBM POWER AC922 are both air and liquid cooled, where water is employed in liquid-cooled systems to cool the high-power electronic components including IBM POWER9 processors and NVIDIA graphics processing units (GPUs). In this paper, we highlight the overview of the thermal and mechanical design strategies applied to these systems. Testing and experimental analysis with comparison to computational modeling is provided. Thermal control strategies are investigated for the optimization of overall system efficiency. In air cooled systems, we discuss the fan and heat sink designs, as well as the preheating effect on the PCIe section. In liquid-cooled systems, which have a unique cold plate design cooling the processors and the GPUs with water, we examine the water flow path design for the central processing units (CPUs), the GPUs, and the thermal performance of the cold plate. An overview of the cooling assemblies such as TIMs and air baffles in these systems is discussed. Unit and rack manifolds and rear door heat exchanger (RDHx) are investigated. Water flow and pressure distribution at the node and rack-level are provided.

The Intrachip Enhanced Cooling Fundamentals (ICECool Fun) effort was launched by the Defense Advanced Research Projects Agency (DARPA) under the leadership of Dr. Avram Bar-Cohen during 2012–2015 to target an order of magnitude improvement in chip level and hot spot heat fluxes, compared to the then state-of-the-art (SOA). Evaporative cooling technologies to achieve potential targets of 1 kW/cm2 at the chip level and 5 kW/cm2 at the hot spot level were targeted. A key goal was to improve fundamental understanding of the evaporative cooling physics at the relevant scales, and a numerical modeling capability to enable the co-design of such solutions in emerging computing and communications systems. A summary of the five projects pursued under this effort is provided, including the key accomplishments and developed capabilities.

High performance computing (HPC), artificial intelligence (AI) and cognitive systems have initiated a new era of computing. Efficient thermal management technologies of these systems have been vital due to the increasing power density in the electronic components. In 2018 IBM delivered the fastest supercomputer of the world through Summit with 200 petaflops computing performance with UNPACK benchmarks. The system is both air and water cooled, where water is employed to cool the high power dissipated electronic components which are the IBM POWER9 processors and NVIDIA GPUs. In this paper, we highlight the overview of the thermal and mechanical design strategies applied on these systems. In air cooled systems, we discuss the fan and heat sink designs, as well as the preheating effect on PCI section. Liquid cooled system has a unique coldplate design which cool the processors and the GPUs with water. We examine the water flow path design for the processor and the GPUs by providing the thermal performance of the coldplate. Also, an overview of the cooling assemblies such as TIMs and air baffles in the servers are discussed. Moreover, unit and rack manifolds are investigated; flow and pressure distribution at the node and rack level are provided.

Achieving optimal data center cooling efficiency requires of water cooling of high-heat-density components, coupled with optimal warmer water temperatures and the correct order of water preheating from any air-cooled components. The Summit and Sierra supercomputers implemented efficient cooling by using high performance cold plates to directly water-cool all central processing units (CPUs) and graphics processing units (GPUs) processors with warm inlet water. Cost performance was maximized by directly air-cooling the 10% to 15% of the compute drawer heat load generated by the lowest heat density components. For the Summit system, a rear-door heat exchanger allowed zero net heat load to air; the overall system efficiency was optimized by using the preheated water from the heat exchanger as an input to cool the higher power CPUs and GPUs.

We report on efforts to develop a high speed, low cost, low energy chip scale optical module for co-packaging on a first-level organic substrate for HPC and Data Center applications.

For high-speed computer systems, the high-power devices such as Central Processor Units (CPUs) and Graphics Processing Units (GPUs) generally must be arranged close together to reduce electrical channel routing distances. In such cases water cooling enables greater device density than that achievable with air cooling. In contrast to typical implementations that utilize a separate coldplate for each high-power device, we use a single flexible coldplate to cool multiple devices. The flexibility allows the single coldplate to adapt to varying device heights and/or tilts. We solder or braze the perimeters of two thin, concave metal sheets to form a coldplate chamber. Cooling fin packs inside the chamber are soldered or brazed to internal chamber walls. Careful control of this metal-to-metal interface is required to minimize voiding and achieve good thermal contact. Under the fin packs the external chamber wall contacts the heat generating devices through a thermal interface material (TIM). In the regions between devices, the thin walls of the empty cavity provide mechanical flexibility. This allows for device height variation while maintaining full contact through a thin TIM. Having a single coldplate reduces both cost and potential water leakage at attachment junctions.

The development of two-phase cooling for both two-dimensional (2D) and three-dimensional (3D) integrated circuits using pumped dielectric coolant has gained recent attention due to the ability to manage high heat densities, compatibility with electronics and low cooling energy usage. Development of this approach requires high fidelity and computationally manageable conjugate thermal models both at the device level as well as the system level. Recent studies have shown micro-scale modeling of in-situ thermal and hydrodynamic phenomena (e.g. boiling and bubble dynamics) of two-phase flow boiling with good accuracy. This paper describes a simulation tool named Sim2Cool which addresses the need for macro-scale system level design simulation for two-phase cooling. The Sim2Cool simulation tool uses an enthalpy-based / energy balance model for the system components and a reduced order model for device-level model. The system simulator tool allows system component parameters to be quickly analyzed including; condenser, reservoir and tubing diameter as well as heat dissipation and flow rate to quantify the impact on the system coefficient of performance (COP). Such tools are critical to accelerating the integration of two phase cooling with microelectronics to realize performance gains by way of high density electronic packaging as Moore’s law on silicon technology advancement reaches physics limits.