The present application relates generally to heat transfer mechanisms used for cooling electronic devices, and more particularly, to liquid cooling apparatuses for removing heat generated by one or more electronic devices.
Recent trends in global digital transformation have created incredible demand for increased processing performance in colocation and edge deployments of data/information processing servers. Datacenters today rely upon high power microprocessor devices, such as central processors (CPUs) and graphic processing units (GPUs), which generate a high level of heat in a small area. Traditional use of air as the heat transfer medium to cool heat dissipating components is unable to meet the thermal dissipation requirement of these high-power microprocessor devices. Thus, liquid cooling using cold plates have become a preferred way to provide the required cooling. Cold plates are a type of heatsink that allows for a cooling liquid to be brought into thermal conduction contact with the heat generating electronic components of servers and other information processing systems. These cold plates rely upon ultra-narrow fluid passages called “microchannels” to dissipate heat from the processors into the cooling liquid.
One challenge of constructing cold plates is that, unlike air-cooled heatsinks, cooling liquids have extremely high heat transfer coefficients that create diminished returns on fin height (commonly referred to as fin efficiency) because it is difficult to conduct heat through narrow passages using conventional metals. Fin efficiency limits typically result in cold plate fin heights that are less than 4 mm. This means that cold plates are effectively a two-dimensional heat sink receiving a single directional flow of cooling liquid over the fins.
The description of the illustrative embodiments can be read in conjunction with the accompanying figures. It will be appreciated that for simplicity and clarity of illustration, elements illustrated in the figures have not necessarily been drawn to scale. For example, the dimensions of some of the elements are exaggerated relative to other elements. Embodiments incorporating teachings of the present disclosure are shown and described with respect to the figures presented herein, in which:
According to a first aspect of the present disclosure, an encapsulated heatsink (“a cold plate assembly”) is configured for cooling heat generating electronic components with liquid coolant that flows around inserted channel obstructions, which generates swirling, resulting in high vorticity flow and increased heat absorption. In one or more embodiments, a cold plate of the cold plate assembly has a conductive substrate with extended fin features providing microchannels of narrow hydraulic diameter that is enhanced for more efficiently liquid cooling with implementation of a swirl enhancing manifold solution. The swirl enhancing manifold solution is provided by a swirl enhancement plate having geometrically optimized posts that extend into the liquid flow path of each microchannel to create high vorticity flow. This high vorticity flow disrupts the thermal boundary layer growth on each surface, which, in turn, induces a significantly larger convection heat transfer coefficient than laminar flow typically observed in microchannel cold plates. In one or more embodiments, the swirl enhancing manifold solution also includes jet impingement nozzles that create high velocity and high heat transfer coefficient convection flows on the heated surfaces of the cold plate impinged by the incoming flow of cooling liquid.
In one or more embodiments, the cold plate assembly includes a cold plate manufactured from a thermally conductive material and having a first surface attachable to a heat generating electronic component. The cold plate has a second surface having an array of extended fins. The second surface is opposite to the first surface. A swirl enhancement plate of the cold plate assembly is positioned as a first layer proximate and parallel to the second surface. The swirl enhancement plate has at least one nozzle opening to channel one or more corresponding impingement nozzle jets of cooling liquid toward the second surface. The swirl enhancement plate has a plurality of swirl enhancement protrusions each extending toward the second surface in a corresponding microchannel between two adjacent extended fins of the array of extended fins. The swirl enhancement protrusions induce the swirl or turbulence to cooling liquid flow between the adjacent extended fins. The swirl or turbulence induced in the cooling liquid flow disrupts a thermal boundary layer of the cooling liquid flow, resulting in increased liquid heat transfer at the second surface. An encapsulating lid of the cold plate assembly is attachable to the second surface to form a liquid cooling cavity encompassing the array of extended fins and swirl enhancement plate. The encapsulating lid includes an intake port for receiving the cooling liquid flow from a cooling liquid source. The encapsulating lid includes an exhaust port for receiving and expelling exhaust cooling liquid that has flowed across the second surface of the cold plate.
According to a second aspect of the present disclosure, an information processing system includes at least one heat generating electronic component and the above-introduced cold plate assembly thermally attached/connected to the at least one heat generating electronic component to provide liquid cooling. In a third aspect of the present disclosure, a data center includes an information processing system rack having an information processing system that includes at least one heat generating electronic component and the above-introduced cold plate assembly thermally attached/connected to the at least one heat generating electronic component to provide liquid cooling.
Future central processing units (CPUs) and application specific integrated circuits (ASICs) are pushing past the current boundaries of heat generation during information processing within an information processing system. While liquid cooling is the preferred method for addressing the high thermal yields of these devices, the standard fin geometries of conventional cold plates have a limited surface area in a relative 2-dimensional (2D) space and thus cannot provide sufficient cooling for these devices as the devices increase their thermal output. Creating larger surface area on a conventional 2D cold plate, geometrically requires creating ever smaller and smaller fin spacings. Current technology allows manufacturers to create 0.2-0.4 mm channels which, while ideal for 2D heat transfer, are very intolerant of debris, particles, and corrosion agents within the cooling liquid, necessitating specialized water quality control via secondary coolant loop (i.e., technology cooling system (TCS)). Also, as the cooling liquid flows from an intake side of the microchannels to the exhaust side, thermal boundaries are created at the surface of the cooling liquid, which results in less efficient heat absorption towards the exhaust end of the microchannel. As a result, more cooling liquid flow is required to complete the heat absorption at the exhaust end of the microchannel, with an imbalance in reduced heat absorption efficiency as the cooling liquid flows along the length of the microchannel. The limits of these conventional systems have been reached. The present disclosure addresses and overcomes these limitations in existing cold plate technology by providing a swirl plate enhancement to disrupt a thermal boundary layer of the cooling liquid at a surface of the cold plate. Additionally, in one or more embodiments, the present disclosure provides more efficient heat transfer and thermal dissipation from the attached heat generating electronic component by providing phase change heat transfer using a saturated working fluid within a phase change cavity (“vapor chamber”) placed between the heat generating electronic component and the bottom surface of the attached cold plate. Alternatively, or in addition, the cold plate incorporates a vapor chamber. Accordingly, the cold plate assembly provides a more efficient heat exchange mechanism for use within a liquid cooling loop of a datacenter facility or other operating environment.
The present disclosure also overcomes deficiencies in the existing liquid cooling solutions caused by the sensitive nature of the convention cold plates, which are susceptible to corrosion and clogging if exposed to a flow of regular liquid. Specifically, one aspect of the present disclosure provides an environmentally hardened cold plate and a corresponding environmentally hardened cold plate assembly and liquid cooling system that are resistive to the fouling from direct exposure to facility water and thus enables the data center cooling solution to be provided with a single cooling loop utilizing the facility water supply.
In the following detailed description of exemplary embodiments of the disclosure, specific exemplary embodiments in which the various aspects of the disclosure may be practiced are described in sufficient detail to enable those skilled in the art to practice the invention, and it is to be understood that other embodiments may be utilized and that logical, architectural, programmatic, mechanical, electrical, and other changes may be made without departing from the spirit or scope of the present disclosure. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present disclosure is defined by the appended claims and equivalents thereof. Within the descriptions of the different views of the figures, similar elements can be provided with similar names and reference numerals as those of the previous figure(s). The specific numerals assigned to the elements are provided solely to aid in the description and are not meant to imply any limitations (structural or functional or otherwise) on the described embodiment. It will be appreciated that for simplicity and clarity of illustration, elements illustrated in the figures have not necessarily been drawn to scale. For example, the dimensions of some of the elements are exaggerated relative to other elements.
It is understood that the use of specific component, device and/or parameter names, such as those of the executing utility, logic, and/or firmware described herein, are for example only and not meant to imply any limitations on the described embodiments. The embodiments may thus be described with different nomenclature and/or terminology utilized to describe the components, devices, parameters, methods and/or functions herein, without limitation. References to any specific protocol or proprietary name in describing one or more elements, features or concepts of the embodiments are provided solely as examples of one implementation, and such references do not limit the extension of the claimed embodiments to embodiments in which different element, feature, protocol, or concept names are utilized. Thus, each term utilized herein is to be given its broadest interpretation given the context in which that term is utilized. As a specific example, reference is made herein to the term facility liquid, facility water, facility cooling liquid, facility-grade cooling liquid, and cooling liquid. It is appreciated that the terms facility liquid or facility cooling liquid are utilized to provide a specific example of a cooling liquid supplied by/at a datacenter facility in which the heat generating electronic components are operating and being cooled in a single liquid cooling loop that includes facility liquid, which is typically unpurified water. Facility-grade cooling liquid is a more general term that can apply to both cooling liquid that is being provided at a datacenter facility or any other cooling liquid that can contain contaminants and particulates, similar to the normal facility liquid found in datacenters. Thus, facility-grade cooling liquid can apply to any type of liquid, regardless of the source of the liquid, and can also apply to liquid cooling that is not provided at a “facility” or datacenter. The descriptions herein are meant to apply to any type of facility-grade cooling liquid. Additionally, it is appreciated that the cooling liquid utilized within the cooling loop that includes the described cold plate assembly can also be a higher-grade cooling liquid than facility-grade cooling liquid, without limitation.
As further described below, implementation of the functional features of the disclosure described herein is provided within processing devices and/or structures and can involve use of a combination of hardware, firmware, as well as several software-level constructs (e.g., program code and/or program instructions and/or pseudo-code) that execute to provide a specific utility for the device or a specific functional logic. The presented figures illustrate both hardware components and software and/or logic components.
Those of ordinary skill in the art will appreciate that the hardware components and basic configurations depicted in the figures may vary. The illustrative components are not intended to be exhaustive, but rather are representative to highlight essential components that are utilized to implement aspects of the described embodiments. For example, other devices/components may be used in addition to or in place of the hardware and/or firmware depicted. The depicted example is not meant to imply architectural or other limitations with respect to the presently described embodiments and/or the general invention. The description of the illustrative embodiments can be read in conjunction with the accompanying figures. Embodiments incorporating teachings of the present disclosure are shown and described with respect to the figures presented herein.
Lateral dimensions of cold plate 110 may accommodate one or more than one heat generating electronic component 102 (
To facilitate use of a facility cooling liquid, such as unpurified water, liquid cooling component 1003 is environmentally hardened, such as by applying a coating in physical vapor deposition (PVD) apparatus 1001. PVD apparatus 1001 includes three-dimensional (3D) vapor chamber 1018 that is filled with sputtering gas 1020 received from sputtering gas supply 1022 and that is maintained at an appropriate pressure by vacuum system 1024 that removes excess gas 1025. Upward oriented surface 1023 of liquid cooling component 1003 is engaged to substrate holder 1026, which is connected to electrical ground 1027. Downward oriented surface 1031 of liquid cooling component 1003 is oriented toward target 1028, which includes one or more coating materials to be deposited onto downward oriented surface 1031 of liquid cooling component 1003 by PVD. Target 1028 is connected to power supply 1030 to cause ionization of sputtered atoms 1032, 1034, and 1036, which are attracted by the voltage difference between target 1028 and liquid cooling component 1003 to that transit across vacuum chamber 1018 and be deposited onto downward oriented surface 1031 of liquid cooling component 1003 to form a protective coating. In an example, first atom 1032 represents a hydrophobic material, second atom 1034 represents a non-conductive material, and third atom 1036 represents an anti-corrosive material. Wetted surfaces cold plate 110 are coated with non-conductive, anti-corrosion surface enhancements such as Zirconium Nitride, Titanium Nitride, and other ceramics applied by PVD. In one or more embodiments, the surface may also be coated with a hydrophobic surface treatment to mitigate scale and sedimentation. The specific description of PVD as the process is thus not intended to be limiting on the scope of the disclosure. To coat all wetted surface of liquid cooling component 1003, liquid cooling component 1003 may be processed by PVD while in more than one orientation so that each wetted surface is coated.
It is appreciated that while the coating process is shown to be completed by PVD, other similar techniques can be utilized to provide the coating layer(s) on the first surface of cold plate to yield the various physical and chemical enhancements that are described herein. In an example, liquid cooling component 1003 or an assembly of liquid cooling components 1003 may be dipped or immersed into liquid that provides the surface treatment.
Facility water, which may contain corrosive chemical and particulate contaminants would be suboptimal for liquid cooling using conventional cold plates. In one or more embodiments, the protective coating applied to the cold plate by the above-described PVD process provides one or more protective characteristics, including: (i) a hydrophobic characteristic to mitigate scaling by calcium carbonate, (ii) an anti-corrosive characteristic to mitigate formation corrosion, and (iii) a non-conductive characteristic to mitigate rusting. Cold plate assembly 100 covered at least in part with the resulting coating is designed to resist heat transfer inhibiting failure modes common to liquid cooling with poorly regulated water quality, namely suspended solid particulates, biological growth potential, and harsh water chemistry conditions, including semi-corrosive mixtures.
In one or more embodiments, as shown by
Cooling liquid supply 501a enters intake port 128 of encapsulating lid 112. Intake manifold 132 channels distributed supply cooling liquid 501b to one or more nozzle opening 136 of swirl enhancement plate 101. In an example, nozzle opening 136 is centrally aligned, creating lateral split flows 501c-501d that move in opposite directions. As depicted, lateral split flow 501c moves to the left into beginning portion 503a of exhaust manifold 138 to start clockwise flow 501e around peripheral portion 401 of liquid cooling cavity 126. As also depicted, lateral split flow 501d moves to the right into ending portion 503b of exhaust manifold 138 to join clockwise flow 501f to exit from exhaust port 130 as heated cooling liquid 501g.
IPS 1102 includes one or more heat generating electronic component(s), presented as example heat generating electronic component 102 (
Aspects of the present disclosure may be applied to an edge mobile datacenter or facility with a single rack IPS. Additionally, aspects of the present disclosure may also be applied to an enterprise datacenter having multiple buildings and rooms with tens, hundreds, or thousands of rack IPSes. In one or more embodiments, a facility source of facility-grade cooling liquid may include a facility outlet port and a facility return port. A first branch of liquid distribution system may be sealably connected between the intake port of the cold plate assembly and an outlet port of the facility source to channel unheated facility water to the cold plate assembly, and a return branch of liquid distribution system may be sealably connected between the exhaust port of the cold plate assembly and the facility return port to channel heated exhaust water from the cold plate assembly to the facility return port.
In one or more embodiments, the liquid distribution system includes a rack liquid cooling manifold system with a supply manifold and a return manifold. The supply manifold includes a supply control valve and a manifold intake port available for sealably coupling to a facility water supply to receive a cooling liquid. The supply manifold includes more than one server supply ports. Each server supply port is available for sealably coupling, for liquid transfer of the cooling liquid, to a respective cooling liquid supply input of a corresponding information processing system node supported by a rack frame capable of supporting multiple information processing system nodes.
Each information processing system node has one or more heat generating electronic component. The return manifold includes a facility water return port for sealably coupling to a facility return to exhaust the cooling liquid. The return manifold includes more than one server return ports. Each server return port is available for sealably coupling, for exhaust liquid transfer, to a respective cooling liquid exhaust output of the corresponding information processing system node. The respective cooling liquid exhaust output and a paired supply liquid cooling input directs cooling liquid flow through one or more cold plate assembly positioned within the corresponding information processing system node to thermally cool the one or more heat generating electronic components.
In one or more particular embodiments, the liquid distribution system further includes a plurality of conduits that sealably couple for liquid transfer: (i) the more than one server supply ports of the supply manifold to the corresponding server supply inputs of the more than one information processing system nodes; (ii) the corresponding server supply input to the one or more cold plate assemblies in the corresponding information processing system node; (iii) the one or more cold plate assemblies in the corresponding information processing system node to the corresponding server return output; and (iv) the more than one server return outputs to the server return ports of the return manifold.
IPS 1102 may be located in a node of an information processing system rack of a datacenter equipment center with rack-level liquid cooling of heat generating electronic components, each utilizing an attached one of the cold plate assembly 100 of
Aspects of the present disclosure may be applied to multiple cooling loop systems. In an example, a liquid cooling system may include four isolated liquid cooling loops in series, a technology cooling system (TCS), a facility cooling liquid system (FCLS), and a condenser liquid system that support rack IPSes. A chiller loop may be counted as an additional liquid cooling loop. TCS is supported by a cooling distribution unit (CDU) that circulates cooling liquid through an IPS via a closed loop. CDU transfers thermal energy from the TCSs to facility cooling liquid such as unpurified water provided by a facility cooling system (FCLS). In an example, FCLS may transfer exhaust thermal energy directly into ambient air via a cooling tower. The liquid cooling system may further include a condenser liquid system (CLS) that circulates cooling water through the cooling tower. The liquid cooling system may further include the chiller loop (or chiller) that stores a thermal buffering quantity of liquid such as water. The chiller circulates the water through a first liquid-to-liquid heat exchanger to receive thermal energy from FCLS. The chiller circulates the heated water through a second liquid-to-liquid heat exchanger to transfer thermal energy to the CLS. The chiller enables intermittent use of the cooling tower while maintaining water in the chiller within a temperature range suitable for both FCLS and CLS.
Utilization of a cold plate assembly to provide liquid cooling of heat generating components within the data center eliminates the need for use of purified liquid in a separated rack level technology cooling system and the multiple separated cooling loops to provide reliable operation of the liquid cooling system. An existing data center cooling system can be retrofitted with IPSes that are configured with a plurality of a cold plate assembly as the mechanism for colling the heat generating components within the IPS racks and replace the more expensive purified cooling liquids within the technology cooling system loop, without having issues related to fouling of the cold plates with the switch to utilizing normal, facility rated liquid supply.
With reference to
The at least one nozzle opening is configured to channel one or more corresponding impingement jets of cooling liquid toward the second surface of the cold plate. Each swirl enhancement protrusion is configured to extend toward the second surface in a corresponding microchannel between two adjacent extended fins of the array of extended fins. The swirl enhancement protrusions induce swirl or turbulence to cooling liquid flow between the adjacent extended fins. The swirl or turbulence disrupts a thermal boundary layer of the cooling liquid flow, resulting in increased liquid heat transfer at the second surface by inducing swirl or turbulence into the cooling liquid flow. In one or more embodiments, the swirl enhancing protrusions are distally tapered to create a larger space between extended fins proximate to the second surface to enable particles in the cooling liquid flow to pass through the extended fins and around the swirl enhancing protrusions. In one or more particular embodiments, the distal taper of the swirl enhancing protrusions are located at least 800 microns from each adjacent extended fin and the encapsulating lid is configured to maintain a flow velocity of the liquid flow to at least 0.7 m/s to prevent sedimentation within the array of extended fins. In one or more particular embodiments, each of the swirl enhancement protrusions have a noncircular geometric shape configured to increase convection heat transfer performance.
In one or more embodiments, method 1200 may include manufacturing the swirl enhancement plate from metal (block 1210). Manufacturing the swirl enhancement plate from metal may precede additional hardening operations such as described below in method 1300 of
With reference to
With the cold plate assembly completed, method 1200 may optionally include positioning a 2D vapor chamber between the heat generating electronic component and the first surface of the cold plate prior to attaching the heat generating electronic component to the first surface of the cold plate (block 1218). Method 1200 includes attaching the heat generating electronic component of an information processing system to the first surface of the cold plate (or to the bottom surface of the included vapor chamber) for heat transfer away from the heat generating electronic component to the second surface of the cold plate (block 1220). The 2D vapor chamber, if included, is interposed between the heat generating electronic component and the cold plate. The 2D vapor chamber laterally distributes and evens out thermal energy. As compared to lateral dimensions, the 2D vapor chamber may be relatively thin and thus is referred herein as “2D”. The 2D vapor chamber does include sufficient vertical space to facilitate lateral distribution and vertical transfer of thermal energy. The resulting heat generating electronic component can then be liquid cooled during operation by attaching a first conduit from a liquid supply to the intake port and a second conduit from the exhaust port to a rack manifold or facility liquid return (block 1222). Then, method 1200 ends.
In one or more embodiments, the cold plate is manufactured by utilizing copper as the thermally conductive material. The first surface of the cold plate is a heat receiving surface for attaching to a heat generating electronic component directly, or indirectly through an attached vapor chamber. The second surface of the cold plate is a heat transfer surface. In one or more embodiments, in manufacturing the cold plate, method 1200 further includes configuring the fin stack with the levels of fins spaced apart at least 800-microns to facilitate passage of the facility liquid particulates. In one or more embodiments, in manufacturing the encapsulating lid of the cold plate assembly, method 1200 further includes configuring the intake port, exhaust port, and volumetric space of the liquid cooling cavity to maintain a flow velocity of at least 0.7 m/s of liquid impinging the fin stack to prevent sedimentation.
With reference to
In one or more embodiments, as presented in blocks 1304, 1306, and 1308, applying the coating may include only one of the three characteristics (i.e., hydrophobic, non-conductive, and anti-corrosive) in a coating, two of the three characteristics in the coating, or all three of the characteristics in a coating. In one or more embodiments, as presented in blocks 1304, 1306, and 1308, applying the coating may further include coating the one or more of the wetted surfaces that may be exposed to cooling liquid with more than one coating of surface treatment, each coating providing one or more of the three characteristics. In one or more embodiments, as presented in block 1304, coating the one or more of the wetted surfaces that may be exposed to cooling liquid includes applying a hydrophobic layer to prevent scaling and sedimentation due to dissolved calcium carbonate in the facility liquid. In one or more embodiments, method 1300 further includes coating the one or more of the wetted surfaces that may be exposed to cooling liquid using physical vapor deposition (PVD). In one or more embodiments, method 1300 further includes coating the one or more of the wetted surfaces that may be exposed to cooling liquid, via PVD, with one or more ceramics from among a group comprising Zirconium Nitride and Titanium Nitride.
With reference to
Aspects of the present innovation are described above with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the innovation. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
As will be appreciated by one skilled in the art, embodiments of the present innovation may be embodied as a system, device, and/or method. Accordingly, embodiments of the present innovation may take the form of an entirely hardware embodiment or an embodiment combining software and hardware embodiments that may all generally be referred to herein as a “component”, “circuit,” “module” or “system.”
While the innovation has been described with reference to exemplary embodiments, it will be understood by those skilled in the art that various changes may be made, and equivalents may be substituted for elements thereof without departing from the scope of the innovation. In addition, many modifications may be made to adapt a particular system, device, or component thereof to the teachings of the innovation without departing from the essential scope thereof. Therefore, it is intended that the innovation not be limited to the particular embodiments disclosed for carrying out this innovation, but that the innovation will include all embodiments falling within the scope of the appended claims. Moreover, the use of the terms first, second, etc. do not denote any order or importance, but rather the terms first, second, etc. are used to distinguish one element from another.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the innovation. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprise” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the present innovation has been presented for purposes of illustration and description but is not intended to be exhaustive or limited to the innovation in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the innovation. The embodiments were chosen and described in order to best explain the principles of the innovation and the practical application, and to enable others of ordinary skill in the art to understand the innovation for various embodiments with various modifications as are suited to the particular use contemplated.