Patent Application
20120097370
The invention relates to a method of and apparatus for cooling a heat producing device in general, and specifically, to a method of and apparatus for cooling server applications using fluid-based cooling systems.
Cooling of high performance integrated circuits with high heat dissipation is presenting significant challenge in the electronics cooling arena. Conventional cooling with heat pipes and fan mounted heat sinks are not adequate for cooling chips with ever increasing wattage requirements.
Electronics servers, such as blade servers and rack servers, are being used in increasing numbers due to the higher processor performance per unit volume one can achieve. However, the high density of integrated circuits also leads to high thermal density, which is beyond the capability of conventional air-cooling methods.
A particular problem with cooling integrated circuits on electronics servers is that multiple electronics servers are typically mounted in close quarters within a server chassis. In such configurations, electronics servers are separated by a limited amount of space, thereby reducing the dimensions within which to provide an adequate cooling solution. Typically, stacking of electronics servers does not provide the mounting of large fans and heat sinks for each electronics server. Often electronics server stacks within a single server chassis are cooled with one or more fans, one or more heat sinks, or a combination of both. Using this configuration, the integrated circuits on each electronics server are cooled using the heat sink and the large fan that blows air over the heat sink, or simply by blowing air directly over the electronics servers. However, considering the limited free space surrounding the stacked electronics servers within the server chassis, the amount of air available for cooling the integrated circuits is limited.
As data centers continue to increase their computer density, electronics servers are being deployed more frequently. Fully populated electronics servers significantly increase rack heat production. This requires supplemental cooling beyond what the Computer Room Air Conditioning (CRAC) units can provide. Supplemental cooling systems can include fluid based cooling systems implemented at the server rack level and distributed to each electronics server mounted within the server rack. In some applications, the supplemental cooling system distributes cooling fluid to each docking bay within the server rack, where each docking bay is configured to receive an electronics server. The cooling fluid distributed to each docking bay is distributed to either a discrete fluid based cooling system included within the received electronics server, or to a discrete fluid based cooling system that is part of each docking bay where the discrete fluid based cooling system is thermally coupled to the received electronics server.
When providing fluid to a server rack, careful consideration must be given to the distribution of the cooling fluid within the rack. Typically, the fluid path contains one or more parallel paths. If the fluid flow is a singular series path, then for a single phase fluid the temperature would rise all along the path. By splitting the fluid into parallel paths, cooler fluid can be presented to the devices requiring cooling, such as the electronics servers, and there is an overall lower pressure drop from the parallel paths.
To balance the flow for each of the parallel paths, the fluid pressure drop can be modified such that the appropriate amount of fluid flows down each path. In some applications, each path may require the same amount of fluid, which would then require the same pressure drop. In other applications, each path may have its own requirement so the resistance is tailored accordingly. This works fine for a single phase coolant such as water. The fluid path pressure drop is largely insensitive to the heat load being applied. However, in a two-phase cooling system, such as a refrigerant based cooling system, as heat is transferred into the fluid, a portion of the fluid undergoes a phase change and changes from liquid to gas. Two-phase cooling systems are often desirable because the temperature of the two-phase fluid is constant throughout the fluid pathway, thereby maintaining a heat exchanger within the cooling system at a substantially constant temperature. Because the mass flow rate is the same, the gas must travel at higher velocity which subsequently causes a higher pressure drop. If there are multiple parallel paths with the same heat load, then the pressure drop along each path stays largely the same and the flow rate stays balanced. However, if the heat load is higher on one path than the other, then the refrigerant flows more toward the low load path since the low load path has a lower pressure drop. An example of this may occur within a refrigerant cooled server rack having multiple electronics servers. If some electronics servers are powered on and others are powered off, the refrigerant will naturally flow toward the electronics servers that are powered off causing the electronics servers that are powered on to overheat. One method of addressing this issue is using an active valve system that closes fluid flow to the electronics servers that are off. This however is expensive and can be prone to failure. Another method is that the overall fluid flow can be increased to make sure there is adequate fluid flow in the path with the high heat load, but this requires a larger fluid pump and more power.
A cooling system is configured to distribute fluid to a plurality of docking bays within a server rack. The server rack is configured to house multiple electronics servers, each electronics server mounted within a docking bay of the server rack. The cooling system is configured with parallel fluid paths, one fluid path to distribute fluid to one docking bay. A two-phase fluid-based heat exchanging system is coupled to each docking bay, which functions to remove heat from an electronics server when mounted in the docking bay. The docking bays are conceptually divided into one or more sections, and a dynamic fluid flow regulator is included within a section input line that supplies a liquid-phase fluid to each section. The section input line branches into a plurality of parallel fluid paths, one parallel fluid path coupled to each docking bay in the section. A fixed fluid flow regulator is included within each branching fluid pathway. The combination of the dynamic fluid flow regulators and fixed fluid flow regulators provides balanced fluid flow to each two-phase heat exchanging system coupled to the docking bays.
In an aspect, a cooling assembly is disclosed that includes a plurality of fluid-based two-phase cooling systems and a fluid input manifold configured to supply a coolant in a liquid-phase to each of the plurality of fluid-based cooling systems. The fluid input manifold includes a fluid input line, one or more dynamic fluid flow regulators, and a plurality of fixed fluid flow regulators. The one or more dynamic fluid flow regulators are coupled in parallel to the fluid input line, wherein each dynamic fluid flow regulator is configured to provide a variable fluid flow resistance. The resistance increases with an increase of pressure on the inlet side. One fixed fluid flow regulator is coupled to each cooling system. The plurality of fixed fluid flow regulators are arranged into one or more groups, one group coupled to each dynamic fluid flow regulator such that all fixed fluid flow regulators in the group are coupled in parallel to the dynamic fluid flow regulator, further wherein each fixed fluid flow regulator is configured to provide a fixed fluid flow orifice.
In some embodiments, the cooling system also includes a fluid output manifold configured to receive two-phase coolant from each of the plurality of cooling systems. In some embodiments, the fluid output manifold includes a plurality of cooling system output lines, one cooling system output line coupled to each cooling system, and a cooling assembly output line coupled to each of the plurality of cooling system output lines. In some embodiments, a diameter of the fluid input line is smaller than a diameter of the cooling assembly output line. In some embodiments, the cooling system also includes a frame including a plurality of docking bays, each docking configured to receive a heat generating electronics device, wherein one of the plurality of cooling systems is coupled to one of the docking bays. In some embodiments, each cooling system is configured to be mounted to the heat generating electronics device when the heat generating electronics device is mounted within the docking bay. In some embodiments, each cooling system includes one or more heat exchangers configured to pass the coolant therethrough. In some embodiments, at least a portion of the coolant undergoes a phase change within the cooling system. In some embodiments, each dynamic fluid flow regulator is configured to output a constant fluid flow rate in response to a range of fluid pressures of the coolant within the fluid input manifold. In some embodiments, the coolant is a refrigerant. In some embodiments, the fluid input manifold also includes a plurality of cooling system input lines, one cooling system input line coupled between one orifice tube and one cooling system. In some embodiments, each fixed fluid flow regulator includes one or more filters.
In another aspect, a cooling assembly includes a frame, a fluid input manifold, and a fluid output manifold. The frame includes a plurality of docking bays, each docking bay configured to receive a heat generating electronics device. The fluid input manifold is configured to supply a coolant in a liquid-phase to each of the plurality of docking bays. The fluid input manifold includes a fluid input line, one or more dynamic fluid flow regulators, and a plurality of orifice tubes. The one or more dynamic fluid flow regulators are coupled in parallel to the fluid input line, wherein each dynamic fluid flow regulator is configured to provide a variable fluid flow resistance. One orifice tube is coupled to each docking bay. The plurality of orifice tubes are arranged into one or more groups, one group coupled to each dynamic fluid flow regulator such that all orifice tubes in the group are coupled in parallel to the dynamic fluid flow regulator, further wherein each orifice tube is configured to provide a fixed fluid flow orifice. The fluid output manifold is configured to receive two-phase coolant from each of the plurality of docking bays.
In yet another aspect, a cooling assembly includes a plurality of fluid-based two-phase cooling systems and a fluid input manifold configured to supply a coolant in a liquid-phase to each of the plurality of fluid-based cooling systems. The fluid input manifold includes a fluid input line and a plurality of fixed fluid flow regulators coupled in parallel to the fluid input line, one fixed fluid flow regulator coupled to each cooling system, wherein each fixed fluid flow regulator is configured to provide a fixed fluid flow orifice.
Other features and advantages of the present invention will become apparent after reviewing the detailed description of the embodiments set forth below.
Several example embodiments are described with reference to the drawings, wherein like components are provided with like reference numerals. The example embodiments are intended to illustrate, but not to limit, the invention. The drawings include the following figures:
Embodiments of the present application are directed to a cooling system. Those of ordinary skill in the art will realize that the following detailed description of the cooling system is illustrative only and is not intended to be in any way limiting. Other embodiments of the cooling system will readily suggest themselves to such skilled persons having the benefit of this disclosure.
Reference will now be made in detail to implementations of the cooling system as illustrated in the accompanying drawings. The same reference indicators will be used throughout the drawings and the following detailed description to refer to the same or like parts. In the interest of clarity, not all of the routine features of the implementations described herein are shown and described. It will, of course, be appreciated that in the development of any such actual implementation, numerous implementation-specific decisions must be made in order to achieve the developer's specific goals, such as compliance with application and business related constraints, and that these specific goals will vary from one implementation to another and from one developer to another. Moreover, it will be appreciated that such a development effort might be complex and time-consuming, but would nevertheless be a routine undertaking of engineering for those of ordinary skill in the art having the benefit of this disclosure.
Embodiments of the present application are directed to a cooling system that distributes fluid to a plurality of docking bays within a server rack. The cooling system described herein can be applied to any electronics sub-system, including but not limited to, a blade server and a rack server, herein referred to collectively as an electronics server. A server rack is configured to house multiple electronics servers, each electronics server mounted within a docking bay of the server rack. Each electronics server includes one or more heat generating devices as is well known in the art.
The cooling system is configured with parallel fluid paths, one fluid path to distribute fluid to one docking bay. A two-phase fluid-based heat exchanging system is coupled to each docking bay, which functions to remove heat from an electronics server when mounted in the docking bay. One solution to addressing the pressure drops associated with the two-phase heat exchanging systems is the use of dynamic fluid flow regulators. In some embodiments, each parallel fluid path includes a dynamic fluid flow regulator. However, dynamic fluid flow regulators are expensive and including a dynamic fluid flow regulator in each parallel fluid path may be cost prohibitive.
Another more passive solution is to artificially raise the pressure drop on each of the parallel fluid paths such that the pressure change caused by evaporation is a fraction of the overall pressure drop. Under these conditions, fluid flow remains largely unaffected preventing starvation of fluid from hot electronics servers when there are cold electronics servers, or no electronics server in the case of a vacant docking bay, in a parallel path. This can be achieved by using a small orifice on each of the parallel paths or by using a very thin tube of a predetermined length to artificially raise the pressure drop. The smaller sized orifice restricts fluid flow. The amount of fluid flow restriction is a function of the diameter of the small orifice. As the size of the small orifice is fixed, so is the fluid flow restricted by the small orifice. In this manner the small orifice functions as a fixed fluid flow regulator. A problem with an orifice is that it can become clogged. Also, a small diameter tube can be crushed or kinked thus preventing flow altogether. A small tube joined to a large tube can also be a problem for manufacturing. Damage can occur to the small tube while brazing a small tube to a larger manifold tube. These issues are addressed by an orifice tube that includes one or more filters and a smaller diameter fluid line positioned so as to float within a larger diameter fluid tube. The fluid is suspended by a support structure that holds both the filters in place relative to the fluid line and the fluid line in place relative to the fluid tube.
Yet another solution is to combine the use of dynamic fluid flow regulators and fixed fluid flow regulators. This solution provides a balance fluid flow in each of parallel fluid paths by using one or more dynamic fluid flow regulators to form smaller manifolds for fluid distribution. These smaller manifolds, also referred to as sections, can additionally supply one or more parallel fluid paths. The fluid paths within each smaller manifold are modified to include fixed fluid flow regulators. In some embodiments, the fixed fluid flow regulators are implemented using orifice tubes. In some embodiments, the orifice tubes are configured with one or more filters to minimize or prevent blocking of a small orifice within the orifice tube. Each orifice tube provides a metered supply of fluid to the two-phase heat exchanging system of one of the docking bays. By selecting appropriately sized orifice tubes, the orifice tubes can each function as an artificially high pressure drop thus minimizing the effect of a phase change on the overall pressure of a particular fluid pathway. This maintains a balanced fluid flow along each of the parallel fluid paths.
If by design, a different amount of fluid needs to flow down each pathway, different orifice tubes having different fixed sized orifices can be selected for each of the fluid pathways. The orifice tubes are very good for use in manufacturing because they include sub-assemblies that are completely enclosed within a larger pipe or tube, where the sub-assemblies include their own filters in some embodiments. The sub-assembly can have a stop or bend to fix the sub-assembly in position within the outer tube.
The fluid input manifold includes the external input line interconnect 8, the server rack input line 10, the one or more dynamic fluid flow regulators 12, the plurality of orifice tubes 14, and the plurality of docking bay input lines 16, shown in part in
A fluid flow regulator is a device that maintains a constant fluid flow rate over a given pressure range. In this manner, the fluid flow regulator functions to restrict fluid flow and therefore is said to have a fluid flow resistance. In the case of a fixed fluid flow regulator, an orifice size in the fluid pathway is fixed. In the case of a dynamic fluid flow regulator, an orifice size in the fluid pathway varies. In some embodiments, the dynamic fluid flow regulator includes a spring-actuated orifice. If the pressure is high, then the spring forces the orifice smaller. If the pressure is low, then the spring forces the orifice larger. Opening and closing the orifice enables a constant amount of fluid to flow through the dynamic fluid flow regulator during ranging pressure conditions. Each dynamic fluid flow regulator is rated to provide a constant output fluid flow rate for a given range of fluid pressures. As applied to the dynamic fluid flow regulators 12 in
Each section input line 24 branches into a plurality of orifice tubes 14. There is one orifice tube 14 for each docking bay 6. Each orifice tube 14 functions as fixed fluid flow regulator having a fixed fluid flow orifice. It is understood that other configuration can be used to implement a fixed fluid flow regulator. For example, a fixed fluid flow regulator can be implemented using a flat disc having a fixed sized orifice.
In some embodiments, the orifice size for all orifice tubes in a given section is the same. In other embodiments, the orifice size of the orifice tubes in different sections can be the same or different. In general, each orifice tube can be independently configured with its own orifice size.
The fluid header 28, the heat exchangers 30, and the fluid header 32 are configured in any manner such that fluid input from the docking bay input line 16 is distributed through the heat exchangers 30 to the fluid header 32 and back through the heat exchangers 30 to the fluid header 28 and output through the docking bay output line 18. In some embodiments, the fluid header 28, the heat exchangers 30, and the fluid header 32 are configured to evenly distribute fluid across the entire area covered by the heat exchangers 30. In some embodiments, the fluid header 28, the heat exchangers 30, and the fluid header 32 are configured such that fluid flows from the fluid header 28 to the fluid header 32 only through select ones of the heat exchangers 30, and fluid flows from the fluid header 32 to the fluid header 28 through other select ones of the heat exchangers 30. In other embodiments, the fluid header 28, the heat exchangers 30, and the fluid header 32 are configured such that fluid flows from the fluid header 28 to the fluid header 32 and from the fluid header 32 to the fluid header 28 in one, some, or all of the heat exchangers 30. In still other embodiments, the fluid header 28, the heat exchangers 30, and the fluid header 32 are configured to selectively provide more fluid to certain areas of select heat exchangers than to other areas so as to selectively cool hot spots in an electronic server mounted in the docking bay. In some embodiments, each heat exchanger is a cold plate. In an exemplary application, a single flexible cold plate is used which can be flexed into thermal contact with the electronics server mounted in the docking bay. Such a flexible cold plate is described in U.S. Pat. No. 8,000,103, which is hereby incorporated in its entirety by reference.
The cooling system is configured to operate as a two-phase cooling system. In such a system, fluid input to the input fluid manifold is in a liquid phase, and the fluid output from the output fluid manifold is in a combination of liquid and gas phase. The fluid remains in the liquid phase until it enters the two-phase heat exchanging system coupled to the docking bays. In an exemplary application, the mass flow rate of the fluid through the entire cooling system is substantially constant. Since fluid in a gas phase has a greater volume than the same fluid in a liquid phase, the output lines in the output fluid manifold are configured with a greater diameter than the input lines within the input fluid manifold. Accordingly, the server rack input line 10 has a smaller diameter than the server rack output line 20. In some embodiments, the docking bay input lines 16 have a smaller diameter than the docking bay output lines 18. Configuring the components in the input path with smaller diameters than the components in the output path functions to alleviate increased pressure due to the phase change of the fluid from liquid to gas.
By using the combination of the dynamic fluid flow regulators and the fixed fluid flow regulators, fluid flow is regulated such that there is substantially equal fluid flow to each docking bay despite the level of heat generated by the individual electronics servers mounted within the docking bays, even when one or more docking bays are unoccupied. In conventional configurations, fluid flow to the unoccupied docking bays would increase and fluid flow to the occupied docking bays would decrease due to the lower pressure drop corresponding to the unoccupied docking bays and the higher pressure drop corresponding to the occupied docking bays.
Using a dynamic fluid regulator enables load balancing of fluid provided to each docking bay. In other words, the fluid flow rate provided to each section is dynamically adjusted to assure the same fluid flow rate is provided to each section under changing conditions. For example, during an initial installation, only a few electronics servers may be loaded into the server rack. As more electronics servers are added to a given section, there is a greater amount of pressure drop as more heat is being generated by the additional electronics servers. If the load is not balanced between sections of the server rack, then more fluid would be diverted away from the section having the higher pressure drop. The dynamic flow regulators assure that a constant fluid flow rate is supplied to each orifice tube in the section. In this manner, the fluid input manifold enables constant fluid flow from low load to high load situations.
The gross fluid flow balancing is happening via the dynamic fluid flow regulators, by branching the input line into sections. The finer balancing is achieved using the orifice tubes. The output from the dynamic fluid flow regulator provides a constant fluid flow and therefore provides an equal opportunity for each docking bay in the section supplied by the dynamic fluid flow regulator to receive the same amount of fluid. Some of the docking bays may be occupied and some may be unoccupied, and some of the electronics servers in the occupied docking bays may be generating more heat than others due to their current level of operation. Such conditions conventionally cause fluid distribution difficulties because due to pressure drops the fluid would flow to those electronics servers generating the least amount of heat. By adding the fixed fluid flow orifice of the orifice tubes, the fluid flow to the docking bay input lines is balanced. The pressure drop is dominated by the orifice tube instead of any heat exchanging system used to cool the electronics server mounted in the docking bay. In an exemplary application, the server rack is able to provide 600 watts of cooling to each docking bay regardless of whether the docking bay is occupied or unoccupied, or the level of operation of an electronics server mounted within the docking bay.
Equal fluid flow distribution to each docking bay is achieved as long as the fluid delivered to each docking bay does not completely evaporate. In such a case, a two-phase condition no longer exists in the heat exchanging system and the increasing pressure leads to reduced fluid flow. As such, it is important to only install electronics servers properly rated to be cooled by the server rack or to a particular docking bay within the server rack, as will be described in greater detail below. For example, a 1 kilowatt electronic server should not be installed into a server rack where each docking bay is rated to cool a 600 watt electronic server. An exemplary server rack is a 20 kilowatt rack, and the four dynamic fluid flow regulators have a fluid flow rate of 0.5 GPM (gallon per minute) flow. The fluid flow rates of the dynamic fluid flow regulators and the orifice tubes determine the cooling capacity of the server rack.
In some embodiments, each dynamic fluid flow regulator has the same rating, for example each flow regulator is a 0.5 GPM flow regulator. In other embodiments, one or more dynamic fluid flow regulators can have a different ratings. Having different types of dynamic fluid flow regulators enables different rated electronics servers to be loaded into the server rack.
Design flexibility is enabled by choosing dynamic fluid flow regulators having desired fluid flow rates, changing the diameter of the fluid line within the orifice tubes, or a combination of the two. Additionally, not all of the dynamic fluid flow regulators need have the same fluid flow rate. Similarly, not all of the orifice tubes need have the same sized orifices. For example, although the four dynamic fluid flow regulators shown and described in regard to
The combination of dynamic fluid flow regulators and orifice tubes provides an optimization of balancing fluid flow, minimizing costs, achieving the desired manufacturability, and reliability with the filtering.
Although the first stage fluid flow regulators, such as the fluid flow regulators 12, are shown and described above as dynamic fluid flow regulators, the cooling system can be alternatively configured such that one or more of these fluid flow regulators are fixed fluid flow regulators.
Although the cooling system is described above as a two-stage configuration having a first stage dynamic fluid flow regulator and a second stage fixed fluid flow regulator, certain applications are also contemplated in which the cooling system has a single stage configuration. In such a single-stage implementation, fluid flow rates, flow rate requirements in the docking bays, and/or the number of docking bays are such that only a single fixed fluid flow regulator is implemented in the fluid pathway to each docking bay. In some embodiments, where there are N docking bays, the server rack input line branches into N parallel input lines, each input line having a fixed fluid flow regulator.
The present application has been described in terms of specific embodiments incorporating details to facilitate the understanding of the principles of construction and operation of the cooling system. Many of the components shown and described in the various figures can be interchanged to achieve the results necessary, and this description should be read to encompass such interchange as well. As such, references herein to specific embodiments and details thereof are not intended to limit the scope of the claims appended hereto. It will be apparent to those skilled in the art that modifications can be made to the embodiments chosen for illustration without departing from the spirit and scope of the application.
This application claims priority of U.S. provisional application Ser. No. 61/406,084, filed Oct. 22, 2010, and entitled “IMPROVED FLOW BALANCING SCHEME FOR 2-PHASE REFRIGERANT COOLED RACK”. This application incorporates U.S. provisional application Ser. No. 61/406,084 in its entirety by reference.
| Number | Date | Country | |
|---|---|---|---|
| 61406084 | Oct 2010 | US |
