Patent Application
20240314973
With the increased performance and power consumption of high performance computing environments (such as data centers), system designers are continually seeking ways to improve the cooling technology of the underlying electronic components that generate heat.
An immersion bath chamber 103 is fluidically coupled to a coolant distribution unit (CDU) 104 that includes a pump 105 and heat exchanger 106. During continued operation of the electronic components, the liquid's temperature will rise as a consequence of the heat it receives from the operating electronics. The pump 105 draws the warmed liquid 102 from the immersion bath chamber 103 to the heat exchanger 106. The heat exchanger 106 transfers heat from the warmed fluid to a secondary liquid within a secondary cooling loop 107 that is fluidically coupled to a cooling tower and/or chilling unit 108. The removal of the heat from the liquid 102 by the heat exchanger 106 reduces the temperature of the liquid which is then returned to the chamber 103 as cooled liquid.
In a high computing environment, such as a data center, the respective CDUs of multiple immersion bath chambers are coupled to the secondary loop 107, and, the cooling tower and/or chilling unit 108 removes the heat generated by the electronics within the multiple immersion bath chambers from the data center.
With the increasing performance and corresponding heat dissipation of the electronic components within the immersion bath 102, engineers and technicians are continually seeking ways to improve the efficiency of the thermal transfer from the circuit boards' respective electronic components to the immersion bath liquid 102.
Ideally, the immersion bath liquid 202 exhibits a high rate of fluid flow through the heat sink fins 212 so that the large amounts of heat generated by the one or more semiconductor chips that are operating within the underlying chip package can be efficiently transferred to the immersion bath 202. Unfortunately, referring to
Firstly, horizontal flow of cooled fluid along the x axis is essentially blocked by the circuit boards 201. Secondly, with the chamber walls, the circuit boards 201 and the circuit boards' respective electronics and packaging introducing large surface areas that the fluid 202 is to flow over, the fluid 202 experiences viscosity forces that resist its flow throughout the chamber 203, generally (the viscosity forces are proportional to the surface areas of the chamber walls, the circuit boards 201 and the circuit boards' respective electronics and packaging).
Thirdly, to the extent attempts have been made to induce high velocity currents that run horizontally along the y axis or along the vertical z axis, such attempts have placed various structures and/or components (e.g., baffles, jets) in peripheral regions 213_1, 213_2 outside the circuit boards 201. Unfortunately, with the currents being directed toward the heat sinks 211 from the periphery 213_1, 213_2, the heat sinks 211 can impose flow impedances that cause the currents to flow around the heat sinks 211 rather than through their fins 212.
Finally, the immersion bath liquid 202 has appreciable density and corresponding mass that causes the fluid 202 to experience downward (−z) gravitational forces that act against upward (buoyant) fluid flow in the +z direction. The lack of upward fluid flow, combined with the aforementioned viscous forces, causes flow stagnations and re-circulations within the chamber 203, which, in turn, result in insufficient fluid flow (upward or otherwise) through the heat sink fins 212.
A solution, referring to
Notably, as compared to previous solutions that only direct or increase fluid flow generally over a multitude of components, e.g., from the periphery 213 of an electronic circuit board (as described above with respect to
For ease of illustration, again, only a single heat sink 311 is depicted on the circuit board 301 of
Examples of such pluggable units include a blade server, a CPU unit, an accelerator unit, a memory unit, storage unit, etc. Here, the function of the pluggable electronic unit largely corresponds to the functions that the assembly's printed circuit board 401 are designed to perform or support (e.g., a computing system in the case of a blade server, one or more multi-core processor chips in the case of a CPU unit, one or more accelerators (e.g., one or more neural network chips, artificial intelligence machine learning chips, artificial intelligence inference engine chips, graphics processing unit (GPU) chips, etc.) in the case of an accelerator unit, multiple memory chips (e.g., multiple dual in-line memory modules (DIMMs)) in the case of a memory unit, multiple storage devices (e.g., multiple solid state drives (SSDs)) in the case of a storage unit, etc.).
As observed in
The frame 431 mechanically supports the printed board 401 whereas the cover 432 mechanically protects the printed circuit board 401 and its electronic components from mechanical shocks/blows that can be imparted to the assembly 400, e.g., during insertion/removal of the assembly to/from the immersion liquid. Notably, the frame 431 and cover 432 have various perforations or other openings that allow fluid to enter the space where the printed circuit board 401 resides so that the electronic components can be cooled by fluidic flow through the assembly 400.
As observed in the particular embodiment of
In still other embodiments, as observed in the side view of
In still other embodiments, the flow enhancement structure is formed from a combination of elements that are formed in the cover 432 and individual elements that are mounted to any/all of the cover 432, printed circuit board 401 and frame 431.
In various embodiments, the immersion bath includes a framework having multiple slots that respective pluggable electronic units can plug into. In the particular embodiment of
During installation of a pluggable unit, the pluggable unit is entered into the slot from the top of the immersion bath and pressed downward in the −z direction along the guide rails 546_1, 546_2 until it is plugged into the backplane 541. As part of the installation into the slot, the first fluidic connector 542 is connected to the second fluidic connector 548 thereby fluidically coupling the channel 545 that emanates from the flow enhancement structure to CDU return line 543.
During operation of the particular embodiment of
The fluid continues to flow 544_1, 544_2 upward above the duct 547. Due to suction from the CDU pump and/or gravity, the immersion coolant flows 544_1, 544_2 are drawn into the intake duct opening 547 of the flow enhancement structure 521. The fluid 544_1, 544_2 then flows through the flow enhancement structure 521 (ideally, with a high fluidic velocity).
Here, the flow enhancement structure 521 is designed as a kind of housing that encompasses the fins of the heat sink 511 of a high performance chip package that resides directly beneath the flow enhancement structure 521. With the immersion coolant 544_1, 544_2 flowing at a high velocity through the flow enhancement structure 521, and with the flow enhancement structure 521 confining the currents 544_1, 544_1 to flow through the space between the fins of the heat sink, heat is transferred from the heat sink fins to the immersion bath flows 544_1, 544_1 with high efficiency (low thermal resistance).
Notably, because the high performance semiconductor chip(s) within the chip package beneath the heat sink 511 generate most of the pluggable unit's heat, as the currents 544_1, 544_2 flow around the flow enhancement structure 521 immediately after injection into the bath, they do not capture significant heat because they flow through/across electronics that generate significantly less heat than the chip(s) in the high performance chip package beneath the heat sink 511. As such, the temperature of the fluid as it enters the intake duct 547 should be relatively cool (the temperature is only slightly warmed than the temperature of the fluid that enters the immersion bath from the CDU).
As discussed above, higher thermal transfer efficiencies from the heat sink fins to the fluid within the flow enhancement structure 521 is achieved with increasing fluid flow velocity through the structure 521. In order to increase the flow rate through the enhancement structure 521, the draw/suction from the CDU pump can be increased.
In other approaches, however, e.g., to avoid excessively powerful/expensive CDU pump equipment, the flow rate through the flow enhancement structure 521 is increased by establishing a sufficiently large height difference 553 between the upper surface 552 of the coolant within the immersion bath chamber 503 (upper liquid free surface 552) and an opening 551 in the CDU return line 543 (lower liquid free surface 551).
Specifically, as observed in
Importantly, the opening 551 in the return line 543 and an opening in the plenum 549 physically connects the lower liquid free surface 551 to the same ambient as the upper liquid free surface 552. With this arrangement, gravity will cause the rate of fluid flow through the enhancement structure 521 to increase as the height difference 553 between the upper and lower liquid free surfaces 552, 551 increases. As such, for example, extremely high flow rates through the flow enhancement structure 521 can be achieved (e.g., without expensive/powerful CDU pump equipment) by setting the opening 551 in the CDU return line 543 and the plenum 549 sufficiently below the immersion bath chamber 503.
It is pertinent to point out that the specific flow patterns 544_1, 544_2 and intake duct arrangement 547 of
Note that electro-mechanical interfaces can also exist at the backplane 541, and/or, can be coupled to the pluggable unit through the upper surface of the immersion bath 551. In various embodiments (as suggested by
Consistent with the discussion of
In various embodiments, one or more traditional CDU return lines are also coupled to the chamber 503 so that less than all of the CDU return flows through the chamber's flow enhancement structure(s).
Although the embodiments of
Certain additional challenges and/or optimization opportunities can arise when attempting to implement an immersion bath chamber having, e.g., a pluggable unit that includes a flow enhancement structure for a high performance semiconductor chip of the pluggable unit as described above, and/or, an immersion bath chamber that is designed to induce gravitational fluid flow as described above. Certain challenges and embodiments of their respective solutions are described immediately below.
The heat removal capacity of a heat sink within a flow enhancement structure as described at length above improves within increasing fluid flow velocity through the flow enhancement structure.
For instance, a higher fluid flow rate will be observed through the heat sink, and/or, the heat sink can be replaced with a heat sink having increased fin density (and therefore greater heat removal capacity). In the case of the former, the higher pressure head results in increased fluid velocity through a same fluidic impedance. In the case of the later, the higher pressure head results in an ability to drive an increased fluidic impedance. For ease of discussion, the remainder of the discussion assumes the heat sink remains unchanged such that higher fluid flow through the heat sink is realized.
As observed in the particular example of
The pressurized air 661 above the liquid coolant 602 within the chamber 603 exerts pressure against the liquid coolant 602, which, in turn, drives additional fluid flow through the flow enhancement structure 621 and CDU return channel 643. The increased fluid flow translates into increased heat removal capacity of the immersed heat sink 611. Again, the increased fluid flow and heat removal capacity is in addition to whatever fluid flow and heat removal capacity is achieved through gravity as a consequence of height 653.
Notably, the pressurization/pumping 662 of the air 661 need not be continuous during operation of the electronics 601 within the chamber 603. For example, during initial installation of the electronic circuit board(s) 601 within the immersion bath 602 and chamber 603 the lid 663 can be secured on the chamber 603 to form a sealed system. Air can then be injected/pumped 662 into the air space 661 within the chamber 603, e.g., by way of a valve that is integrated on the lid 663.
The pumping of the air into the space 661 increases the pressure of the air space 661 which, as described above, will provide additional fluid flow “pressure head” through the flow enhancement structure 621. The valve that is integrated into the lid 663 is then closed and the injection/pumping of air into the space 661 stops. With the chamber 603 being sealed, the pressurized air condition within space 661 will remain approximately constant over extended periods of time.
Here, the increased fluid flow out of the chamber 603 (through the flow enhancement structure 621 and CDU return channel 643) that results from the pressurized space 661, and that left alone will act to reduce the pressure within the space 661, can be compensated for by increasing the rate at which fluid 644_1, 644_2 is pumped into the chamber 603.
Said another way, any lowering of level 651 that can/could result from the increased pressure within space 661 can be compensated for (or otherwise mitigated) by increasing the rate of fluid flow rate 644_1, 644_2 into the chamber 603. In this state, the chamber is essentially a closed system that is able to maintain the enhanced fluid flow that results from the pressurized air space 661 for an extended period of time in which the electronics 601 continually operate.
Notably, the increased pressure from input liquid flows 644_1, 644_1 can be effected with an input valve 664 that is located at a fluid input 644_0 to the chamber 403. By increasing the opening of the input valve 664, increased fluid flow rate will be observed for input flows 644_1, 644_2. The input valve 664 can be precisely opened to a wider opening to achieve a desired increase in fluid flow rate through the flow enhancement structure 621 and CDU return channel 643 for embodiments that rely on increased air pressure (
Alternatively or in combination, an output valve 665 can be placed along the chamber's fluid return line to achieve a similar effect. Namely, the output valve 665 can be precisely narrowed to a narrower opening to increase the fluidic impedance out of the chamber which, in turn, increases the pressure of the liquid coolant 602 within the chamber 603 (which in turn increases the fluid flow through the flow enhancement structure 621).
Note that combined embodiments can exist where the pressure of the liquid coolant 602 is increased to effect higher flow rate through the flow enhancement structure 621 with both increased air pressure (
Although embodiments above have stressed that a “lid” is the component that seals the chamber, in other embodiments, another type of chamber cover (e.g., a portion of a chamber sidewall) can be used to seal the chamber to enhance the pressure head of the coolant.
Another challenge is ensuring that a variety of different pluggable electronic units can easily “plug into” the backplane of any particular immersion chamber. Here, recall from the discussion of
Here, briefly referring back to
With respect to different pluggable electronic units having different backplane interface 471, 571 designs, as just one example, a first blade server may have two fluid flow exit ports of a first connector type that are spaced a first distance apart on the first server's chassis backplane interface, whereas, a second type of blade server may have four fluid flow exit ports of a second connector type that are spaced a second (different than the first) distance apart on the second server's chassis.
Notably, the different chassis backplane interface designs can cause complications when an operator desires to install a set of pluggable units having different backplane interface designs into any single immersion chamber.
A solution, as observed in
Here, the immersion chamber 703 has mechanical mounting fixtures (e.g., alignment posts, alignment holes, threaded holes, threaded studs, etc.) that receive corresponding mechanical mounting fixtures that are integrated on the “downward” face of the transfer plate 772_1 (which faces the bottom of the immersion chamber 703 as is not observable in
Importantly, the “upward” face of the transfer plate 772_1 (which is observed in
In the particular example of
Thus, when a pluggable unit having a particular backplane interface design is to be plugged into the immersion bath chamber 703, an operator first mounts the transfer plate for the pluggable unit's particular chassis backplane interface design to the immersion chamber 703. The selected transfer plate 772_1, as described just above, has integrated fluid flow connectors whose type and location on the transfer plate are designed to align with and couple to the pluggable unit's fluid flow exit flow port(s).
The operator of the immersion chamber can therefore possess a collection of different transfer plates to support pluggability of a corresponding collection of different pluggable units having different respective backplane interface designs. For example, an operator may possess a first transfer plate 772_1, as observed in
In this case, if the operator chooses to replace twelve single exit port pluggable units that are currently installed in an immersion chamber with twelve dual exit port pluggable units, the operator need only: 1) remove the twelve single port pluggable units that from the immersion chamber; 2) remove the transfer plate 772_1 for the twelve single port pluggable units from the immersion chamber; 3) install the transfer plate 772_2 for twelve dual exit port pluggable units into the immersion chamber; and, 4) plug the twelve dual exit port pluggable units into the newly installed transfer plate 772_2 within the immersion chamber.
Notably, as indicated just above, often times multiple pluggable units are to be simultaneously installed into the immersion chamber. As indicated by the transfer plate embodiments 772_1 and 772_2 of
In other embodiments, a single transfer plate that supports multiple pluggable units having different respective backplane interfaces has correspondingly different arrangements of fluid flow connectors.
An example is depicted in
Thus, the transfer plate 772_3 of
In another approach, referring to
Here, a base transfer plate 872_a is permanently or quasi permanently installed in the immersion chamber (e.g., by way of mechanical mounting fixtures as discussed above). Notably, the base transfer plate 872_a has multiple slots (for case of drawing only one 879 of the slots is labeled with a reference number). Each slot is an opening in the base transfer plate 872_a and corresponds to a location where one or more pluggable units may be installed into the immersion chamber.
An adaptor transfer plate 872_b is mounted to the base transfer plate 872_a where the adaptor transfer plate 872_b has a particular arrangement of one or more fluid connectors that are designed to align with and couple to a particular one or more electronic pluggable units having a specific backplane interface design. In the particular example of
Notably, another adaptor transfer plate having a completely different arrangement of fluid connector(s) to align and couple with a pluggable unit having a different backplane interface than the particular interface that adaptor plate 872_b is designed to mate with can be mounted to the base transfer plate 872_b at any of the slots other than slot 879. As such, in this manner, each pluggable unit can have its own dedicated adaptor transfer plate and any arrangement of different pluggable units can be simultaneously plugged into the immersion chamber.
In various embodiments a single adaptor transfer plate can be designed to couple with more than one pluggable unit and the corresponding slot in the base transfer plate can be, e.g., wider to accommodate the more than one pluggable unit.
As discussed above, the slots corresponds to openings in the base transfer plate 872_b that permit the fluid flow that exits a pluggable unit to flow “beneath” the base transfer plate 872_b (as observed in
In any/all of the embodiments discussed above with respect to
That is, referring back to
Fluid that is warmed by the operating electronics enters the lower region 774 from the upper region 773 by passing through the fluid exit ports of the respective electronic pluggable units and then through the fluid connectors on the transfer plate 772_1 into the lower chamber 774. Notably, cooled fluid from the CDU can, e.g., enter the upper region 773 by way of at least one input port 744 that is located on the side of the chamber 703 at the upper region 773.
In various embodiments, in order to form the barrier between the upper 773 and lower 774 regions, the transfer plate 772_1 is designed to be mounted to the chamber 703 such that the two regions 773, 774 are isolated/scaled from one another. So doing forces fluid that flows from the upper region 773 to the lower region 774 to only flow through the connectors in the transfer plate solution, which, in turn, increases the fluid flow through the flow enhancement structures of the pluggable units that are installed in the immersion chamber.
In the case of a base transfer plate 872_a having multiple slots as discussed above with respect to
The cooled fluid flows 1044_1, 1044_2 enter the region of the chamber above the transfer plate by way of holes that are formed in the transfer plate above the cooled fluid input channels that reside beneath the transfer plate. In this case the immersion chamber can operate, e.g., as described above with respect to
In further embodiments, e.g., in order to support additional configuration flexibility regarding the combinations of pluggable units having different backplane interface designs that can be simultaneously installed in the immersion chamber, the backplane interface of the pluggable unit is replaced with an adapter backplane interface that changes the nominal backplane interface of the pluggable unit.
Note that, for any of the embodiments described above with respect to
Note that embodiments above have emphasized a transfer plate that is substantially rectangular and planar. Notably, other embodiments may include other shapes (e.g., oval, circular) and/or that are not planar (e.g., have curved surfaces). As such, the teachings of Section 2 can be extended more generally to a transfer member rather than a transfer plate, specifically.
3. Immersion Bath Chamber with Overflow Chamber
In the approach of
In the approach of
Notably, in this particular approach, the fluid that exits the system from the main chamber CDU return line 1243 can be fluid that is heated by high temperature/performance chips that are cooled by fluid flow through a flow enhancement structure 1221. By contrast, the fluid that exits the system from the overflow CDU return line 1254 can be fluid that is merely warmed by lower temperature/performance chips that are cooled via convection from coolant current flows within the main chamber 1203. Thus, the main chamber CDU return line 1243 can be coupled to a higher performance CDU with a high heat removal capacity, whereas, the overflow chamber CDU return line 1254 can be coupled to lower performance (e.g., cheaper) CDU with a lower heat removal capacity.
Alternatively, one or both the CDU return lines 1254, 1243 can be coupled to a plenum to induce a gravitationally induced flow out of the particular chamber that the CDU return line is coupled to. Likewise, in the approach of
In the system of
By contrast, in the system of
These and other features are described in more detail further below.
Inset 1380 shows sets of holes 1361, 1362, 1363 in a sidewall of the main chamber to allow fluid to flow from the main chamber into the overflow chamber. Notably, the density of the holes increases moving up the sidewall. That is: 1) at a first lower level there is a lowest density of holes 1361; 2) at a second middle level there is a medium density of holes 1362; and, 3) at a third highest level there is a highest density of holes 1363. With this arrangement, the rate of overflow into the overflow chamber will increase as the level of coolant in the main chamber rises within the main chamber.
Specifically: 1) when the coolant level in the main chamber is below the level of the lower set of holes 1361 there is no overflow into the overflow chamber; 2) when the coolant level in the main chamber rises above the level of the lower set of holes 1361 but remains beneath the level of the second set of holes 1362 there is lesser overflow into the overflow chamber; 3) when the coolant level in the main chamber rises above the level of the second set of holes 1362 but remains beneath the level of the third set of holes 1363 there is medium overflow into the overflow chamber; and, 4) when the coolant level in the main chamber rises above the level of the third set of holes 1363 there is high overflow into the overflow chamber.
Such hole designs can affect the flow rate into the overflow chamber resulting in the level of fluid within the overflow chamber having an effect on the fluid level within the overflow chamber (along with flow rate into the main chamber and the flow rate along the overflow CDU return line).
The particular embodiment of
Referring now to the particular approach of
At a first extreme, the valve 1264 is fully closed and the CDU return line 1243 is coupled to a CDU pump. In this case, the draw of fluid flow from the main chamber is determined by the CDU pump speed and there is no fluid flow from the overflow chamber 1213. At the other extreme, the valve 1264 is fully open and the CDU pump is turned off. In this case, the open valve 1264 causes the overflow chamber 1213 to act akin to a plenum and the draw of fluid flow 1245 from the main chamber is determined by gravitational forces as described above with respect to
These two extremes can be used to modulate the energy consumed by the CDU as a function of the workload being performed by the electronics within the immersion bath chamber.
Specifically, if the workload of the high performance semiconductor chips within the immersion bath is high, the system can be configured into the first extreme in which the valve 1264 is closed and the fluid draw 1245 from the main chamber is determined by the CDU pump speed. In this case, the rate at which fluid is drawn 1245 from the main chamber can exceed by many factors whatever rate could otherwise be achieved with gravity induced draw if the system were configured in the second extreme. Said another way, when configured in the first extreme, the CDU pump consumes (potentially high amounts of) energy to induce sufficiently high fluid flow draw 1245 from the main chamber to adequately cool the electronics when the electronics are under heavy workload.
By contrast, if the workload of the high performance semiconductor chips is minimal, the system can be configured in the second extreme in which the valve 1264 is fully opened and the CDU pump is turned off. In this case, fluid flow 1245 from the main chamber is induced by gravity and the CDU consumes little/no energy. Thus, when configured in the second extreme, the CDU pump consumes little/no energy but the gravity induced fluid flow draw from the main chamber is sufficient to cool the high performance semiconductor chips when they are operating, e.g., at light workload.
The system can also be placed in any of a wide range of operational configurations between these two extremes. For example, as the workload of the high performance semiconductor chips decreases, CDU pump speed can be lowered and/or the valve opening can be widened. Likewise, e.g., as the workload of the high performance semiconductor chips increases, CDU pump speed can be raised and/or the valve opening can be narrowed.
Here, the total fluid draw from the system through CDU return line 1243 can be caused by a combination of CDU pump draw and gravity where the CDU pump speed determines the pump draw component of the total draw and the setting of the valve 1264 opening determines the gravity induced component of the total draw.
Because of the CDU pump energy efficiencies can be gained with gravitational induced flow, a number of embodiments choose to emphasize gravity induced flow when feasible. In these embodiments, CDU pump speed is reduced and/or is greatly reduced. Nevertheless, CDU pump speed can remain non-zero to, e.g., effect a “baseline” fluid flow draw from the system.
In any/all of these situations, the fluid flow draw 1472 from the main chamber will be a function of the height difference ΔH 1493 between the fluid level 1492 in the main chamber and the fluid level 1491 in the overflow chamber.
Specifically, when ΔH 1493 is small (valve opening is narrow), there is a small fluid flow 1473 from the overflow chamber into the CDU return line which results in larger fluid flow draw 1472 from the main chamber. By contrast, when ΔH is large (valve opening is large), there is a large fluid flow 1473 from the overflow chamber into the main CDU return line which results in smaller fluid flow draw 1472 from the main chamber.
In this manner, with constant CDU pump speed, the rate of the fluid flow draw 1472 from the main chamber can be modulated/varied by modulating/varying the size of the valve opening. Specifically, if chip workload increases the valve opening can be narrowed to increase the fluid flow draw 1472 from the main chamber, whereas, if chip workload decreases the valve opening can be widened to reduce the fluid flow draw 1472 from the main chamber.
As observed in
Moreover, minimum and/or maximum fluid levels can also be established for the fluid level 1491 within the overflow chamber. Here, the minimum and/or maximum levels can be adjusted for any desired ΔH. Additionally, in various embodiments, there is a “warning line” 1556 that the fluid level within the overflow chamber is not supposed to fall beneath irrespective of the total fluid draw from the system. In various embodiments the warning line 1556 is set above the valve so that the effectiveness of the valve is ensured.
In still other embodiments, even though gravity induced flow is being utilized (the valve is open), the CDU pump can active and the CDU pump speed can be modulated to adjust, e.g., the baseline rate of fluid flow draw from the main chamber in response to, e.g., changing conditions within the chamber and/or the workload of the high performance semiconductor chips.
If the fluid level in the overflow chamber falls below 1603 the “warning line” 1556 the CDU pump speed is significantly reduced 1604 to reduce the rate at which fluid is drawn from the system, which, in turn, reduces the rate at which liquid is drawn from the overflow chamber. With reduced fluid flow draw from the overflow chamber the fluid level in the overflow chamber should rise.
By contrast, if the fluid level is where it should be (above the minimum level setting and beneath the maximum level setting), the pump speed is not changed to ideally keep the existing fluid level within the overflow chamber 1603, 1605, 1606.
If the fluid level within the overflow chamber is above the maximum level but the fluid level is falling within the overflow chamber, the pump speed is kept constant 1607, 1608, 1612 (under the current conditions, the fluid level within the chamber is expected to eventually fall below the maximum level).
If the fluid level within the overflow chamber is above the maximum level but is not falling, the CDU pump speed is increased to increase total fluid draw from the system, which, in turn, increases the fluid flow rate from the overflow chamber 1607, 1608, 1613. The increased fluid flow rate from the chamber should lower the fluid level within the overflow chamber.
If the fluid level within the overflow chamber is beneath its minimum level, the control system keeps the pump speed constant if the fluid level is rising 1609, 1610. Otherwise the pump speed is tweaked downward 1609, 1611.
Specifically, if both temperatures are within their assigned ranges the CDU pump speed is kept constant 1628, 1635, 1638. If the temperature of the fluid being drawn from the main chamber is within its assigned range but the temperature of the overflow fluid is above its maximum setting the CDU pump speed is increased 1628, 1635, 1636, 1637 (which increases the rate at which heat is removed from the system which should decrease the temperature of the fluid in the main chamber).
If the temperature of the fluid being drawn from the chamber is within its assigned range but the temperature of the overflow fluid is beneath its minimum setting the CDU pump speed is kept constant 1628, 1635, 1636, 1638. Here, e.g., with a flow enhancement structure in place for the high performance semiconductor chips, the temperature of the overflow liquid (which is measuring the temperature of the fluid that is cooling the chips other than the high performance chips) is less important. As such, rather than lower the CDU pump speed and risk raising the temperature of the fluid that is cooling the high performance semiconductor chips, the non-high performance chips are allowed to be cooled at a temperature that is cooler than necessary.
If the temperature of the fluid being drawn from the main chamber is above its maximum setting the pump speed is increased to increase the overall heat removal capacity of the system 1628, 1629, 1632.
If the temperature of the fluid being drawn from the main chamber is beneath its minimum setting and the temperature of the overflow fluid is within its assigned range, the CDU pump speed remains constant 1628, 1629, 1630, 1631 (with the high performance chips being more than adequately cooled, the system maintains the overflow temperature within its assigned min/max levels).
By contrast, if the temperature of the fluid being drawn from the main chamber is beneath its minimum setting and the temperature of the overflow fluid is not within its assigned range 1628, 1629, 1630, 1633, the setting the pump speed is adjusted 1632, 1634 in attempt to bring the overflow fluid within its assigned range (with the high performance chips being more than adequately cooled, the system attempts to bring the overflow temperature within its assigned min/max levels).
The CDU pump speed is then adjusted based on readings of the rate of the overflow fluid's flow (“overflow_flowrate”) and the rate of the fluid flow draw from the main chamber (“HS_flowrate”). Both monitored flows are assigned respective flow rate target min/max settings (ranges). Here, flow rate monitors 1552, 1554 are placed in the system to measure the two flow rates.
As observed in
If the flow rate from the main chamber is within its target range but the overflow rate is above its maximum target, the valve opening is narrowed 1648, 1652, 1653, 1656. By contrast, if the flow rate from the main chamber is within target but the overflow rate is beneath its minimum target, the valve opening is widened 1648, 1652, 1653, 1654. The CDU pump speed is kept constant with either of these scenarios. Here, with the flow rate that is drawn from the main chamber (which is the flow that cools the higher performance chips) being within target, CDU pump is not tampered with to avoid not adequately cooling the high performance chips.
By contrast, if the flow rate being drawn from the main chamber is beneath its minimum target setting the CDU pump speed is increased 1648, 1649, 1651 whereas if the flow rate being drawn from the main chamber is above its maximum target setting the CDU pump speed is decreased 1648, 1649, 1650. In both scenarios the valve opening is kept constant. Here, again, emphasis is placed on ensuring the high performance chips are adequately cooled. Specifically, main chamber fluid flow draw (which determines heat removal capacity from the high performance chips) is controlled with adjustments made in CDU pump speed directly from the main chamber flow readings.
In any of the above described embodiments of Section 3 a control system can be communicatively coupled to any/all of the sensors and monitors 1551, 1552, 1553, 1554, 1555, the setting mechanism for the valve (e.g., an electro-mechanical servo motor) and/or the setting mechanism for the pump speed (e.g., an input voltage and/or current that is applied to the pump motor). The control system can include logic circuitry and/or a processor that executes software to implement any of the control operations described just above including adjusting a valve setting or a pump speed in view of operational workload of electronics within the main chamber, and/or, an observed temperature of the main chamber's fluid and/or the overflow chamber's fluid, and/or, an observed flow rate of the main chamber's fluid and/or the overflow chamber's fluid.
Note also that the sidewall holes of inset 1380 of
Note that the immersion bath chamber improvements described above in Sections 1 and 3 can (but need not) include a transfer plate as described above in Section 2. Moreover, the immersion bath chamber improvements described above in Section 3 can include a sealed/pressurized main chamber as described above in Section 1. An immersion bath chamber having any of the improvements as described above in Sections 1, 2 and 3 can receive one or more pluggable units having at least one flow enhancement structure as described above, e.g., in reference to
Networked based computer services, such as those provided by cloud services and/or large enterprise data centers, commonly execute application software programs for remote clients. Here, the application software programs typically execute a specific (e.g., “business”) end-function (e.g., customer servicing, purchasing, supply-chain management, email, etc.).
Remote clients invoke/use these applications through temporary network sessions/connections that are established by the data center between the clients and the applications. A recent trend is to strip down the functionality of at least some of the applications into more finer grained, atomic functions (“micro-services”) that are called by client programs as needed. Micro-services typically strive to charge the client/customers based on their actual usage (function call invocations) of the micro-service application.
In order to support the network sessions and/or the applications' functionality, however, certain underlying computationally intensive and/or trafficking intensive functions (“infrastructure” functions) are performed.
Examples of infrastructure functions include encryption/decryption for secure network connections, compression/decompression for smaller footprint data storage and/or network communications, virtual networking between clients and applications and/or between applications, packet processing, ingress/egress queuing of the networking traffic between clients and applications and/or between applications, ingress/egress queueing of the command/response traffic between the applications and mass storage devices, error checking (including checksum calculations to ensure data integrity), distributed computing remote memory access functions, etc.
Traditionally, these infrastructure functions have been performed by the CPU units “beneath” their end-function applications. However, the intensity of the infrastructure functions has begun to affect the ability of the CPUs to perform their end-function applications in a timely manner relative to the expectations of the clients, and/or, perform their end-functions in a power efficient manner relative to the expectations of data center operators. Moreover, the CPUs, which are typically complex instruction set (CISC) processors, are better utilized executing the processes of a wide variety of different application software programs than the more mundane and/or more focused infrastructure processes.
As such, as observed in
As observed in
Notably, each pool 1701, 1702, 1703 has an IPU 1707_1, 1707_2, 1707_3 on its front end or network side. Here, each IPU 1707 performs pre-configured infrastructure functions on the inbound (request) packets it receives from the network 1704 before delivering the requests to its respective pool's end function (e.g., executing software in the case of the CPU pool 1701, memory in the case of memory pool 1702 and storage in the case of mass storage pool 1703). As the end functions send certain communications into the network 1704, the IPU 1707 performs pre-configured infrastructure functions on the outbound communications before transmitting them into the network 1704.
Depending on implementation, one or more CPU pools 1701, memory pools 1702, mass storage pools 1703 and network 1704 can exist within a single chassis, e.g., as a traditional computing system (e.g., server computer). In a disaggregated computing system implementation, one or more CPU pools 301, memory pools 1702, and mass storage pools 1703 are, e.g., separate pluggable electronic units (e.g., pluggable CPU units, pluggable memory units (M), pluggable mass storage units (S)). Although not depicted in
Notably, a traditional computing system and/or any of the above mentioned pluggable units can be mechanically configured to be immersed in an immersion bath within an immersion chamber that includes any of the teachings described above Sections 1, 2 and 3.
In various embodiments, the software platform on which the applications 1705 are executed include a virtual machine monitor (VMM), or hypervisor, that instantiates multiple virtual machines (VMs). Operating system (OS) instances respectively execute on the VMs and the applications execute on the OS instances. Alternatively or combined, container engines (e.g., Kubernetes container engines) respectively execute on the OS instances. The container engines provide virtualized OS instances and containers respectively execute on the virtualized OS instances. The containers provide isolated execution environment for a suite of applications which can include, applications for micro-services.
Embodiments of the invention may include various processes as set forth above. The processes may be embodied in program code (e.g., machine-executable instructions). The program code, when processed, causes a general-purpose or special-purpose processor to perform the program code's processes. Alternatively, these processes may be performed by specific/custom hardware components that contain hard wired interconnected logic circuitry (e.g., application specific integrated circuit (ASIC) logic circuitry) or programmable logic circuitry (e.g., field programmable gate array (FPGA) logic circuitry, programmable logic device (PLD) logic circuitry) for performing the processes, or by any combination of program code and logic circuitry.
Elements of the present invention may also be provided as a machine-readable medium for storing the program code. The machine-readable medium can include, but is not limited to, floppy diskettes, optical disks, CD-ROMs, and magneto-optical disks, FLASH memory, ROMs, RAMS, EPROMs, EEPROMs, magnetic or optical cards or other type of media/machine-readable medium suitable for storing electronic instructions.
In the foregoing specification, the invention has been described with reference to specific exemplary embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention as set forth in the appended claims. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.
| Number | Date | Country | Kind |
|---|---|---|---|
| PCT/CN2024/088760 | Apr 2024 | WO | international |
The present application claims the benefit of priority to Patent Cooperation Treaty (PCT) Application No. PCT/CN2024/088760, filed Apr. 19, 2024, the entire content of which is incorporated herein by reference.
