System and method for cooling computing devices using a primary circuit dielectric cooling fluid

Information

  • Patent Grant
  • 12089368
  • Patent Number
    12,089,368
  • Date Filed
    Monday, September 11, 2023
    a year ago
  • Date Issued
    Tuesday, September 10, 2024
    6 months ago
Abstract
An immersion cooling system is configured for uniform fluid distribution across computing devices and includes a tank defining an open interior volume, a distribution pipe having a plurality of ports, a distribution plate having a pattern of holes is positioned over the distribution pipe. A siphon wall divides the open interior volume into a first chamber and a second chamber, and the distribution plate and the distribution pipe are in the first chamber. The dielectric cooling fluid enters the first chamber of the tank through the plurality of ports in the distribution pipe, and flows through the pattern of holes in the distribution plate to contact the at least one computing device. The heat dissipates from the at least one computing device into the dielectric cooling fluid, which flows through the siphon wall into the second chamber via a transfer port that is located below a dielectric cooling fluid surface.
Description
BACKGROUND

Data centers may be used by organizations to assemble, process, store, and disseminate large amounts of data. Many enterprises rely heavily on the applications, services and data contained within a data center, making it a critical asset for everyday operations.


A data center facility may include, for example, systems for storing, sharing, accessing, and processing data, physical infrastructure for supporting data processing and data communications, and utilities such as cooling, electricity, and network security access. Conventional data center challenges may include, for example, cooling ever-increasing heat loads; the cost of power, sustainability, and the ability to scale quickly and easily. As high-density technology continues to increase in usage, the amount of heat that is generated continues to increase. The increased heat results in pushing the computing hardware to its limit. This trend is expected to continue into the future.


Data centers typically have a significant carbon footprint. As the need for these data centers continues to surge, an increase in power usage and greenhouse gas emissions is expected to surge with the demand for data centers. Greenhouse gas emissions can contribute to extreme weather and put the health of our planet at risk. There is growing demand from both the public and industry to reduce the environmental impact of all businesses. Thus, sustainability has moved into the corporate domain and, due to regulations and the cost implications of reducing a carbon footprint, mitigation of greenhouse gas emissions and generated heat may be critical. With these economic and political pressures driving data centers to seek alternative approaches to conventional facility design, owners and operators are turning to liquid immersion cooling to reduce power use and drive sustainability efforts.


As data crypto mining evolves, the computational power of the servers (e.g., miners) continues to be pushed, requiring greater heat dissipation capability. With these pressures driving data centers to seek alternative approaches to conventional facility design. Immersion cooling is a popular and proven solution for heat dissipation to reduce power use and drive sustainability efforts. Specifically, the dielectric fluid, in which computing devices such as servers may be immersed, is a far more efficient thermal dissipator than air. In addition, immersion cooling does not require fans which means more power may be directed to the operation of the computing device (i.e., server). Thus, in terms of crypto-mining, more power may be directed to higher hash rates.


A typical crypto-mining installation may include several computing device servers (also referred to as miners) that are mounted in a single immersion cooling tank. While the immersion of the computing devices in the immersion cooling tank may promote cooling, a non-uniform flow distribution of the dielectric cooling fluid may cause inefficient heat dissipation. As a result, overheating may occur in some of the computing devices that are position away from the peak flow distribution. Computing devices that are located further away from the inlet are prone to receive a lower flow rate of the dielectric cooling fluid across them. As a result, these computing devices may be operating at increased temperatures and may possess lower than expected hash rates. However, conventional solutions such as variable duct sizes and tapered geometries can cause significant large-scale manufacturing complexities, compromising the fidelity of the proposed system. Therefore, a low-cost and low-complexity solution is required that ensures uniform flow distribution across a multi-server installation.


SUMMARY

Various aspects include devices, systems, and methods for providing uniform fluid distribution of dielectric cooling fluid to a plurality of computing devices in an immersion cooling system. Some aspects may include a tank defining an open interior volume, a distribution pipe positioned on the floor and running across the length of the tank, a distribution plate positioned over the distribution pipe, and at least one computing device mounted on the distribution plate. In some aspects, the dielectric cooling fluid may enter the open interior volume of the tank through the plurality of ports in the distribution pipe. In some aspects, the dielectric cooling fluid may flow through multiple repeated sets of the pattern of holes in the distribution plate to contact the at least one computing device. In some aspects, heat dissipates from the at least one computing device into the dielectric cooling fluid.


In some aspects, a floor of the tank may include a plurality of alternating elevated platforms and alleys. In some aspects, the distribution pipe may include a plurality of ports that each align with one of the alleys. In some aspects, the distribution plate may have a pattern of holes in multiple repeated sets. In some aspects, each of the at least one computing device may be aligned with one set of the pattern of holes.


In some aspects, the distribution pipe may be connected on one end to an inlet of the tank, and is capped on an opposite end.


Some aspects may further include a rack that may house twenty-four computing devices in the at least one tank. In some aspects, the plurality of ports may include thirteen equally spaced ports along each of two opposite sides of the distribution pipe. In some aspects, the pattern of holes in the multiple repeated sets may be twenty holes repeated twenty-four times over the distribution plate.


In some aspects, each of the plurality of ports in the distribution pipe may have an internal diameter of 10 mm, and each hole in the pattern of holes in the distribution plate may have an internal diameter of 5 mm.


In some aspects, the distribution plate may be positioned such that there is a 6.5 mm clearance gap between the top of the distribution pipe and the bottom of the distribution plate.


In some aspects, the dielectric cooling fluid may be oil. Some aspects may further include an oil-to-water heat exchanger.


In some aspects, the at least one computing device may include twenty-four servers housed in each of at least one rack within the tank.


In some aspect, the floor of the tank may include twelve elevated platforms and thirteen alleys.


Various aspects are disclosed herein that may include devices, systems, and methods for providing uniform fluid distribution of dielectric cooling fluid to a plurality of computing devices in an immersion cooling system. Some aspects may include a tank defining an open interior volume, a distribution pipe positioned on the lower portion of the tank and the distribution pipe has a plurality of ports, a distribution plate positioned over the distribution pipe, the distribution plate has a pattern of holes, a siphon wall mounted inside of the tank and extends from the floor, the siphon wall may divide the open interior volume to define a first chamber and a second chamber, and the distribution plate and the distribution pipe may be located in the first chamber, an outlet may be located on the tank along the second chamber. In some aspects, at least one computing device may be mounted on the distribution plate, the primary circuit dielectric cooling fluid may enter the first chamber of the tank through the plurality of ports in the distribution pipe, the primary circuit dielectric cooling fluid may flow through the multiple repeated sets of the pattern of holes in the distribution plate to contact the at least one computing device, and the heat may dissipate from the at least one computing device into the dielectric cooling fluid, and the dielectric cooling fluid may flow through the siphon wall into the second chamber when the surface of the dielectric cooling fluid is above the level of a transfer port and then exits the tank via the outlet.


In some aspects, the tank may have a pair of side walls and an attachment structure mounted on the floor, and siphon wall may be mounted to the floor via the attachment structure and the siphon wall may abut the pair of side walls.


In some aspects, the siphon wall may have a first edge and a second edge that are spaced away from each other, the second edge may be proximal to the floor, a plurality of transfer ports may be proximal to the first edge and located below the surface of the dielectric cooling fluid such that the plurality of transfer ports are located between the floor and the surface of the dielectric cooling fluid, and the plurality of transfer ports may be spaced away from each other. Put another way, the volume of cooling fluid that fills both the first chamber of the tank and the second chamber of the tank is configured such that the plurality of transfer ports is disposed below the fluid surface of the volume of dielectric cooling fluid.


In some aspects, the siphon wall may have an elongated transport port that is located below the surface of the dielectric cooling fluid such that the elongated transfer port is located between the floor and the surface of the dielectric cooling fluid, and the tank has a length, and the elongated transfer port may extend the length of the tank. The transfer port may provide a fluid communication channel between the first chamber of the tank and the second chamber of the tank.


In some aspects, the siphon wall may include a back wall such that a cavity may be formed between the siphon wall and the back wall. The cavity may provide a vertical fluid communication channel between a transfer port and a siphon wall outlet that is vertically spaced away from the transfer port. In some aspects, the primary circuit dielectric cooling fluid flows from the first chamber to the second chamber by flowing from first chamber of the tank through the transfer port into the cavity and through the cavity and out of the siphon outlet.


In some aspects, the siphon wall can be directly fastened to the distribution plate.


Various aspects include devices, systems, and methods for providing uniform fluid distribution of cooling fluid to a plurality of computing devices in an immersion cooling system. Some aspects may include distributing primary circuit dielectric cooling fluid into a first chamber of a tank through a first set of a plurality of holes in a distribution pipe and a second set of a plurality of holes in a distribution plate, the first set and second set of the plurality of holes may be aligned, flowing the primary circuit dielectric cooling fluid onto at least one computing device, and the at least one computing device may be mounted on the distribution plate. In some aspects, dividing the tank into the first chamber and a second chamber with a siphon wall, the siphon wall has at least a transfer port, passing the primary circuit dielectric cooling fluid from the first chamber to the second chamber through the transfer port of the siphon wall, and sending the primary circuit dielectric cooling fluid out of the tank via an outlet located in the second chamber.





BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated herein and constitute part of this specification, illustrate exemplary embodiments of the claims and together with the general description given above and the detailed description given below, serve to explain the features of the claims.



FIGS. 1A and 1B are schematic diagrams of example systems for cooling one or more computing devices in accordance with various embodiments.



FIG. 2 is an isometric view of a tank for use in accordance with various embodiments.



FIG. 3 illustrates an example immersion cooling system for a plurality of computing devices in accordance with various embodiments.



FIGS. 4A and 4B are top and isometric views of an example tank of a multi-computing device immersion cooling system in accordance with various embodiments.



FIG. 5 is an isometric view of an example distribution pipe for use in a multi-computing device immersion cooling system in accordance with various embodiments.



FIG. 6 is an isometric view of the example distribution pipe of FIG. 5 installed in the example tank of FIGS. 4A and 4B in accordance with various embodiments.



FIG. 7 is an illustration of a distribution plate for use in a multi-computing device immersion cooling system in accordance with various embodiments.



FIGS. 8A-8C are top and side views of an example distribution pipe and distribution plate in a tank of a multi-computing device immersion cooling system in accordance with various embodiments.



FIG. 9 is a top perspective view of an example system for use in a multi-computing device immersion cooling system in accordance with various embodiments.



FIG. 10 is a front view of an example siphon wall according to the system of FIG. 11 in accordance with various embodiments.



FIG. 11 is a back perspective view of the example siphon wall of FIG. 10 according to the system of FIG. 9 in accordance with various embodiments.



FIG. 12 is a top perspective view of an example alternative embodiment of the system of FIG. 9 for use in a multi-computing device immersion cooling system in accordance with various embodiments.



FIG. 13 is a cross-sectional side view of a portion of the example system of FIG. 12 with a modification in accordance with various embodiments.



FIG. 14 is a top perspective view of a portion of the example system of FIG. 12 in accordance with various embodiments.





DETAILED DESCRIPTION

Various embodiments will be described in detail with reference to the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts. References made to particular examples and implementations are for illustrative purposes and are not intended to limit the scope of the claims.


Immersion cooling racks, in accordance with various embodiments provide a pool of dielectric cooling fluid in a tank. The dielectric cooling fluid may be circulated in the tank such that heat may be rejected from the heated dielectric cooling fluid to the atmosphere (typically via an external cooling device such an evaporative cooling tower) and cooled dielectric cooling fluid may then be delivered to the heat-generating electronic components that would otherwise overheat.


As dielectric cooling fluid interacts with computing devices, the dielectric cooling fluid may absorb portions of the heat that is generated by the operating computing devices. The absorbed heat may warm the dielectric cooling fluid. As the dielectric cooling fluid warms, the dielectric cooling fluid becomes less effective as a cooling agent. Thus, there is a desire to transport the warmed dielectric cooling fluid into a reservoir and then subsequently out of the tank, so that the warmed dielectric cooling fluid may be cooled and re-introduced into the tank. There are various methods for distributing the flow of the dielectric cooling fluid.


Siphon walls are a method of distributing the flow of the dielectric cooling fluid that may result in reduced wear and tear as well as maintenance requirements. Siphon walls may be modular and scaled to fit a variety of embodiments. Thus, siphon walls may be adaptable to different fluid source configurations. Siphon walls utilize minimal active mechanical interactions in which energy can be lost due to friction, siphon walls may be very efficient to transfer energy.


Various embodiments disclosed herein provide for systems and methods for ensuring a uniform flow distribution of the cooled dielectric cooling fluid through the immersion tank configured to house multiple computing devices. Such systems may provide the uniform flow distribution without varying the size of a distribution pipe (sometimes referred to as a duct). Rather, the systems and methods of various embodiments operate through an optimum ratio of outlet and inlet surface areas of a distribution pipe and distribution plate in order to attain sufficient back pressure for uniform flow. The various embodiments may also include a siphon wall that provides for the transport of dielectric cooling fluid that has absorbed heat generated from the multiple computing devices for cooling and subsequent re-cycling into the immersion tank.


Components of the cooling system may be scalable to different form factors and input variables. In particular, a change in the number of computing devices, power rating, distribution pipe clearance, and inlet flow rate may require the number and diameter of the holes in the distribution pipe and in the distribution plate to be adjusted accordingly. In various embodiments, the hole pattern on the distribution plate may be adjusted based on the particular type and layout of the computing device(s) being used.


Dielectric immersion cooling systems for use in various embodiments may include at least one rack, independently operable computing devices (e.g., servers) configured to be inserted into the rack vertically, and primary flow of dielectric cooling fluid through the rack, with heat removal through a secondary flow of water or other cooling fluid loop.


The terms “dielectric cooling fluid” and “oil” may be used interchangeably herein to refer to any single-phase dielectric cooling fluid, including but not limited to, a mineral oil or a synthetic derivative thereof.


The term “water” as used herein refers to water or a water mix, such as water and glycol mix commonly used in combination with a chiller or dry cooler, or water and chemical mix, such as commonly used in direct cooling of water by a cooling tower. As will be understood by one of ordinary skill in the art, water is merely an example of one secondary circuit cooling fluid according to various embodiments, and is not intended to be limiting.



FIG. 1A is a schematic block diagram that illustrates a system 200 for cooling one or more rack-mountable computing devices 230. The rack-mountable computing device may include at least one computing device 230, arranged in one or more rack, for example, in a data center. A data center in various embodiments may be a physical location housing one or more computing devices. While a rack according to various embodiments may be a frame or enclosure that contains multiple mounting slots, each designed to hold a hardware unit. In various embodiments, the hardware unit may represent any equipment module, such as a server, computing device, crypto-miner, network router, hard-drive arrays, data acquisition equipment, etc.


As shown in FIGS. 1A-1B, a system 200 may include an immersion tank 210 (also referred to as simply the “tank”) having an interior volume containing a volume of primary circuit dielectric cooling fluid, which has a surface 250. Mounting members or rails (but not limited to such, can include those known in the art) may be positioned within the interior volume of the tank 210 and configured to receive and mount a plurality of computing devices 230 into the tank 210. In embodiments in which the computing devices 230 are servers, at least a portion of each computing device 230 may be submerged within the dielectric cooling fluid in order to sufficiently cool each respective computing device 230 when the tank 210 is sufficiently full of the primary circuit dielectric cooling fluid. Preferably, each such computing device 230 during operation is below surface 250 of the primary circuit dielectric cooling fluid.


In some embodiments, the hardware units mounted in the tank 210 may include at least one hard disk drive (HDD) (not shown). Since some HDDs are not encased in a sealed enclosure, such components may be damaged by submersion in the primary circuit dielectric cooling fluid. Therefore, mounting members may be designed such that when the HDDs are mounted thereon, the HDDs remain above the surface 250 of the primary circuit dielectric cooling fluid in the tank 210. In other embodiments, some HDDs may be sealed units and filled with helium such that the sealed HDDs may be mounted below the surface of the dielectric cooling fluid.


The primary circuit dielectric cooling fluid may absorb heat that is generated by the computing devices 230. The primary circuit dielectric cooling fluid may be fluidly coupled through suitable piping or lines to a pump 212. The pump 212 pumps the primary circuit dielectric cooling fluid, which has absorbed heat that has been generated by the computing devices 230, through suitable piping or lines to a heat exchanger 216. The heat exchanger 216 may be associated with a heat-rejection or cooling apparatus 218. Before getting to the heat exchanger 216, however, the primary circuit dielectric cooling fluid may pass through a filter 214 to filter out any foreign material that may have entered into the primary circuit dielectric cooling fluid.


The heat exchanger 216 may reject the heat that is absorbed in the incoming dielectric cooling fluid, and fluidly couple the cooled primary circuit dielectric cooling fluid through a return fluid line or piping 220 back into the tank 210. The heat rejected from the heated primary circuit dielectric cooling fluid through the heat exchanger 216 may then be selectively used by alternative heat rejection or cooling apparatus 218 to dissipate, recover, or beneficially use the rejected heat depending on different environmental conditions and/or computing system operating conditions to which the system 200 is subjected.


Either or both the heat exchanger 216 and the cooling apparatus 218 may be local or remote to the cooling system 200. The system 200 may also include a controller 270 with software for implementing the methods according to various embodiments. The controller 270 may receive monitoring signals of various operational parameters from components of the cooling system 200 as well as the environment. The controller 270 may generate control signals to control various components of the cooling system 200 in order to sufficiently cool the computing devices 230. Particularly, the controller 270 may monitor the temperature of the primary circuit dielectric cooling fluid in at least one location within the fluid circuit, for example where the heated primary circuit dielectric cooling fluid exits the plurality of computing devices 230 and heat conductive extensions.


The controller 270 may also monitor the temperature of the heat generating electronic components in the computing devices 230 by electrically connecting the controller 270 to the diagnostic output signals generated by conventional rack-mountable computing devices.


The controller 270 may also monitor the flow of the primary circuit dielectric cooling fluid. Based upon such information, the controller 270 may output control signals to the pump 212 and heat rejection or cooling apparatus 218 to adjust the flow of the primary circuit dielectric cooling fluid through the fluid circuit and the amount of the heat being rejected by the heat rejection or cooling apparatus 218.



FIG. 1B illustrates an alternative configuration for a system 200 for cooling one or more rack-mountable computing devices 230. With reference to FIG. 1B, the system 200 may be configured for cooling one or more computing devices 230, such as a server. The computing devices 230 may be arranged in one or more racks in a data center.


The system 200 includes a tank 210 having an interior volume containing a primary circuit dielectric cooling fluid. The primary circuit dielectric cooling fluid has a surface 250. The computing devices 230 may be mounted inside the tank 210 using mounting members.


The flow circuit of the primary circuit dielectric cooling fluid may be internal to the tank 210. A thermal coupling device 280, such as a heat exchanger, may be mounted within the tank 210 and the fluid circuit passes over the computing device 230, so that at least a portion of the heat generated by the computing devices 230 may be absorbed by the primary circuit dielectric cooling fluid that flows through the thermal coupling device 280. Cooled primary circuit dielectric cooling fluid exits the coupling device 280 and at least a portion of the cooled primary circuit dielectric cooling fluid circulates in the internal fluid circuit back through the computing devices 230 and the heat conductive extensions.


The system 200 in FIG. 1B includes a secondary heat rejection or cooling apparatus 218 having a secondary cooling fluid, such as a gas or liquid flowing in piping or lines, forming a second fluid circuit. The secondary cooling apparatus 218 may include an associated heat exchanger (not shown) t to reject heat from the primary circuit dielectric cooling fluid in the secondary fluid circuit. The heat rejected from the heated primary circuit dielectric cooling fluid in the secondary fluid circuit through the heat exchanger 280 associated with the secondary cooling apparatus 218 may then be selectively dissipated, recovered, or beneficially used depending on the different environmental conditions and/or computing system operating conditions to which the system is subject.


The system 200 may include a controller 270 that may receive monitor signals of various operational parameters from various components of the cooling system 200 and the environment. The controller 270 may generate control signals to control various components of the cooling system 200 to maintain the heated primary circuit dielectric cooling fluid exiting the computing devices 230 in the tank 210 at a specific temperature. In this manner, each of the plurality of computing devices 230 may be sufficiently cooled while reducing the total amount of energy needed to cool the computing devices 230.


The controller 270 may monitor the temperature of the primary circuit dielectric cooling fluid at least one location within the internal fluid circuit, for example, where the heated primary circuit dielectric cooling fluid exits the computing devices 230 immersed in the tank 210. The controller 270 may also monitor the temperature of the heat-generating electronic components in the computing devices 230 by electrically connecting the controller to the diagnostic output signals generated by conventional rack-mountable computing systems.


The controller 270 may also monitor the flow and temperature of the primary circuit dielectric cooling fluid in the external fluid circuit. Based upon such information, the controller 270 may output signals to the heat rejection or cooling apparatus 218 to adjust the flow of the primary circuit dielectric cooling fluid through the external fluid circuit and the amount of the heat being rejected by the heat rejection or cooling apparatus 218 for sufficiently cooling each respective computing devices 230 while maintaining the heated primary circuit dielectric cooling fluid exiting the computing systems and heat conductive extensions at the specific temperature to reduce the amount of energy consumed to sufficiently cool each of the computing devices 230.


By maintaining the existing primary circuit dielectric cooling fluid at an elevated level, the cooling system may be used with a number of different techniques for using or dissipating the heat (e.g., heat recapture, low power heat dissipation, or refrigeration).



FIG. 2 illustrates a tank 122 in an example immersion cooling system that may be used in various embodiments. With reference to FIGS. 1A-2, components of each rack-mounted computing device 230 (e.g., server) may generate heat within a rack 320. The heat from the rack may be dissipated in a primary circuit dielectric cooling fluid in which it is submerged in the rack 320, such as oil. The heated primary circuit dielectric cooling fluid may exit the rack 320 and/or the tank 122, and may be pumped to a heat exchanger 280 (e.g., a dielectric cooling fluid-to-water heat exchanger). The heat exchanger 280 may pass the heat from the primary circuit dielectric cooling fluid to a heat removal system, and the primary circuit dielectric cooling fluid may return to the rack 320. The heat removal system may cause the heat captured heat to be expelled or reused, after which the cooled secondary cooling fluid (i.e., water or a water/glycol mixture) may be returned to the heat exchanger 280.


The tank 122 may optionally have a hinged or removable lid, or an open top. The tank 122 may be fabricated of steel, a sufficiently strong plastic that is compatible with the liquid coolant used as a cooling medium, or other suitable material. The tank 122 may contain a plurality of independently operable data processing modules 310 (e.g., computing devices) mounted vertically. Each data processing module 310 may be independently removable and replaceable without affecting the position or operation of other data processing modules. The independently operable data processing modules 310 may be mounted in an array that is arranged horizontally on a rack, and immersed at least partially in a dielectric cooling fluid.



FIG. 3 illustrates another immersion cooling system according to various embodiments. With reference to FIGS. 1A-3, system 300 may include one or a plurality of power distribution units (PDUs), and may be capable of supporting two racks 320 and up to forty-eight (48) computing devices 230 (e.g., servers). Although a greater or fewer number of computing devices may be supported. In various embodiments, twenty-four (24) computing devices 230 may be housed in each rack. Although a greater or fewer number of computing devices may be housed in each rack. The system 300 may include a frame made of low-profile metal tubes engineered for strength, and may hold a cooling distribution unit (CDU) under the rack(s), providing easy access to components and a minimal footprint.


In various embodiments, the system 300 may also include a monitoring system that checks coolant pressure, temperatures, and levels to ensure efficient operation, and that identifies faults in order to maximize uptime. For example, embodiment systems may be configured with one or more sensors to monitor various parameters and detect conditions associated with failure modes. In particular, such conditions may include that the temperature and/or conductivity of the primary circuit dielectric cooling fluid in the rack is too high, that the fluid volume in the rack is too high, or that the pressure of water in the circuit supplying water to an oil-to-water heat exchanger is too low.


In order to supply a uniform distribution of primary circuit dielectric cooling fluid to all of the computing devices (e.g., servers) in a tank, the various embodiment systems may include a distribution pipe that is enclosed on top by a distribution plate having a plurality of strategically located holes. In various embodiments, the floor 401 of the tank may have an alternating pattern elevated platforms 402 and depressed alleys 404. For example, the size and/or location of the plurality of strategically located holes in the distribution plate 700 may be configured to optimize the flow of coolant based on the specific types and number of computing devices 230, 310 housed in the rack 320.



FIGS. 4A and 4B show a portion of a tank according to various embodiments. With reference to FIGS. 1A-4A, the portion of the tank 400 has a floor 401 that includes twelve alternating elevated platforms 402 and thirteen depressed alleys 404. In various embodiments, a duct 406 in the middle of the floor 401 of the tank interior is configured to receive a distribution pipe 500 which may be connected to an inlet of the tank 400 at one end. The configuration of the tank 400 is merely an example, and is not intended to be limiting. Other configurations, such as those including more or less elevated platforms and/or valleys, etc. may be substituted for the example configuration of the tank 400, and other configurations are in accordance with the various embodiments. The tank 400 may be thermally formed using a poly-carbonate thermal vacuum process. However, other processes and materials may be used to form the tank 400.


In various embodiments, the distribution pipe 500 may run across the interior of the tank 400, and may be capped at the opposite end of the connection to the tank inlet. FIG. 5 illustrates a distribution pipe according to various embodiments. With reference to FIGS. 1A-5, the distribution pipe 500 may have a plurality of equally spaced ports, each having a fixed internal diameter.


In some embodiments, the internal diameter of the ports in the distribution pipe 500 may be 10 mm. In some embodiments, the distribution pipe 500 may have twenty-six total ports, with thirteen on each side (i.e., in each direction facing the depressed alleys (e.g., 404)). In various embodiments, the number and placement of the ports in the distribution pipe 500 may be configured to match the depressed alleys 404 in the tank 400, such as providing two ports per alley 404.


The configuration of the distribution pipe 500 is merely an example and is not intended to be limiting. Other configurations, such as those including more or less ports, or ports of different sizes and/or placements, etc. may be substituted for the example configuration of the distribution pipe 500, and other configurations are in accordance with the various embodiments.



FIG. 6 illustrates a distribution pipe 500 installed in a tank 400 according to various embodiments. With reference to FIGS. 1A-6, system 600 includes the tank 400 and the distribution pipe 500, which is connected to the inlet port of the tank 400 on one end and capped on the other end.


In various embodiments, an immersion cooling system may also include a distribution plate. With reference to FIGS. 1A-7, the distribution plate 700 may be positioned in the tank 400 to enclose an interior volume around the distribution pipe 500. In some embodiments, the distribution plate 700 may be perforated to include multiple holes, which may be positioned in a pattern that is repeated. For example, in some embodiments the distribution plate 700 may be configured with holes having an equally-sized internal diameter of 5 mm. For example, the pattern 702 may be twenty (20) holes, configured as a block of eighteen (18) holes with two (2) holes spaced further apart to the side. In some embodiments, the distribution plate 700 may have a total number of four hundred eighty (480) holes, such as twenty-four (24) repeated sets of the pattern 702. In some embodiments, the holes in the distribution plate 700 may have equally sized internal diameters, while in other embodiments the holes may have internal diameters of unequal sizes.


In various embodiments, a plurality of computing devices 230 (e.g., servers) may be mounted over the distribution plate 700 in the tank, and the repeating patterns of holes 702 may be configured to align with the computing devices 230. For example, in some embodiments, each repeating pattern 702 of twenty (20) holes may correspond to a computing device 230, with the two (2) holes on the side positioned to a power supply of a computing device 230, and the remaining eighteen (18) holes positioned to align with the rest of the computing device 230. In some embodiments, the twenty-four (24) repeating sets of the pattern 720 may match the twenty-four (24) computing devices 230 that may be mounted in the tank 400. As shown in FIG. 8B, the twenty-four (24) repeating sets of the pattern 720 may form a two by twelve (2×12) array arranged to provide a uniform flow distribution of primary circuit cooling fluid over and across twenty-four (24) mounted computing devices 230. In other embodiments, the repeating sets of the pattern 720 may form other dimensioned arrays, such as a one by twenty-four (1×24), three by eight (3×8), four by six (4×6) array. In the various embodiments that are configured with varying arrays, the size of ports and perforated holes may vary to provide the uniform flow distribution.


The configuration of the distribution plate 700 is merely an example, and is not intended to be limiting. Other configurations, such as those including more or less holes, and/or different patterns, sizes, or placements of holes, etc. may be substituted for the example configuration of the distribution plate 700, and other configurations are in accordance with the various embodiments.


As shown in FIG. 8A, there may be a small clearance gap between the distribution pipe and the distribution plate according to various embodiments. With reference to FIGS. 1A-8C, the system 800 may include a clearance gap of 6.5 mm between the distribution pipe 500, 802 and the distribution plate 700, 804 within the tank 400.


In various embodiments, the primary circuit dielectric cooling fluid may enter the system 800 through the distribution pipe 500 and may flow into the interior of the tank 400 through ports in the distribution pipe 500. In various embodiments, the primary circuit dielectric cooling fluid exiting the ports may flow directly into the alleys (e.g., 404) of the tank 400 floor 401.


Such ports may have an interior diameter of, for example, 10 mm. In various embodiments, the primary circuit dielectric cooling fluid may fill the alleys 404 and the interior volume of the tank 400, and may eventually pass through the holes in the distribution plate 700 to surround the casings of the computing devices 230 mounted in the tank (not shown in FIGS. 8A-8C).


By pumping the primary circuit cooling fluid through the distribution pipe 500, the ports in the distribution pipe 500, into the alleys 404 and up through the pattern of holes in the distribution plate 700, the various embodiments may be designed to achieve a total inlet mass flow rate of 3.95 kg/s per tank at 40° C. inlet temperature. Such inlet flow rate may ensure efficient heat dissipation in a 6 kW server. Computation fluid dynamics (CFD) methodology demonstrates that flow is well distributed and uniform throughout the various positions along the distribution plate 700, with a mean of 0.165 kg/s and a standard deviation of 0.010 kg/s. In a laboratory setting for verification, the experimental dataset had a mean of 0.161 kg/s and a standard deviation of 0.011 kg/s.


Pressure at the outlet ranged from 88 Pa to 116 Pa with a mean of 102 Pa and a standard deviation of 9 Pa. Minimum and maximum velocities at the outlet were 0.41 m/s and 0.48 m/s, respectively, with a mean of 0.45 m/s and a standard deviation of 0.025 m/s.


The tank 400 is scalable to different form factors and input variables. A change in the number of computing devices (miners), power rating, distribution pipe clearance, and inlet flow rate would require the number and diameter of the holes on the distribution pipe 500 and distribution plate 700 to be adjusted accordingly. The hole pattern 702 on the distribution plate 700 in the embodiment disclosed above may be set for a specific type of miner. The hole pattern 702 can also be modified to suit a particular miner layout.



FIG. 9 illustrates an embodiment system 900 includes the tank 400. The tank may have an elongated front wall 902 and rear wall 904 that are attached to each other by a first side wall 906 and a second side wall 908. The walls 902, 904, 906, 908 may extend upward from a bottom surface. The walls 902, 904, 906, 908 may include a plurality of continuous ridges 916 that are vertically spaced away from each other. In some embodiments, the continuous ridges may provide support for components such as a siphon wall 920. The walls 902, 904, 906, 908 may surround the distribution plate 700 (see FIGS. 7, 12, and 14). A base attachment 909 may be mounted to both side walls 906, 908 and extends across a length of the tank 400. The base attachment 909 may be proximal to the distribution plate 700. In various embodiments, a siphon wall 920 may be mounted to the base attachment 909. In other embodiments, the siphon wall 920 may be directly attached to the side walls 906, 908.


Referring to FIGS. 10 and 11, in various embodiments, the siphon wall 920 may have a top edge 921, a bottom edge 923, a first side edge 926, and a second side edge 928. The side edges 926, 928 each have a first step 929 and a second step 930 that may be vertically spaced away from each other. The steps 929, 930 may be used to mount the siphon wall 920 into the side walls 906, 908. For example, in some embodiments, steps 929 and 930 may engage with continuous ridges 916 to mount the siphon wall 920 within the tank. The siphon wall 920 may include a back wall 932 that is mounted to a back side of the siphon wall 920. The back wall 932 may be distal from the siphon wall 920. The back wall 932 extends from the side edges 926, 928 of the siphon wall 920, and the back wall 932 is spaced away from the top edge 921 and the bottom edge 923. During attachment, the back wall 932 is directly adjacent to the base attachment 909. In an exemplary embodiment, as shown in FIGS. 11 and 13, the back wall 932 may form a chamber for a cavity 1024. The cavity 1024 may be formed along the back wall 932. With reference to FIGS. 9-14, the transfer port (922, 924, 1022) may be defined by a continuous perimeter. In various embodiments all edges of the continuous perimeter may be located substantially below the fluid surface 250 of the volume of dielectric cooling fluid that fills the tank 900, 1000. In these exemplary embodiments, the fluid is transferred through the siphon wall 920, 1020 when the dielectric cooling fluid surface 250 is higher than the transfer ports (922, 924, 1022). In this manner a positive pressure may be exerted to transport dielectric cooling fluid that has absorbed heat generated by the computing devices 230, 310 in the first chamber 910 of the tank into a second chamber 912 of the tank via the transfer port(s) (922, 924, 1022). The main chamber transport port (922, 924, 1022) may be a singular transport port that may extend for a majority of the length of the siphon wall 920 or may comprise multiple transfer ports such as 922 and 924 as shown in FIGS. 9 and 10. While FIGS. 9 and 10 illustrate two (2) transfer ports 922, 924, a greater or fewer number of transfer ports may be used. In some embodiments, a screen or mesh may cover the transfer ports (922, 924, 1022) in order to prevent solids of stray material from clogging the transfer port.


The cavity 1024 may provide a vertical fluid communication channel between the transfer port (922, 924, 1022) and a siphon outlet 934 formed at the bottom of the cavity 1024. The siphon outlet 934 may be vertically spaced away from the transfer port (922, 924, 1022). The siphon outlet 934 may be in fluid communication with the cavity 1024 and the transfer ports (922, 924, 1022). The dielectric fluid enters the second chamber 912 (also referred to as a reservoir chamber) via the outlet 934. The outlet 934 may have one or a plurality of outlet ports. For example, FIG. 11 illustrates two (2) outlets 934, but greater or fewer outlets may be used. The cavity 1024 may protect the dielectric cooling fluid from the atmosphere during fluid transfer because the atmosphere may act as an oxidant that may cause degradation of the dielectric cooling fluid. In an exemplary embodiment, the siphon wall 920 may be approximately 330 mm in height, and the back wall 932 may be 15.88 mm×2451.1 mm wide×273.05 mm.


As discussed above, a plurality of transfer ports—e.g., a first transfer port 922 and a second transfer port 924—may be formed within the siphon wall 920. The transfer ports 922, 924 may be formed on a surface of the siphon wall 920 that faces away from the back wall 932. The transfer ports 922, 924 may be more proximal to the top edge 921 than to the bottom edge 923. The transfer ports 922, 924 are located below a fluid surface 250 of the volume of dielectric cooling fluid that fills both the first chamber of the tank and the second chamber of the tank such that the transfer ports 922, 924 are between the floor 401 and the fluid surface 250 when the dielectric cooling fluid is moving through the siphon wall 920. This can be contrasted to a weir, in that the weir must be proximal to a fluid surface 250 so that the fluid may flow over the weir. Whereas with a siphon, the portion that transfer fluid is below a fluid level line such that the hydraulic pressure causes the fluid to move through the siphon. The fluid level line may be at various levels above the transfer ports 922, 924 of the siphon wall 920. The transfer ports 922, 924 may be spaced away from each other. The transfer ports 922, 924 may extend between the steps 929, 930 of the side edges 926, 928. In an exemplary embodiment, the transfer ports 922, 924 may each be 25.4 mm wide and 1219.2 mm long and may have a total of 768 holes (192×4) of 3.18 mm ID. The transfer ports 922, 924, in exemplary embodiments, may be 23.81 mm apart and 20.62 mm from the top edge 921 of the siphon wall 920.


In various embodiments, as shown in FIG. 9, the siphon wall 920 may divide the tank 400 to create a main chamber 910 and a buffer chamber (i.e., reservoir chamber) 912. The transfer ports 922, 924 face the main chamber 910 and the back wall 932 faces the buffer chamber 912. The main chamber 910 may have a greater volume than the reservoir chamber 912. In some embodiments, the main chamber 910 may be wider and deeper than the buffer chamber 912. The transfer ports 922, 924 are located such that a top line of the transfer ports 922, 924 is below the surface of the dielectric cooling fluid that fills the main chamber 910.


In various embodiments, as shown in FIGS. 9-12, the main chamber 910 may have installed therein computing device equipment and the buffer chamber 912 may act as a buffer fluid compartment after the dielectric cooling fluid has been warmed. As the dielectric cooling fluid flows through the ports in the distribution pipe 500, into the depressed alleys 404 and up through the pattern 702 of holes in the distribution plate 700—the dielectric cooling fluid flows up through the main chamber 910. The operation of the installed computing device equipment generates heat that eventually warms the dielectric cooling fluid. Thus, the exemplary following process (of the various embodiments) may be used to transport the warmed dielectric cooling fluid out of the system 900, into a cooling apparatus 218, and then re-introduce the cooled dielectric cooling fluid back into the system 900.


In instances in which the dielectric cooling fluid reaches a predetermined fluid level in the main chamber 910, the dielectric cooling fluid may move through the siphon wall 920 into the reservoir chamber 912. This may occur when the dielectric cooling fluid passes through the transfer ports 922, 924. Transfer ports may be included on the transfer ports 922, 924 to act as screens to filter and collect debris from the dielectric cooling fluid. Once the dielectric cooling fluid passes through the transfer ports 922, 924, the cooling fluid may enter the cavity 1024 of the back wall 932 (enters cavity 1024 via the ports 922, 924). Then, the cooling fluid may pass through the cavity 1024 into the buffer chamber 912. The enclosed cavity 1024 may prevent air bubbles from forming into the dielectric cooling fluid, as the dielectric cooling fluid enters the buffer chamber 912). The dielectric cooling fluid in the buffer chamber 912 may then pass through the tank outlet 914 (i.e., tank port) of the buffer chamber 912. The tank outlet 914 may transfer the dielectric cooling fluid into the heat exchanger 216, 280 (FIGS. 1A and 1B). The dielectric cooling fluid may then be put into a cooling apparatus 218 and then re-circulated into the system 900.


In an exemplary embodiment, the system 900 may have a total inlet mass flow rate of 3.95 kg/s per tank 400 at 40° C. inlet temperature. The inlet flow rate may be set at the above-mentioned value to ensure efficient heat dissipation in a 6 kW miner. However, in other embodiments, total mass flow rates may be greater or lesser. Computation fluid dynamics (CFD) results show that flow is well distributed with a mean of 0.165 kg/s and a standard deviation of 0.010 kg/s. The experimental dataset may have a mean of 0.161 kg/s and a standard deviation of 0.011 kg/s. Pressure at the outlets may range from 88 Pa to 116 Pa with a mean of 102 Pa and a standard deviation of 9 Pa. Minimum and maximum velocities at the outlet can be 0.41 m/s and 0.48 m/s, respectively, with a mean of 0.45 m/s and a standard deviation of 0.025 m/s. In this exemplary embodiment, 750 Pa may be acting on the tank outlet 914 of the buffer chamber 912 to stabilize the fluid level in the buffer chamber 912 and ensure that no air is sucked into the system 900. The tank 400 may be scalable to different form factors and input variables. A change in the number of computing device equipment (e.g., miners), power rating, distribution pipe clearance, and inlet flow rate would require the number and diameter of the holes on the distribution pipe 500 and the distribution plate 700 to be adjusted accordingly. The hole pattern on 702 the distribution plate 700 in the exemplary embodiment may be adjusted for specific types of miners. The hole pattern 702 can also be modified to suit a particular miner layout. The siphon wall 920 can also be modified according to the size of the tank 400 and therefore the miners served.


In an alternative embodiment, as shown in in FIGS. 12-14, the system 1000 may include a front wall 1002 and a rear wall 1004 as well as a first side wall 1006 and a second side wall 1008. A continuous ridge 1016 may be provided to support the siphon wall 1020 in some embodiments. A base attachment 1009 may be mounted on the side walls 1006, 1008 and the base attachment 1009 may abut the distribution plate 700. A siphon wall 1020 may be mounted into the base attachment 1009, and the siphon wall 1020 may be connected to the side walls 1006, 1008. The siphon wall 1020 may form a first chamber 1010 (sometimes referred to as a main chamber) and a second chamber 1012 (sometimes referred to as a buffer or reservoir chamber).


In an exemplary embodiment of the system 1000, as shown in FIG. 12-14, the transfer port 1022 may be located below the fluid surface 250 in the first chamber 1010. The tank outlet 1014 (i.e., tank port) in the second chamber 1012 may be located below the fluid level in the second chamber 1012.


Unlike the system 900 of FIG. 9, the system 1000 (FIGS. 12-14) may have one elongated transfer port 1022 on the siphon wall 1020. A pair of base fasteners 1018 may abut the base attachment 1009 and fasten the siphon wall 1020 to the distribution plate 700. A plurality of wall fasteners 1019 may be mounted to the rear wall 1004 and extend to and fasten to a back side of the siphon wall 1020. In an exemplary embodiment, there are four wall fasteners 1019 that are spaced away from each other and are connected to a side of siphon wall 1020 that faces away from the transfer port 1022, and the wall fasteners 1019 may be mounted just above the cavity 1024. The tank outlet 1014 may have a bolt-like cap on the end (FIG. 12) or a mesh-like screen outlet 1015 (FIG. 13).


Referring to all drawings and according to various embodiments of the present disclosure, an exemplary embodiment of the system 300, 600, 800 for providing uniform fluid distribution of dielectric cooling fluid to a plurality of computing devices in an immersion cooling system may include a tank 400 defining an open interior volume, a distribution pipe 500 positioned on the floor 401 and running across the length of the tank 400. A distribution plate 700 may be positioned over the distribution pipe 500, and at least one computing device 230, 310 mounted on the distribution plate 700. The dielectric cooling fluid may enter the open interior volume of the tank 400 through the plurality of ports in the distribution pipe 500. In the dielectric cooling fluid may flow through multiple repeated sets of the pattern of holes 702 in the distribution plate 700 to contact the at least one computing device 230, 310. Heat dissipates from the at least one computing device 230, 310 into the dielectric cooling fluid.


In one embodiment, a floor 401 of the tank 400 may include a plurality of alternating elevated platforms and alleys 402, 404. The distribution pipe 500 may include a plurality of ports that each align with one of the alleys 404. The distribution plate 700 may have a pattern of holes 702 in multiple repeated sets. Each of the at least one computing device 230, 310 may be aligned with one set of the pattern of holes 702.


In one embodiment, the distribution pipe 500 may be connected on one end to an inlet of the tank 400 and is capped on an opposite end.


Exemplary embodiments may further include a rack configured to house twenty-four computing devices 230 in the at least one tank 400. The plurality of ports may include thirteen equally spaced ports along each of two opposite sides (404) of the distribution pipe 500. The pattern of holes in the multiple repeated sets 702 may be twenty holes repeated twenty-four times over the distribution plate 700.


In one embodiment, each of the plurality of ports in the distribution pipe 500 may have an internal diameter of 10 mm, and each hole in the pattern of holes 702 in the distribution plate 700 may have an internal diameter of 5 mm.


In one embodiment, the distribution plate 804 may be positioned such that there is a 6.5 mm clearance gap between the top of the distribution pipe 802 and the bottom of the distribution plate 804.


In one embodiment, dielectric cooling fluid may be oil. In one embodiment, the system 300, 600, 800 may further include an oil-to-water heat exchanger 216, 280.


In one embodiment, at least one computing device 230 may include twenty-four servers housed in each of at least one rack within the tank 400.


In one embodiment, at least one computing device 230 may include twenty-four servers housed in each of at least one rack within the tank 400.


In one embodiment, the floor 401 of the tank 400 may include twelve elevated platforms 402 and thirteen alleys 404.


According to an aspect of the present disclosure, the system 900 may include uniform fluid distribution of dielectric cooling fluid to a plurality of computing devices 230 in an immersion cooling system include a tank 400 defining an open interior volume; a distribution pipe 500 positioned on a floor and the distribution pipe 500 has a plurality of ports; a distribution plate 700 positioned over the distribution pipe 500, wherein the distribution plate 700 has a pattern of holes 702; a siphon wall 920 mounted within the tank 400 and extends from the floor, the siphon wall 920 is configured to divide the open interior volume to define a first chamber 910 and a second chamber 912, and wherein the distribution plate 700 and the distribution pipe 500 are located in the first chamber 910; a tank outlet 914, 1014 is located on the tank 400 along the second chamber 912; and a rack configured to house at least one computing device 230 over the distribution plate 700, wherein the primary circuit dielectric cooling fluid enters the first chamber 910 of the tank 400 through the plurality of ports in the distribution pipe 500, wherein the primary circuit dielectric cooling fluid flows through the pattern of holes 702 in the distribution plate 700 to contact the at least one computing device 230, and wherein heat dissipates from the at least one computing device 230 into the dielectric cooling fluid, and the dielectric cooling fluid is configured to flow through a transfer port 922, 924 into the second chamber 912, wherein the transfer port 922, 924 is located below the dielectric cooling fluid level line, and the dielectric cooling fluid exits the tank 400 via the tank outlet 914, 1014.


In one embodiment, the tank may have a pair of side walls 906, 908, 1006, 1008 and an attachment structure 909, 1009 mounted on the floor 401, and siphon wall 920, 1020 may be mounted to the floor 401 via the attachment structure 909, 1009 and the siphon wall 920, 1020 may abut the pair of side walls 906, 908, 1006, 1008.


In one embodiment, the siphon wall 920 may have a first edge 921 and a second edge 923 that are spaced away from each other, the second edge 923 may be proximal to the floor 401, a plurality of transfer ports 922, 924 may be proximal to the first edge 921 and located below the fluid surface 250 such that the plurality of transfer ports 922, 924 are located between the floor 401 and the fluid surface 250, and the plurality of transfer ports 922, 924 may be spaced away from each other.


In one embodiment, the siphon wall 1020 may have an elongated transport port 1022 that is located below the fluid surface 250 such that the elongated transfer port 1022 is located between the floor 401 and the fluid surface 250, the tank has a length, and the elongated transfer port may extend the length of the tank.


In one embodiment, the siphon wall 920, 1020 may have a back wall 932 that forms a cavity 1024, the cavity 1024 providing a vertical fluid communication channel between the transfer port 922, 924, 1022 and a siphon outlet 934, wherein the primary circuit dielectric cooling fluid flows from the first chamber 910 to the second chamber 912 by flowing through the transfer port 922, 924, 1022 through the cavity 1024 and out of the siphon outlet 934.


In one embodiment, the siphon wall 1020 may be directly fastened to the distribution plate 700.


As explained above and shown in FIGS. 9-14, an exemplary embodiment of the method for providing uniform fluid distribution of cooling fluid to a plurality of computing devices in an immersion cooling system may include distributing primary circuit dielectric cooling fluid into a first chamber 910, 1010 of a tank 400 through a first set of a plurality of holes in a distribution pipe 500 and a second set of a plurality of holes 702 in a distribution plate 700, the first set and second set of the plurality of holes may be aligned, flowing the primary circuit dielectric cooling fluid onto at least one computing device 230, and the at least one computing device 230 may be mounted on the distribution plate 700. In some aspects, dividing the tank into the first chamber 910, 1010 and a second chamber 912, 1012 with a siphon wall 920, 1020, the siphon wall 920, 1020 has at least a transfer port, passing the primary circuit dielectric cooling fluid from the first chamber 910, 1010 to the second chamber 912, 1012 through the transfer port 922, 924, 1022 of the siphon wall 920, 1020, and sending the primary circuit dielectric cooling fluid out of the tank 400 via a tank outlet 914, 1014 located in the second chamber 912, 1012.


The foregoing descriptions of systems, devices, and methods are provided merely as illustrative examples and are not intended to require or imply that the steps of the various embodiments must be performed in the order presented. As will be appreciated by one of skill in the art the order of steps in the foregoing embodiments may be performed in any order. Words such as “thereafter,” “then,” “next,” etc. are not intended to limit the order of the steps; these words are used to guide the reader through the description of the methods. Further, any reference to claim elements in the singular, for example, using the articles “a,” “an” or “the” is not to be construed as limiting the element to the singular.


The preceding description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the following claims and the principles and novel features disclosed herein.

Claims
  • 1. An immersion cooling system configured to cool a plurality of computing devices using a primary circuit dielectric cooling fluid, the system comprising: a tank defining an open interior volume;a distribution pipe positioned on a floor and the distribution pipe has a plurality of ports;a distribution plate positioned over the distribution pipe, wherein the distribution plate has a pattern of holes;a siphon wall mounted within the tank and extends from the floor, the siphon wall is configured to divide the open interior volume to define a first chamber and a second chamber, and wherein the distribution plate and the distribution pipe are located in the first chamber;a tank outlet is located on the tank along the second chamber; anda rack configured to house at least one of the plurality of computing devices over the distribution plate, wherein the primary circuit dielectric cooling fluid enters the first chamber of the tank through the plurality of ports in the distribution pipe,wherein the primary circuit dielectric cooling fluid flows through the pattern of holes in the distribution plate to contact the at least one computing device, andwherein heat dissipates from the at least one computing device into the dielectric cooling fluid, and the dielectric cooling fluid is configured to flow through a transfer port into the second chamber, wherein the transfer port is located below the dielectric cooling fluid surface, and the dielectric cooling fluid exits the tank via the tank outlet.
  • 2. The immersion cooling system of claim 1, wherein the tank has a pair of side walls and an attachment structure mounted on the floor, and siphon wall is mounted to the floor via the attachment structure and the siphon wall abuts the pair of side walls.
  • 3. The immersion cooling system of claim 1, wherein the siphon wall has a first edge and a second edge that are spaced away from each other, and the second edge is proximal to the floor,wherein the transfer port comprises a plurality of transfer ports proximal to the first edge and each located below the dielectric cooling fluid surface such that the plurality of transfer ports are located between the floor and the dielectric cooling fluid surface, andthe plurality of transfer ports are spaced away from each other.
  • 4. The immersion cooling system of claim 1, wherein the transfer port is an elongated transfer port that is located below the dielectric cooling fluid surface such that the elongated transfer port is located between the floor and the dielectric cooling fluid surface, andthe tank has a length and the elongated transfer port extends the length of the tank.
  • 5. The immersion cooling system of claim 1, wherein the siphon wall further comprises a back wall that forms a cavity, the cavity providing a vertical fluid communication channel between the transfer port and a siphon outletwherein the primary circuit dielectric cooling fluid flows from the first chamber to the second chamber by flowing through the transfer port through the cavity and out of the siphon outlet.
  • 6. The immersion cooling system of claim 1, wherein the siphon wall is directly fastened to the distribution plate.
  • 7. A method for an immersion cooling system to distribute a primary circuit dielectric cooling fluid to a plurality of computing devices, the method comprising: distributing the primary circuit dielectric cooling fluid into a first chamber of a tank through a first set of a plurality of holes in a distribution pipe and a second set of a plurality of holes in a distribution plate, wherein the first set and second set of the plurality of holes are aligned;flowing the primary circuit dielectric cooling fluid onto at least one computing device, wherein the at least one computing device is mounted on the distribution plate;dividing the tank into the first chamber and a second chamber with a siphon wall, and wherein the siphon wall has at least a transfer port;passing the primary circuit dielectric cooling fluid from the first chamber through the siphon wall into the second chamber, wherein the dielectric cooling fluid flows through a transfer port located in the siphon wall below a dielectric cooling fluid surface; andsending the primary circuit dielectric cooling fluid out of the tank via an outlet located in the second chamber.
  • 8. The immersion cooling system of claim 1, wherein the tank has a one end with an inlet and an opposite end, the distribution pipe is connected on the one end to the inlet of the tank and is capped on the opposite end.
  • 9. The immersion cooling system of claim 1, further comprising: a rack configured to house twenty-four computing devices in the at least one tank, wherein: the plurality of ports comprises thirteen equally spaced ports along each of two opposite sides of the distribution pipe; andthe pattern of holes in the multiple repeated sets comprises twenty holes repeated twenty-four times over the distribution plate.
  • 10. The immersion cooling system of claim 1, wherein: each of the plurality of ports in the distribution pipe has an internal diameter of 10 mm; andeach hole in the pattern of holes in the distribution plate has an internal diameter of 5 mm.
  • 11. The immersion cooling system of claim 1, wherein the distribution plate is positioned such that there is a 6.5 mm clearance gap between the top of the distribution pipe and the bottom of the distribution plate.
  • 12. The immersion cooling system of claim 1, wherein the primary circuit dielectric cooling fluid comprises oil.
  • 13. The immersion cooling system of claim 12, further comprising: an oil-to-water heat exchanger.
  • 14. The immersion cooling system of claim 1, wherein the at least one computing device comprises twenty-four servers housed in each of at least one rack within the tank.
  • 15. The immersion cooling system of claim 1, wherein the floor of the tank comprises twelve elevated platforms and thirteen alleys.
  • 16. The immersion cooling system of claim 1, further comprising a screen disposed over the transfer port, wherein the screen is configured to prevent stray material from clogging the transfer port.
  • 17. The immersion cooling system of claim 16, wherein the screen is disposed proximate to the first chamber.
RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent Application No. 63/406,558 entitled “System and Method for Supplying Uniform Flow of Dielectric Cooling Fluid for Data Servers” filed on Sep. 14, 2022, the entire contents of which are hereby incorporated by reference for all purposes.

US Referenced Citations (259)
Number Name Date Kind
1606025 Gruendler Nov 1926 A
2440930 Guglielmo et al. May 1948 A
2525457 Paluev Oct 1950 A
2643282 Greene Jun 1953 A
3094133 Treanor Jun 1963 A
3320762 Murdoch May 1967 A
3406244 Sevgin Oct 1968 A
3450265 Kreusch et al. Jun 1969 A
3489207 Miller Jan 1970 A
3600636 Petersen Aug 1971 A
3754741 Whitehurst et al. Aug 1973 A
3858090 Lehmann Dec 1974 A
4034335 Harazoe et al. Jul 1977 A
4245668 Lindstrom Jan 1981 A
4302793 Rohner Nov 1981 A
4313310 Kobayashi et al. Feb 1982 A
4372632 Mlla et al. Feb 1983 A
4399501 Masselin Aug 1983 A
4460008 O'Leary et al. Jul 1984 A
4464315 O'Leary Aug 1984 A
4493010 Morrison et al. Jan 1985 A
4590538 Cray, Jr. May 1986 A
4600230 Ise Jul 1986 A
4648043 O'Leary Mar 1987 A
4659459 O'Leary et al. Apr 1987 A
4704658 Yokouchi et al. Nov 1987 A
4722577 Mewborne Feb 1988 A
4741385 Bergles et al. May 1988 A
4834257 Book et al. May 1989 A
5102503 Silinski et al. Apr 1992 A
5145585 Coke Sep 1992 A
5260850 Sherwood et al. Nov 1993 A
5294916 Bolton et al. Mar 1994 A
5297621 Taraci et al. Mar 1994 A
5329418 Tanabe Jul 1994 A
5332494 Eden et al. Jul 1994 A
5414591 Kimura et al. May 1995 A
5574627 Porter Nov 1996 A
5655459 O'Connor et al. Aug 1997 A
5851143 Hamid Dec 1998 A
5907473 Przilas et al. May 1999 A
6019167 Bishop et al. Feb 2000 A
6374627 Schumacher et al. Apr 2002 B1
D477617 Resch Jul 2003 S
6600656 Mori et al. Jul 2003 B1
6616851 Sforza-Heinen et al. Sep 2003 B1
6621707 Ishimine et al. Sep 2003 B2
D483045 Resch Dec 2003 S
6909606 Barsun et al. Jun 2005 B2
7086247 Campbell et al. Aug 2006 B2
D530346 Resch Oct 2006 S
7184269 Campbell et al. Feb 2007 B2
7210304 Nagashima et al. May 2007 B2
7278273 Whitted et al. Oct 2007 B1
7307841 Berlin et al. Dec 2007 B2
7318322 Ota et al. Jan 2008 B2
7403392 Attlesey et al. Jul 2008 B2
7413394 Risser Aug 2008 B2
7511959 Belady et al. Mar 2009 B2
7511960 Hillis et al. Mar 2009 B2
7551971 Hillis Jun 2009 B2
7604027 Thatcher Oct 2009 B2
7609518 Hopton et al. Oct 2009 B2
7686175 Waisanen Mar 2010 B2
7724513 Coglitore et al. May 2010 B2
7724517 Attlesey et al. May 2010 B2
7757506 Ellsworth, Jr. et al. Jul 2010 B2
7843298 Hosokawa et al. Nov 2010 B2
7854652 Yates et al. Dec 2010 B2
7856838 Hillis et al. Dec 2010 B2
7905106 Attlesey Mar 2011 B2
7911782 Attlesey et al. Mar 2011 B2
7911793 Attlesey Mar 2011 B2
7934386 Rummel et al. May 2011 B2
7961463 Belady et al. Jun 2011 B2
7971632 Eriksen Jul 2011 B2
7983040 Campbell et al. Jul 2011 B2
8009419 Attlesey et al. Aug 2011 B2
8047904 Yates et al. Nov 2011 B2
8213261 Imhof et al. Jul 2012 B2
8295047 Hamburgen et al. Oct 2012 B1
8310829 Monk et al. Nov 2012 B2
8654529 Attlesey Feb 2014 B2
8817465 Campbell et al. Aug 2014 B2
8964391 Campbell et al. Feb 2015 B2
9042098 Campbell et al. May 2015 B2
9049800 Shelnutt et al. Jun 2015 B2
9086859 Attlesey Jul 2015 B2
D744996 Keisling et al. Dec 2015 S
9269544 Koo et al. Feb 2016 B2
9386727 Barringer et al. Jul 2016 B2
9439327 Sheng et al. Sep 2016 B1
9504190 Best Nov 2016 B2
9664180 Saavedra May 2017 B2
9699938 Shelnutt et al. Jul 2017 B2
D796654 Baker et al. Sep 2017 S
9756766 Best Sep 2017 B2
9801465 Finch, Jr. Oct 2017 B1
9861194 Park Jan 2018 B1
D821326 Kreiner Jun 2018 S
9992914 Best et al. Jun 2018 B2
10123463 Best et al. Nov 2018 B2
10212849 Matsumoto et al. Feb 2019 B2
10306804 Chester et al. May 2019 B2
D852936 Baker et al. Jul 2019 S
10342164 So et al. Jul 2019 B2
10405457 Boyd et al. Sep 2019 B2
D875206 Gourgel et al. Feb 2020 S
10667427 Gao May 2020 B2
10743438 Wakino et al. Aug 2020 B2
10820446 Boyd et al. Oct 2020 B2
10939581 Chen et al. Mar 2021 B1
11000943 Reynolds May 2021 B2
11026344 Midgley Jun 2021 B2
11359865 Stone et al. Jun 2022 B2
11369041 Yang et al. Jun 2022 B2
11457735 D'Lsep Oct 2022 B1
11516943 Bilan Nov 2022 B2
11856727 Montes Monteserin Dec 2023 B2
11925946 Bean, Jr. Mar 2024 B2
20020014460 McKay Feb 2002 A1
20020080575 Nam et al. Jun 2002 A1
20020151799 Pantages et al. Oct 2002 A1
20020185262 Baer Dec 2002 A1
20020189173 Staschik Dec 2002 A1
20030053293 Beitelmal et al. Mar 2003 A1
20030127240 Beckbissinger et al. Jul 2003 A1
20040008490 Cheon Jan 2004 A1
20040013563 Romer et al. Jan 2004 A1
20040050491 Miya et al. Mar 2004 A1
20040223300 Fink et al. Nov 2004 A1
20040246683 Honsberg-Riedl et al. Dec 2004 A1
20040254682 Kast Dec 2004 A1
20050011839 Dart et al. Jan 2005 A1
20050024826 Bash et al. Feb 2005 A1
20050052847 Hamman Mar 2005 A1
20050083657 Hamman Apr 2005 A1
20050111184 Cliff et al. May 2005 A1
20050114876 Atarashi et al. May 2005 A1
20050152112 Holmes et al. Jul 2005 A1
20050259402 Yasui et al. Nov 2005 A1
20060026610 Sasao et al. Feb 2006 A1
20060064709 Throckmorton et al. Mar 2006 A1
20060123436 Tanaka et al. Jun 2006 A1
20060135042 Frost et al. Jun 2006 A1
20060250755 Tilton et al. Nov 2006 A1
20060274501 Miller Dec 2006 A1
20070006599 Kawamura et al. Jan 2007 A1
20070025081 Berlin et al. Feb 2007 A1
20070029011 Thatcher Feb 2007 A1
20070034360 Hall Feb 2007 A1
20070199340 Knight et al. Aug 2007 A1
20070213000 Day Sep 2007 A1
20070227710 Belady et al. Oct 2007 A1
20070267741 Attlesey et al. Nov 2007 A1
20080002364 Campbell et al. Jan 2008 A1
20080017355 Attlesey et al. Jan 2008 A1
20080026509 Campbell et al. Jan 2008 A1
20080029250 Carlson et al. Feb 2008 A1
20080030945 Mojaver et al. Feb 2008 A1
20080055845 Murakami et al. Mar 2008 A1
20080158818 Clidaras et al. Jul 2008 A1
20080174954 Vangilder et al. Jul 2008 A1
20080180908 Wexler Jul 2008 A1
20080196868 Attlesey et al. Aug 2008 A1
20080196870 Attlesey et al. Aug 2008 A1
20080209931 Stevens Sep 2008 A1
20080266726 Murakami et al. Oct 2008 A1
20080270572 Belady et al. Oct 2008 A1
20080273306 Campbell et al. Nov 2008 A1
20090168345 Martini Jul 2009 A1
20090219681 Goth et al. Sep 2009 A1
20090229194 Armillas Sep 2009 A1
20090251860 Belady et al. Oct 2009 A1
20090260777 Attlesey Oct 2009 A1
20090262455 Merrow Oct 2009 A1
20090295167 Clidaras et al. Dec 2009 A1
20100027212 Daunert Armillas Feb 2010 A1
20100030267 Winslow et al. Feb 2010 A1
20100033070 Cheng Feb 2010 A1
20100061057 Dersch et al. Mar 2010 A1
20100103618 Campbell et al. Apr 2010 A1
20100139887 Slessman Jun 2010 A1
20100165565 Hellriegal et al. Jul 2010 A1
20100226094 Attlesey et al. Sep 2010 A1
20100246118 Attlesey Sep 2010 A1
20100263885 Tuma Oct 2010 A1
20100275441 Rasmussen et al. Nov 2010 A1
20100290190 Chester et al. Nov 2010 A1
20100302678 Merrow Dec 2010 A1
20110075353 Attlesey et al. Mar 2011 A1
20110103019 Campbell et al. May 2011 A1
20110120885 Miller et al. May 2011 A1
20110132579 Best et al. Jun 2011 A1
20110134604 Attlesey Jun 2011 A1
20110151765 Chen et al. Jun 2011 A1
20110157829 Wormsbecher et al. Jun 2011 A1
20110183051 Skender Jul 2011 A1
20110240281 Avery Oct 2011 A1
20110267775 Vanderveen et al. Nov 2011 A1
20120026691 Campbell et al. Feb 2012 A1
20120035773 Stabinski et al. Feb 2012 A1
20120123595 Bower, III et al. May 2012 A1
20120155027 Broome et al. Jun 2012 A1
20120171943 Dunnavant Jul 2012 A1
20120236487 Wallace et al. Sep 2012 A1
20120294737 Singh et al. Nov 2012 A1
20120300391 Keisling et al. Nov 2012 A1
20130025888 Eckholm et al. Jan 2013 A1
20130032217 Pesek et al. Feb 2013 A1
20130075066 Reytblat Mar 2013 A1
20130105120 Campbell et al. May 2013 A1
20130105139 Campbell et al. May 2013 A1
20130146273 Chester et al. Jun 2013 A1
20140060773 Wajima Mar 2014 A1
20140060799 Eckberg et al. Mar 2014 A1
20140085817 Campbell et al. Mar 2014 A1
20140085821 Regimbal et al. Mar 2014 A1
20140109610 Wulf et al. Apr 2014 A1
20140211412 Best Jul 2014 A1
20140216686 Shelnutt et al. Aug 2014 A1
20140218858 Shelnutt et al. Aug 2014 A1
20140301037 Best Oct 2014 A1
20140307384 Best Oct 2014 A1
20140362527 Best Dec 2014 A1
20150048950 Zeighami et al. Feb 2015 A1
20150062806 Shelnutt et al. Mar 2015 A1
20150138723 Shedd et al. May 2015 A1
20150181762 Boyd et al. Jun 2015 A1
20150195953 Best et al. Jul 2015 A1
20150276292 Son et al. Oct 2015 A1
20150305209 Inaba et al. Oct 2015 A1
20150334880 Best Nov 2015 A1
20150359145 Zhang et al. Dec 2015 A1
20160234970 Shelnutt et al. Aug 2016 A1
20170142868 Chen et al. May 2017 A1
20170265328 Sasaki et al. Sep 2017 A1
20170303443 Inano et al. Oct 2017 A1
20170332514 Saito Nov 2017 A1
20170359918 Klaba et al. Dec 2017 A1
20180020571 Saito Jan 2018 A1
20180020572 Fujiwara et al. Jan 2018 A1
20180288906 Hopton et al. Oct 2018 A1
20180368281 Wang et al. Dec 2018 A1
20190090383 Tufty et al. Mar 2019 A1
20190200482 Boyd et al. Jun 2019 A1
20190329973 Chabot Oct 2019 A1
20190344424 Reynolds Nov 2019 A1
20190387641 Aoki et al. Dec 2019 A1
20200093038 Enright et al. Mar 2020 A1
20200323108 Bilan et al. Oct 2020 A1
20210105910 McCarthy Apr 2021 A1
20210368656 Heydari Nov 2021 A1
20210382533 Heydari Dec 2021 A1
20210400834 Shen et al. Dec 2021 A1
20220220976 Perevozchikov et al. Jul 2022 A1
20220248564 Moon Aug 2022 A1
20230052701 Chen et al. Feb 2023 A1
20230269900 Kelly et al. Aug 2023 A1
Foreign Referenced Citations (25)
Number Date Country
2009282170 Nov 2014 AU
101443724 May 2009 CN
109769381 May 2019 CN
109874275 Jun 2019 CN
111240448 Jun 2020 CN
113795118 Dec 2021 CN
114138084 Mar 2022 CN
115407848 Nov 2022 CN
82014500002000 May 2022 GB
2004319628 Nov 2004 JP
2004363308 Dec 2004 JP
2005015171 Jan 2005 JP
2011518395 Jun 2011 JP
2018019038 Feb 2018 JP
2019161201 Sep 2019 JP
2021519978 Aug 2021 JP
20150018884 Feb 2015 KR
2007023130 Mar 2007 WO
2007098078 Aug 2007 WO
2008027931 Mar 2008 WO
2008089322 Jul 2008 WO
2010019517 Feb 2010 WO
2013022805 Feb 2013 WO
2019061721 Apr 2019 WO
2020234600 Nov 2020 WO
Non-Patent Literature Citations (80)
Entry
Office Action for Chinese Application No. 200980131707.3, mailed Apr. 18, 2014, 7 pages.
Office Action for Chinese Application No. 200980131707.3, mailed Dec. 31, 2014, 10 pages.
Office Action for Chinese Application No. 200980131707.3, mailed Jul. 31, 2013, 8 pages.
Office Action for Chinese Application No. 201280049039.1, mailed Oct. 27, 2015, 33 pages.
Office Action from U.S. Appl. No. 13/057,881, dated Apr. 11, 2014, 16 pages.
Office Action from U.S. Appl. No. 13/057,881, dated Jan. 9, 2015, 3 pages.
Office Action from U.S. Appl. No. 13/057,881, dated Jun. 4, 2013, 13 pages.
Office Action from U.S. Appl. No. 13/057,881, dated Mar. 3, 2015, 20 pages.
Park J, “New Cooling Strategies for Greater Data Center Energy Efficiency,” Facebook, Nov. 4, 2010, 4 pages.
Patent Examination Report No. 2 for Australian Application No. 2009282170, mailed Jun. 18, 2014, 4 pages.
Patent Examination Report No. 2 for Australian Application No. 2009282170, mailed Nov. 15, 2013, 4 pages.
Substantive Examination Adverse Report for Malaysian Application No. PI2011000494, mailed May 15, 2015, 3 pages.
Supplementary Examination Report for Singapore Application No. 11202000306X, mailed Nov. 18, 2022, 4 pages.
“The Green500 List—Nov. 2013,” The Green 500, Nov. 20, 2013, 3 pages.
U.S. Appl. No. 13/057,881, filed Feb. 7, 2011, Christiaan Scott Best, 72 pages.
U.S. Appl. No. 14/338,013, filed Jul. 22, 2014, Christiaan Scott Best, 60 pages.
U.S. Appl. No. 14/338,020, filed Jul. 22, 2014, Christiaan Scott Best, 60 pages.
U.S. Appl. No. 14/338,026, filed Jul. 22, 2014, Christiaan Scott Best, 60 pages.
U.S. Appl. No. 14/338,035, filed Jul. 22, 2014, Christiaan Scott Best, 60 pages.
U.S. Appl. No. 14/667,091, filed Mar. 24, 2015, Christiaan Scott Best, 65 pages.
Written Opinion and Search Report for Singapore Application No. 201100595-6, mailed Apr. 16, 2012, 20 pages.
Written Opinion and Search Report for Singapore Application No. 201100595-6, mailed May 2, 2012, 21 pages.
Applicant-Initiated Interview Summary from U.S. Appl. No. 13/057,881, dated Feb. 25, 2014, 4 pages.
Applicant-Initiated Interview Summary from U.S. Appl. No. 13/057,881, dated Jul. 1, 2014, 3 pages.
Applicant-Initiated Interview Summary from U.S. Appl. No. 13/057,881, dated Oct. 9, 2014, 3 pages.
Bakker A., “Modeling Flow Fields in Stirred Tanks,” Reacting Flows—Lecture 7, 2006, 40 pages.
Communication pursuant to Article 94(3) EPC for European Patent Application No. 09807136.8, dated Nov. 26, 2018, 13 pages.
“Disruptive Technologies,” SC|09, Disruptive Technologies Presentations, Retrieved from http://sc09.supercomputing.org/index-pg=disrupttech.html, Nov. 14-20, 2009, 2 pages.
“Disruptive Technologies,” SC10 Disruptive Technologies, Retrieved from http://sc10.supercomputing.org/index-pg=disrupttech.html, Nov. 13-19, 2010, 8 pages.
Extended European Search Report for European Application No. 09807136.8, mailed Jun. 6, 2017, 12 Pages.
Extended European Search Report for European Application No. 12821634.8, mailed Jul. 15, 2015, 6 pages.
Final Office Action from U.S. Appl. No. 13/057,881, dated Jun. 16, 2015, 30 pages.
Final Office Action from U.S. Appl. No. 13/057,881, dated Nov. 22, 2013, 13 pages.
Final Office Action from U.S. Appl. No. 13/057,881, dated Nov. 3, 2014, 23 pages.
“GRC ICE Immersion Cooling Solutions Brochure,” Retrieved from https://www.grcooling.com/wp-content/uploads/2018/06/GRC_ICE_Product_Brochure.pdf, Jun. 2018, 4 pages.
“GRC ICEraQ® Series 10 Launch Event Presentation,” Retrieved from https://digitalinfranetwork.com/videos/655447036-2/, Dec. 27, 2021, 1 page.
“GRC® Releases the Next Generation of Data Center Liquid Immersion Cooling, Providing Increased Sustainability Capable of Delivering a PUE Yield of 1.03 and a Significant Reduction in Operating Costs,” Retrieved https://www.businesswire.com/news/home/20210518005857/en/GRC%C2%AE-Releases-the-Next-Generation-of-Data-Center-Liquid-Immersion-Cooling-Providing-Increased-Sustainability-Capable-of-Delivering-a-PUE-Yield-of-1.03-and-a-Significant-Reduction-in-Operating-Costs, May 18, 2021, 3 pages.
“Green Revolution Cooling Named a Disruptive Technology of the Year at SC'10,” Press Release, Nov. 1, 2010, 1 page.
“Green Revolution Cooling Named Disruptive at SC'09,” Press Release, Aug. 3, 2009, 1 page.
Hernandez P., “Are Liquid-Cooled Servers Coming to a Data Center Near You?,” CleanTech, Dec. 4, 2009, 2 pages.
Hernandez P., “Will IT Managers Warm to Liquid-Cooled Servers?,” Green Revolution Cooling, Dec. 8, 2009, 3 pages.
“ICEraQ 10 Quad Duo-Specification Sheet,” Retrieved from https://www.grcooling.com/wp-content/uploads/2018/06/GRC_Data_Sheet_ICEraQ_Quad-Duo-One.pdf, 2021, 2 pages.
Intel, “Intel® Core™ 2 Duo Processor on 65 nm Process for Embedded Applications,” Thermal Design Guide, Aug. 2007, pp. 1, 14, and 21.
Intel, “Intel® Pentium® 4 Processor on 90 nm Process,” Datasheet, Feb. 2005, pp. 64-65.
“Intel Xeon Processor E7-4800/8800 v3 Product Families”, May 2015, p. 46 (1 page).
International Preliminary Report on Patentability, Application No. PCT/US2014/033081, mailed Oct. 15, 2015, 7 pages.
International Preliminary Report on Patentability, Application No. PCT/US2014/037005, mailed Nov. 19, 2015, 12 pages.
International Preliminary Report on Patentability for International Application No. PCT/US2009/053305, mailed Feb. 24, 2011, 19 pages.
International Preliminary Report on Patentability for International Application No. PCT/US2015/030618, mailed Nov. 24, 2016, 10 pages.
International Search Report and Written Opinion for International Application No. PCT/US2009/053305, mailed Oct. 14, 2009, 17 pages.
International Search Report and Written Opinion for International Application No. PCT/US20115/030618, mailed Aug. 5, 2015, 12 pages.
International Search Report and Written Opinion for International Application No. PCT/US2012/049668, mailed Oct. 19, 2012, 10 pages.
International Search Report and Written Opinion for International Application No. PCT/US2014/017835, mailed Jun. 13, 2014, 9 pages.
International Search Report and Written Opinion for International Application No. PCT/US2014/033081, mailed Aug. 27, 2014, 9 pages.
International Search Report and Written Opinion for International Application No. PCT/US2014/033834, mailed Sep. 15, 2014, 16 pages.
International Search Report and Written Opinion for International Application No. PCT/US2014/037005, mailed Sep. 24, 2014, 18 pages.
International Search Report and Written Opinion for International Application No. PCT/US2015/43468, mailed Oct. 30, 2015, 11 pages.
International Search Report and Written Opinion for International Application No. PCT/US2021/058991, dated Mar. 3, 2022, 10 pages.
International Search Report and Written Opinion for International Application No. PCT/US2022/043817, mailed Jan. 10, 2023, 10 pages.
International Search Report and Written Opinion for International Application No. PCT/US2023/016342, mailed Jul. 13, 2023, 8 pages.
Lee S.Y., “Hydraulics and Mixing Evaluations for NT-21/41 Tanks,” Revision 0, Oct. 2014, 32 pages.
Markoff J., “Data Centers' Power Use Less Than Was Expected,” The New York Times, Jul. 31, 2011, 3 pages.
Miller R, “Data Center Used to Heat Swimming Pool,” Data Center Knowledge, Retrieved from https://www.datacenterknowledge.com/archives/2008/04/02/data-center-used-to-heat-swimming-pool, Apr. 2, 2008, 2 pages.
Non-Final Office Action in U.S. Appl. No. 14/667,091, mailed May 22, 2015, 30 pages.
Non-final Office Action, U.S. Appl. No. 14/237,100, mailed Sep. 30, 2015, 21 pages.
Non-final Office Action, U.S. Appl. No. 14/245,978, mailed Jan. 13, 2016, 11 pages.
Non-final Office Action, U.S. Appl. No. 14/251,014, mailed Dec. 4, 2015, 13 pages.
Non-final Office Action, U.S. Appl. No. 14/271,386, mailed Dec. 17, 2015, 17 pages.
Notice of Acceptance for Australian Application No. 2009282170, mailed Nov. 14, 2014, 2 pages.
Office Action and Search Report for Chinese Application No. 200980131707.3, mailed Dec. 20, 2012, 17 pages.
Office Action for Canadian Application No. 2731994, mailed Dec. 3, 2015, 6 pages.
Office Action for Canadian Application No. 2731994, mailed Nov. 8, 2016, 4 pages.
International Search Report and Written Opinion in PCT/US2023/032438, mailed Dec. 27, 2023, 9 pages.
International Search Report and Written Opinion for International Application No. PCT/US2023/026959, mailed Oct. 23, 2023, 8 pages.
Office Action for Chinese Application No. 200980131707.3, mailed Apr. 3, 2013, 3 pages.
Office Action for Chinese Application No. 200980131707.3, mailed Dec. 13, 2013, 4 pages.
International Preliminary Report on Patentability in corresponding Application No. PCT/US2022/043817, mailed Mar. 28, 2024 (6 pages).
International Search Report and Written Opinion in corresponding Application No. PCT/US2024/010084, mailed May 1, 2024 (11 pages).
Office Action from U.S. Appl. No. 17/523,403, dated Nov. 10, 2021, 26 pages.
Non-Final Office Action in U.S. Appl. No. 17/900,541, mailed Jun. 17, 2024, 8 pages.
Related Publications (1)
Number Date Country
20240090169 A1 Mar 2024 US
Provisional Applications (1)
Number Date Country
63406558 Sep 2022 US