Patent Grant
9273906
The power dissipation of integrated circuit chips, and the modules containing the chips, continues to increase in order to achieve increases in processor performance. This trend poses cooling challenges at the module, system, rack and data center levels.
In many large server applications, processors along with their associated electronics (e.g., memory, disk drives, power supplies, etc.) are packaged in removable drawer configurations stacked within an electronics rack or frame comprising information technology (IT) equipment. In other cases, the electronics may be in fixed locations within the rack or frame. Conventionally, the components have been cooled by air moving in parallel airflow paths, usually front-to-back, impelled by one or more air moving devices (e.g., fans or blowers). In some cases it has been possible to handle increased power dissipation within a single drawer or system by providing greater airflow, for example, through the use of more powerful air moving devices or by increasing the rotational speed (i.e., RPMs) of existing air moving devices. However, this approach is becoming problematic, particularly in the context of a computer center installation (i.e., data center).
The sensible heat load carried by the air exiting the rack(s) is stressing the capability of the room air-conditioning to effectively handle the load. This is especially true for large installations with “server farms” or large banks of computer racks located close together. In such installations, liquid-cooling is an attractive technology to manage the higher heat fluxes. The liquid absorbs the heat dissipated by the components/modules in an efficient manner. Typically, the heat is ultimately transferred from the liquid coolant to a heat sink, whether air or other liquid.
The shortcomings of the prior art are overcome and additional advantages are provided through the provision, in one aspect, of an apparatus comprising a modular pumping unit configured to facilitate pumping coolant through a cooling apparatus assisting in removal of heat generated by one or more electronic systems. The cooling apparatus includes at least one heat exchange assembly discharging heat from coolant passing through the cooling apparatus. The modular pumping unit is a field-replaceable unit which couples to the cooling apparatus in parallel fluid communication with one or more other modular pumping units. The modular pumping unit includes: a housing; a coolant inlet to the housing; a coolant reservoir tank disposed within the housing and in fluid communication with the coolant inlet; a coolant pump disposed within the housing and configured to pump coolant from the coolant reservoir tank; and a coolant outlet of the housing, wherein the coolant pump is coupled in fluid communication between the coolant reservoir tank and the coolant outlet, and the coolant inlet and outlet facilitate coupling of the modular pumping unit in fluid communication with the cooling apparatus. The apparatus further includes a controller associated with the modular pumping unit, which automatically adjusts operation of the coolant pump based, at least in part, upon one or more sensed parameters.
In another aspect, a cooling system is provided which includes a coolant-based cooling apparatus, multiple modular pumping units, and at least one modular pumping unit controller. The coolant-based cooling apparatus is configured to assist in removal of heat generated by one or more electronic systems, and includes at least one heat exchange assembly discharging heat from coolant of the coolant-based cooling apparatus. The multiple modular pumping units facilitate pumping of coolant through the coolant-based cooling apparatus to assist in removal of heat generated by the one or more electronic systems, and discharge of the heat via the at least one heat exchange assembly. One modular pumping unit of the multiple modular pumping units is a field-replaceable unit, and wherein the multiple modular pumping units are coupled to the cooling apparatus in parallel fluid communication to separately provide pumping of coolant through the cooling apparatus. The one modular pumping unit includes: a housing; a coolant inlet to the housing; a coolant reservoir tank disposed within the housing and in fluid communication with the coolant inlet; a coolant pump disposed within the housing and configured to pump coolant from the coolant reservoir tank; and a coolant outlet of the housing, the coolant pump being coupled in fluid communication between the coolant reservoir tank and the coolant outlet. The coolant inlet and the coolant outlet facilitate coupling of the one modular pumping unit in fluid communication with the cooling apparatus. A modular pumping unit controller is associated with the one modular pumping unit, and automatically adjusts operation of the coolant pump within the one modular pumping unit based, at least in part, on one or more sensed parameters.
In a further aspect, a method is provided which includes: providing a modular pumping unit configured to facilitate pumping of coolant through a cooling apparatus assisting in removal of heat generated by one or more electronic systems, the cooling apparatus comprising at least one heat exchange assembly discharging heat from the coolant passing through the cooling apparatus; and the modular pumping unit being a field-replaceable unit which couples to the cooling apparatus in parallel fluid communication with one or more other modular pumping units, the modular pumping unit comprising: a housing; a coolant inlet to the housing; a coolant reservoir tank disposed within the housing and in fluid communication with the coolant inlet; a coolant pump disposed within the housing and configured to pump coolant from the coolant reservoir tank; and a coolant outlet of the housing, the coolant pump being coupled in fluid communication between the coolant reservoir tank and the coolant outlet, wherein the coolant inlet and the coolant outlet facilitate coupling of the modular pumping unit in fluid communication with the cooling apparatus; and providing a modular pumping unit controller associated with the modular pumping unit, the modular pumping unit controller automatically adjusting operation of the coolant pump based, at least in part, upon one or more sensed parameters.
Additional features and advantages are realized through the techniques of the present invention. Other embodiments and aspects of the invention are described in detail herein and are considered a part of the claimed invention.
One or more aspects of the present invention are particularly pointed out and distinctly claimed as examples in the claims at the conclusion of the specification. The foregoing and other objects, features, and advantages of the invention are apparent from the following detailed description taken in conjunction with the accompanying drawings in which:
As used herein, the terms “electronics rack”, and “rack unit” are used interchangeably, and unless otherwise specified include any housing, frame, rack, compartment, blade server system, etc., having one or more heat-generating components of a computer system, electronic system, or information technology equipment, and may be, for example, a stand alone computer processor having high-, mid- or low-end processing capability. In one embodiment, an electronics rack may comprise a portion of an electronic system, a single electronic system, or multiple electronic systems, for example, in one or more sub-housings, blades, drawers, nodes, compartments, boards, etc., having one or more heat-generating electronic components disposed therein or thereon. An electronic system may be movable or fixed, for example, relative to an electronics rack, with rack-mounted electronic drawers of a rack unit and blades of a blade center system being two examples of electronic systems (or subsystems) of an electronics rack to be cooled. In one embodiment, an electronic system may comprise multiple different types of electronic components, and may be, in one example, a server unit.
“Electronic component” refers to any heat generating electronic component of, for example, an electronic system requiring cooling. By way of example, an electronic component may comprise one or more integrated circuit dies and/or other electronic devices to be cooled, including one or more processor dies, memory dies or memory support dies. As a further example, an electronic component may comprise one or more bare dies or one or more packaged dies disposed on a common carrier. Further, unless otherwise specified herein, the terms “liquid-cooled cold plate” or “liquid-cooled structure” refer to any conventional thermally conductive, heat transfer structure having a plurality of channels or passageways formed therein for flowing of liquid-coolant therethrough.
As used herein, an “air-to-liquid heat exchanger”, “liquid-to-air heat exchanger”, or “coolant-to-air heat exchanger” means any heat exchange mechanism characterized as described herein, across which air passes and through which liquid coolant can circulate; and includes, one or more discrete heat exchangers, coupled either in series or in parallel. An air-to-liquid heat exchanger may comprise, for example, one or more coolant flow paths, formed of thermally conductive tubing (such as copper or other tubing) thermally coupled to a plurality of fins across which air passes. Size, configuration and construction of the air-to-liquid heat exchanger can vary without departing from the scope of the invention disclosed herein. A “liquid-to-liquid heat exchanger” may comprise, for example, two or more coolant flow paths, formed of thermally conductive tubing (such as copper or other tubing) in thermal or mechanical contact with each other. Size, configuration and construction of the liquid-to-liquid heat exchanger can vary without departing from the scope of the invention disclosed herein. Further, “data center” refers to a computer installation containing one or more electronics racks to be cooled. As a specific example, a data center may include one or more rows of rack-mounted computing units, such as server units.
One example of facility coolant and system coolant is water. However, the concepts disclosed herein are readily adapted to use with other types of coolant on the facility side and/or on the system side. For example, one or more of these coolants may comprise a brine, a dielectric liquid, a fluorocarbon liquid, a liquid metal, or other coolant, or refrigerant, while still maintaining the advantages and unique features of the present invention.
Reference is made below to the drawings (which are not drawn to scale for ease of understanding), wherein the same reference numbers used throughout different figures designate the same or similar components.
As shown in
In addition to MCUs 230, the cooling apparatus includes a system coolant supply manifold 231, a system coolant return manifold 232, and manifold-to-node fluid connect hoses 233 coupling system coolant supply manifold 231 to electronic systems 210 (for example, to cold plates disposed within the systems) and node-to-manifold fluid connect hoses 234 coupling the individual electronic subsystems 210 to system coolant return manifold 232. Each MCU 230 is in fluid communication with system coolant supply manifold 231 via a respective system coolant supply hose 235, and each MCU 230 is in fluid communication with system coolant return manifold 232 via a respective system coolant return hose 236.
Heat load of the electronic systems is transferred from the system coolant to cooler facility coolant within the MCUs 230 provided via facility coolant supply line 240 and facility coolant return line 241 disposed, in the illustrated embodiment, in the space between raised floor 145 and base floor 165.
The illustrated cooling apparatus further includes multiple coolant-carrying tubes connected to and in fluid communication with liquid-cooled cold plates 520. The coolant-carrying tubes comprise sets of coolant-carrying tubes, with each set including (for example) a coolant supply tube 540, a bridge tube 541 and a coolant return tube 542. In this example, each set of tubes provides liquid-coolant to a series-connected pair of cold plates 520 (coupled to a pair of processor modules). Coolant flows into a first cold plate of each pair via the coolant supply tube 540 and from the first cold plate to a second cold plate of the pair via bridge tube or line 541, which may or may not be thermally conductive. From the second cold plate of the pair, coolant is returned through the respective coolant return tube 542.
The cooling apparatus is shown to include one or more modular cooling units (MCUs) 620 disposed, by way of example, in a lower portion of electronics rack 601. Each modular cooling unit 620 may be similar to the modular cooling unit depicted in
In the depicted embodiment, the heat exchange tube sections are fed coolant by coolant inlet plenum 631 and exhaust coolant via coolant outlet plenum 632. Flexible hoses (not shown) may be employed for connecting to hard plumbing disposed near the electronics rack. These hoses would be brought into air-to-liquid heat exchanger 640 adjacent to the hinge axis of the door.
Each heat exchange tube section may comprise at least one of a continuous tube or multiple tubes connected together to form one continuous serpentine cooling channel. In the embodiment shown, each heat exchange tube section is a continuous tube having a first diameter, and each plenum 631, 632, is a tube having a second diameter, wherein the second diameter is greater than the first diameter. The first and second diameters are chosen to ensure adequate supply of coolant flow through the multiple tube sections. In one embodiment, each heat exchange tube section may align to a respective electronics subsystem of the electronics rack.
Although not shown in
As illustrated, coolant flowing through warm-liquid coolant loop 720, after circulating through air-to-coolant heat exchanger 715, flows via coolant supply plenum 730 to one or more electronic systems of electronics rack 700, and in particular, one or more cold plates and/or cold rails 735 associated with the electronic systems, before returning via coolant return manifold 731 to warm-liquid coolant loop 720, and subsequently to a cooling unit 740 disposed (for example) outdoors from the data center. In the embodiment illustrated, cooling unit 740 includes a filter 741 for filtering the circulating coolant, a condenser (or air-to-coolant heat exchanger) 742 for removing heat from the coolant, and a pump 743 for returning the coolant through warm-liquid coolant loop 720 to air-to-coolant heat exchanger 715, and subsequently to the coolant-cooled electronics rack 700. By way of example, hose barb fittings 750 and quick disconnect couplings 755 may be employed to facilitate assembly or disassembly of warm-liquid coolant loop 720.
In one example of the warm coolant-cooling approach of
Typically, the heat exchanger or heat exchange assemblies employed by cooling systems such as described above in connection with
Generally stated, disclosed herein is an apparatus which comprises a modular pumping unit (MPU) configured to couple to and facilitate pumping of coolant through a cooling apparatus assisting in removal of heat generated by one or more electronic systems. The modular pumping unit is a field-replaceable unit which couples to the cooling apparatus in parallel fluid communication with one or more other modular pumping units. In one embodiment, each modular pumping unit includes: a housing; a coolant inlet to the housing; a coolant reservoir tank disposed within the housing and in fluid communication with the coolant inlet; a coolant pump disposed within the housing and configured to pump coolant from the coolant reservoir tank; and a coolant outlet of the housing, the coolant pump being coupled in fluid communication between the coolant reservoir tank and the coolant outlet, wherein the coolant inlet and the coolant outlet facilitate coupling of the modular pumping unit in fluid communication with the cooling apparatus. The apparatus further includes a controller associated with the modular pumping unit. The controller controls the coolant pump of the modular pumping unit, and (in one embodiment) automatically adjusts operation of the coolant pump based, at least in part, upon one or more sensed parameters.
For example, one or more coolant-level sensors may be associated with the coolant reservoir tank to sense coolant level within the coolant reservoir tank, and the controller may automatically adjust operation of the coolant pump based upon the sensed level of coolant within the coolant reservoir tank. Also, the modular pumping unit may include one or more coolant temperature sensors disposed to sense temperature of coolant passing through the housing, wherein the MPU controller automatically adjusts an operational speed of the coolant pump based upon coolant temperature sensed by the at least one coolant temperature sensor. If used with a cooling apparatus comprising a coolant-to-air heat exchanger, the MPU may be disposed so that a portion of the airflow across the coolant-to-air heat exchanger also passes through the MPU, allowing a temperature sensor to be incorporated into the MPU to sense temperature of airflow across the liquid-to-air heat exchanger. This sensed ambient air temperature may be employed to, for example, automatically adjust operation of the pump unit. Further details of such a modular pumping unit are described below in reference to the exemplary embodiment thereof depicted in
More specifically, disclosed herein is a modular pumping unit which comprises a densely integrated, field-replaceable unit, which in one embodiment, provides substantially all functional and sensor needs for pumping and monitoring a liquid coolant used to cool, for example, one or more electronic components (such as one or more processor modules). The modular pumping unit is designed to couple, in parallel with one or more other modular pumping units, to a cooling apparatus comprising a heat exchange assembly, such as one or more of a liquid-to-liquid heat exchanger, a coolant-to-refrigerant heat exchanger, a coolant-to-air heat exchanger, etc., and may be located internal to, for example, an IT rack, or remotely from the one or more electronics racks or electronic systems being cooled by the cooling apparatus. In the embodiments disclosed herein, the apparatus further comprises a modular pumping unit controller, as well as a system-level (or frame-level) controller. The full-functional MPU disclosed herein provides coolant of the proper chemistry, filtering, and monitoring, to a customer's cooling apparatus, which includes the separate heat exchange assembly, and offers the ability of the customer to reject heat from the coolant to (for instance) the data center's water system, or to ambient air, or even to a refrigerant-based circuit, while cooling the same rack's or system's temperature-sensitive components. Redundancy at various levels is readily achieved by connecting in parallel fluid communication two or more such modular pumping units to, for example, coolant supply and coolant return manifolds of the cooling apparatus.
In operation, heat generated within the electronic systems 901 is extracted by coolant flowing through (for example) respective cold plates, and is returned via the coolant return manifold 930 and the active modular pumping unit(s), for example, MPU #1941 (in one example) to the coolant-to-air heat exchanger 920 for rejection of the heat from the coolant to the ambient air passing across the heat exchanger. In this example, only one modular pumping unit need be active at a time, and the MPU redundancy allows for, for example, servicing or replacement of an inactive modular pumping unit from the cooling system, without requiring shut-off of the electronic systems or electronics rack being cooled. By way of specific example, quick connect couplings may be employed, along with appropriately sized and configured hoses to couple, for example, the heat exchanger, cold plates, return manifold, and pumping units. Redundant air-moving devices 921, with appropriate drive cards, may be mounted to direct ambient airflow across the coolant-to-air heat exchanger. These drive cards may be controlled by system-level controller 960, in one embodiment. By way of example, multiple air-moving devices may be running at the same time.
The MPU controllers associated with the respective MPUs may be disposed on or within the respective MPU or, for example, associated with the MPU. In one embodiment, the MPU controllers can turn on/off the respective coolant pumps, as well as adjust speed of the coolant pump. The state of the MPU is relayed by the MPU controller 942, 952 to the system-level controller 960. The system-level controller 960 provides system level control for, at least in part, the cooling system. The system-level controller may be disposed, for example, within the electronics rack 900, or remotely from the electronics rack, for example, at a central data center location. As described below, the system-level controller determines, in one embodiment, when switchover of MPUs is to be made and, for example, determines when an MPU has a defect requiring switchover to a redundant MPU for replacement of the defective MPU.
As noted, although depicted in
The modular pumping unit(s) comprises a recirculation coolant loop which: receives exhausted coolant from the electronics rack into a coolant reservoir tank, pressurizes the coolant via a coolant pump (such as a magnetically coupled pump), passes the pressurized coolant through a check valve, and discharges the coolant back to the electronic systems of the electronics rack via the heat exchange assembly.
The modular pumping unit 1000 further comprises a coolant loop 1001 within the housing through which coolant received via the coolant inlet is re-circulated to the coolant outlet. As illustrated, coolant loop 1001 couples in fluid communication coolant inlet 1011 to a coolant reservoir tank 1015 and couples coolant reservoir tank 1015 via a coolant pump 1016 to coolant outlet 1013. A check valve 1019 is also provided within the coolant loop 1001 to prevent backflow of coolant into the modular pumping unit when the modular pumping unit is off, but coupled in fluid communication with the cooling apparatus. In one example, the coolant pump 1016 comprises a centrifugal pump, and a portion of the coolant pumped from coolant reservoir tank 1015 via the coolant pump 1016 is returned via a coolant return line 1017 through a coolant filter 1018 to the coolant reservoir tank 1015. One or more coolant fill or drain connections 1020, 1021 may be provided at housing 1010 into coolant reservoir tank 1015 to, for example, facilitate filling or draining of coolant or air from the coolant reservoir tank, and thereby facilitate field-replaceability of the modular pumping unit in parallel fluid communication with one or more other modular pumping units, without requiring shutdown of the respective electronic systems or electronics rack being cooled.
Advantageously, modular pumping unit 1000 further comprises multiple sensors, and has associated therewith an MPU controller 1030 for facilitating automated monitoring of coolant passing through the MPU, as well as operation of the MPU itself. In the depicted embodiment, modular pumping unit 1000 comprises, for example, a lower-level coolant reservoir sensor LV1, an upper-level coolant reservoir sensor LV2, an outlet pressure sensor P1, a coolant flow rate sensor F1, multiple coolant temperature sensors T1, T2 & T3, an ambient airflow temperature sensor T4, and a coolant leak sensor LK1. In one embodiment, these sensors are disposed within the MPU and allow the controller to control, for example, operation and/or an operational speed of coolant pump 1016, in order (for example) to provide an appropriate level of cooling to the electronic systems or rack. The MPU controller reads the sensed values and responds to the sensor values, along with providing diagnostic information to the system-level controller (such as described above in connection with
The control process also determines whether both level sensors in the coolant reservoir tank indicate the presence of coolant 1125. If “no”, then processing determines whether the lower-level sensor indicates the presence of coolant 1130, and if “no” again, determines whether the upper-level sensor indicates the presence of coolant 1135. If neither sensor indicates the presence of coolant, then the MPU controller provides a no coolant indication to the system-level controller, and shuts off the MPU's coolant pump 1140. Alternatively, if the upper-level sensor indicates the presence of coolant but not the lower-level sensor, then a bad coolant level signal is provided to the system-level controller, since an invalid sensor state has been identified 1145. If the lower-level sensor indicates the presence of coolant but not the upper-level sensor, then a bad coolant level indication is provided to the system-level controller, indicating that coolant needs to be added to the coolant reservoir tank 1150. If both level sensors indicate the presence of coolant, then a good coolant level indication is provided to the system-level controller 1155.
Additionally, the MPU controller may provide a coolant outlet pressure reading and a coolant flow reading to the system-level controller, for example, for diagnostic purposes 1160. The MPU controller may also determine the temperature of the coolant flowing, for example, to the MPU outlet 1165 (see
Advantageously, the MPU controller may also utilize coolant temperature to adjust the coolant pump's RPMs to, for example, maintain coolant temperature close to a desired value 1180. After this automatic adjustment of the coolant pump, processing may wait time interval t1 1185 before obtaining a new set of sensor readings 1105. In one example, time interval t1 may be 1 second.
Advantageously, disclosed hereinabove (in one implementation) is a modular pumping unit comprising a field-replaceable unit that comprises, for example, a single dense housing containing a multitude of functional and sensor requirements for a liquid-cooled electronic system including (for instance): coolant pumping; a reservoir for slowing down coolant to allow any entrained air to leave the coolant, as well as providing a location to replace any entrained air with coolant during a fill process; level sensing; leak sensing; ambient air temperature sensing; coolant flow rate sensing; pressure sensing; liquid filtering; drain and fill locations that enable draining and filling of the field-replaceable unit with coolant, either connected or disconnected from the full cooling apparatus; and an MPU controller comprising, for example, an MPU drive or control card which may be readily accessed, and thereby replaced by service personnel, if required.
Advantageously, provided herein is an ability to field-service the above-noted modular pumping unit as a single field-replaceable unit, with the functionality thereof being concurrently maintained via at least one other MPU coupled therewith in parallel fluid communication to the cooling apparatus. A single MPU as a field-replaceable unit completes the cooling system, with the addition to several passive components, that is, a heat exchange assembly and one or more cold plates, along with interconnecting hoses. Advantageously, multiple MPUs may be operated in parallel to, for example, increase liquid coolant flow rate to a downstream heat exchanger and cold plates. To provide redundancy, at least one MPU is maintained as a spare MPU. That is, if only one MPU is required, then two MPUs are coupled in parallel fluid communication. If four active MPUs are desired at a time, then five or more MPUs are coupled in parallel fluid communication.
As noted, the heat exchanger (or heat exchange assembly) through which liquid coolant from the MPU is pumped may exchange its heat to, for example, air or other coolant or refrigerant, or other type of liquid. In this manner, a single MPU may be developed and qualified for many different heat sink applications. Further, the location of the heat exchanger may be within the electronics rack being cooled, or remote from the rack or electronic system, perhaps used in common with other electronics racks or systems within a data center. Associated with each MPU is an MPU controller which is used, in part, to read and respond to various MPU sensors, as well as control flow of coolant through the MPU (as described above by way of example only). A system-level controller may also be associated with the multiple parallel-connected MPUs. The system-level controller may read information from the multiple MPUs, and make control decisions to ensure that the cooling system runs uninterrupted, as well as control flows, as described above in one example with reference to
As will be appreciated by one skilled in the art, one or more control aspects of the present invention may be embodied as a system, method or computer program product. Accordingly, one or more control aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system”. Furthermore, one or more control aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.
Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.
Referring to
Program code embodied on a computer readable medium may be transmitted using an appropriate medium, including but not limited to, wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for one or more aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language, such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the “C” programming language, assembler or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).
One or more control aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of one or more control aspects of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition to the above, one or more aspects of the present invention may be provided, offered, deployed, managed, serviced, etc. by a service provider who offers management of customer environments. For instance, the service provider can create, maintain, support, etc. computer code and/or a computer infrastructure that performs one or more control aspects of the present invention for one or more customers. In return, the service provider may receive payment from the customer under a subscription and/or fee agreement, as examples. Additionally or alternatively, the service provider may receive payment from the sale of advertising content to one or more third parties.
In one aspect of the present invention, an application may be deployed for performing one or more control aspects of the present invention. As one example, the deploying of an application comprises providing computer infrastructure operable to perform one or more control aspects of the present invention.
As a further aspect of the present invention, a computing infrastructure may be deployed comprising integrating computer readable code into a computing system, in which the code in combination with the computing system is capable of performing one or more aspects of the present invention.
As yet a further aspect of the present invention, a process for integrating computing infrastructure comprising integrating computer readable code into a computer system may be provided. The computer system comprises a computer readable medium, in which the computer medium comprises one or more aspects of the present invention. The code in combination with the computer system is capable of performing one or more control aspects of the present invention.
Although various embodiments are described above, these are only examples. Further, other types of computing environments can benefit from one or more aspects of the present invention.
As a further example, a data processing system suitable for storing and/or executing program code is usable that includes at least one processor coupled directly or indirectly to memory elements through a system bus. The memory elements include, for instance, local memory employed during actual execution of the program code, bulk storage, and cache memory which provide temporary storage of at least some program code in order to reduce the number of times code must be retrieved from bulk storage during execution.
Input/Output or I/O devices (including, but not limited to, keyboards, displays, pointing devices, DASD, tape, CDs, DVDs, thumb drives and other memory media, etc.) can be coupled to the system either directly or through intervening I/O controllers. Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modems, and Ethernet cards are just a few of the available types of network adapters.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprise” (and any form of comprise, such as “comprises” and “comprising”), “have” (and any form of have, such as “has” and “having”), “include” (and any form of include, such as “includes” and “including”), and “contain” (and any form contain, such as “contains” and “containing”) are open-ended linking verbs. As a result, a method or device that “comprises”, “has”, “includes” or “contains” one or more steps or elements possesses those one or more steps or elements, but is not limited to possessing only those one or more steps or elements. Likewise, a step of a method or an element of a device that “comprises”, “has”, “includes” or “contains” one or more features possesses those one or more features, but is not limited to possessing only those one or more features. Furthermore, a device or structure that is configured in a certain way is configured in at least that way, but may also be configured in ways that are not listed.
The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below, if any, are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the present invention has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the invention. The embodiment was chosen and described in order to best explain the principles of one or more aspects of the invention and the practical application, and to enable others of ordinary skill in the art to understand one or more aspects of the invention for various embodiments with various modifications as are suited to the particular use contemplated.
| Number | Name | Date | Kind |
|---|---|---|---|
| 3807493 | Stewart | Apr 1974 | A |
| 5168921 | Meyer | Dec 1992 | A |
| 5202837 | Coe et al. | Apr 1993 | A |
| 5414591 | Kimura et al. | May 1995 | A |
| 5596507 | Jones et al. | Jan 1997 | A |
| 5794454 | Harris et al. | Aug 1998 | A |
| 6006168 | Schumann et al. | Dec 1999 | A |
| 6011371 | Van Brocklin et al. | Jan 2000 | A |
| 6263959 | Ikeda et al. | Jul 2001 | B1 |
| 6400045 | Hosokawa et al. | Jun 2002 | B1 |
| 6411512 | Mankaruse et al. | Jun 2002 | B1 |
| 6434000 | Pandolfi et al. | Aug 2002 | B1 |
| 6446026 | Dean et al. | Sep 2002 | B1 |
| 6462949 | Parish et al. | Oct 2002 | B1 |
| 6480014 | Li et al. | Nov 2002 | B1 |
| 6557354 | Chu et al. | May 2003 | B1 |
| 6675875 | Vafai et al. | Jan 2004 | B1 |
| 6698218 | Goth et al. | Mar 2004 | B2 |
| 6775137 | Chu et al. | Aug 2004 | B2 |
| 6924981 | Chu et al. | Aug 2005 | B2 |
| 6934118 | Hidaka et al. | Aug 2005 | B2 |
| 6968709 | Goth et al. | Nov 2005 | B2 |
| 7002799 | Malone et al. | Feb 2006 | B2 |
| 7042726 | Cader et al. | May 2006 | B2 |
| 7043933 | Knight | May 2006 | B1 |
| 7103509 | Shah et al. | Sep 2006 | B2 |
| 7151668 | Stathakis | Dec 2006 | B1 |
| 7218129 | Beaman et al. | May 2007 | B2 |
| 7233501 | Ingalz | Jun 2007 | B1 |
| 7273090 | Crocker et al. | Sep 2007 | B2 |
| 7286355 | Cheon | Oct 2007 | B2 |
| 7310737 | Patel et al. | Dec 2007 | B2 |
| 7346468 | Bashor et al. | Mar 2008 | B2 |
| 7365973 | Rasmussen et al. | Apr 2008 | B2 |
| 7365985 | Ni | Apr 2008 | B1 |
| 7385810 | Chu et al. | Jun 2008 | B2 |
| 7388749 | Feroli et al. | Jun 2008 | B1 |
| 7395851 | Lee et al. | Jul 2008 | B2 |
| 7436663 | Matsushima et al. | Oct 2008 | B2 |
| 7440278 | Cheng | Oct 2008 | B2 |
| 7443672 | Peng et al. | Oct 2008 | B2 |
| 7450385 | Campbell et al. | Nov 2008 | B1 |
| 7469551 | Tilton et al. | Dec 2008 | B2 |
| 7573714 | Ali | Aug 2009 | B2 |
| 7583043 | Chung et al. | Sep 2009 | B2 |
| 7595550 | Cady et al. | Sep 2009 | B2 |
| 7626820 | Konshak et al. | Dec 2009 | B1 |
| 7660109 | Iyengar et al. | Feb 2010 | B2 |
| 7684196 | Eckberg et al. | Mar 2010 | B2 |
| 7701714 | Shabany | Apr 2010 | B2 |
| 7703291 | Bushnik et al. | Apr 2010 | B2 |
| 7746634 | Hom et al. | Jun 2010 | B2 |
| 7791882 | Chu et al. | Sep 2010 | B2 |
| 7796393 | Lengen et al. | Sep 2010 | B2 |
| 7796399 | Clayton et al. | Sep 2010 | B2 |
| 7817412 | Sullivan | Oct 2010 | B2 |
| 7826216 | Moss | Nov 2010 | B2 |
| 7830657 | Chu et al. | Nov 2010 | B2 |
| 2907406 | Campbell et al. | Mar 2011 | A1 |
| 7983040 | Campbell et al. | Jul 2011 | B2 |
| 8027162 | Campbell et al. | Sep 2011 | B2 |
| 20040060305 | Singh et al. | Apr 2004 | A1 |
| 20040079100 | Monfarad | Apr 2004 | A1 |
| 20040160741 | Moss et al. | Aug 2004 | A1 |
| 20040221604 | Ota et al. | Nov 2004 | A1 |
| 20050109049 | Chan | May 2005 | A1 |
| 20050122684 | Chu et al. | Jun 2005 | A1 |
| 20050247433 | Corrado et al. | Nov 2005 | A1 |
| 20060126296 | Campbell et al. | Jun 2006 | A1 |
| 20060146497 | Gauche et al. | Jul 2006 | A1 |
| 20060174643 | Ostrom et al. | Aug 2006 | A1 |
| 20060176186 | Larson et al. | Aug 2006 | A1 |
| 20060224254 | Rumi et al. | Oct 2006 | A1 |
| 20060225433 | Jammu et al. | Oct 2006 | A1 |
| 20060250772 | Salmonson et al. | Nov 2006 | A1 |
| 20070044493 | Kearney et al. | Mar 2007 | A1 |
| 20070211430 | Bechtolsheim | Sep 2007 | A1 |
| 20080112075 | Farquhar et al. | May 2008 | A1 |
| 20080239564 | Farquhar et al. | Oct 2008 | A1 |
| 20080245083 | Tutunoglu et al. | Oct 2008 | A1 |
| 20080259566 | Fried | Oct 2008 | A1 |
| 20080310105 | Cheng | Dec 2008 | A1 |
| 20080313492 | Hansen | Dec 2008 | A1 |
| 20090002951 | Legen et al. | Jan 2009 | A1 |
| 20090080151 | Kalms et al. | Mar 2009 | A1 |
| 20090086428 | Campbell et al. | Apr 2009 | A1 |
| 20090120607 | Cheon et al. | May 2009 | A1 |
| 20090126910 | Campbell et al. | May 2009 | A1 |
| 20090177334 | Artman et al. | Jul 2009 | A1 |
| 20090219687 | Lin | Sep 2009 | A1 |
| 20090259347 | Gross et al. | Oct 2009 | A1 |
| 20090268404 | Chu et al. | Oct 2009 | A1 |
| 20090277616 | Cipolla et al. | Nov 2009 | A1 |
| 20090284924 | Konshak et al. | Nov 2009 | A1 |
| 20100025010 | Cipolla et al. | Feb 2010 | A1 |
| 20100027220 | Hughes et al. | Feb 2010 | A1 |
| 20100085712 | Hrehor et al. | Apr 2010 | A1 |
| 20100094593 | Reddy et al. | Apr 2010 | A1 |
| 20100126696 | Novotny et al. | May 2010 | A1 |
| 20100142147 | Chang et al. | Jun 2010 | A1 |
| 20100236772 | Novotny et al. | Sep 2010 | A1 |
| 20100252234 | Campbell et al. | Oct 2010 | A1 |
| 20100263855 | Arimilli et al. | Oct 2010 | A1 |
| 20100274522 | Sri-Jayantha et al. | Oct 2010 | A1 |
| 20100305775 | Bean et al. | Dec 2010 | A1 |
| 20100306598 | Ackaret et al. | Dec 2010 | A1 |
| 20100313590 | Campbell et al. | Dec 2010 | A1 |
| 20110051372 | Barringer et al. | Mar 2011 | A1 |
| 20110057803 | Yamaoka et al. | Mar 2011 | A1 |
| 20110060470 | Campbell et al. | Mar 2011 | A1 |
| 20110069454 | Campbell et al. | Mar 2011 | A1 |
| 20110112694 | Bash et al. | May 2011 | A1 |
| 20110168379 | Morgan et al. | Jul 2011 | A1 |
| 20110197612 | Campbell et al. | Aug 2011 | A1 |
| 20110224947 | Kriss | Sep 2011 | A1 |
| 20110253347 | Harrington | Oct 2011 | A1 |
| 20110277967 | Fried et al. | Nov 2011 | A1 |
| 20110292600 | Campbell et al. | Dec 2011 | A1 |
| 20110315343 | Campbell et al. | Dec 2011 | A1 |
| 20110315344 | Campbell et al. | Dec 2011 | A1 |
| 20110315353 | Campbell et al. | Dec 2011 | A1 |
| 20110315355 | Campbell et al. | Dec 2011 | A1 |
| 20110317367 | Campbell | Dec 2011 | A1 |
| 20120118534 | Goth et al. | May 2012 | A1 |
| 20120120603 | Campbell et al. | May 2012 | A1 |
| Number | Date | Country |
|---|---|---|
| 201222836 | Apr 2009 | CN |
| 19845821 | Apr 2000 | DE |
| 2436669 | May 2011 | GB |
| 01077199 | Mar 1989 | JP |
| 2010040886 | Feb 2010 | JP |
| 323643 | Dec 2007 | TW |
| 200937175 | Sep 2009 | TW |
| WO 9747167 | Dec 1997 | WO |
| WO 0237918 | May 2002 | WO |
| WO 2007140110 | Dec 2007 | WO |
| WO 2010075358 | Jul 2010 | WO |
| Entry |
|---|
| Simons et al., “A Survey of Vapor Phase Cooling Systems”, EDN Magazine, vol. 14, No. 1, pp. 53-56 (Jan. 1969). |
| Crispell et al., “Use of Liquid Metal to Promote Boiling on Immersed Integrated Circuit Chips”, IBM Technical Disclosure Bulletin, IPCOM000058250D, TDB 09-88, pp. 365-366 (Sep. 1, 1988). |
| Delia et al., “System Cooling Design for the Water-Cooled IBM Enterprise System/900 Processors”, IBM Journal of Research and Development, vol. 36, No. 4, pp. 791-803 (Jul. 1992). |
| Mudawar, et al., “Immersion-Cooled Standard Electronic Clamshell Module: A Building Block for Future High-Flux Avionic Systems”, Journal of Electronic Packaging, vol. 116, pp. 116-125 (Jun. 1994). |
| Nelson et al. “Thermal Performance of an Integral Immersion Cooled Multichip Module Package”, IEEE on Components, Packages, and Manufacturing Technology—Part A, vol. 17, No. 3, pp. 405-412 (Sep. 1994). |
| Arent, “Liquid Coolant Distribution Unit with Convertible Air or Water Cooling”, IBM Technical Disclosure Bulletin, IP Prior Art Database, IP.com No. IPCOM000044828D (Feb. 6, 2005). |
| Morgan, “IBM Hints at Triple Redundancy in Power6”, (http://www.itjungle.com/breaking/bn032906-story-1.html) (Mar. 29, 2006). |
| Ellsworth, Jr. et al., “The Evolution of Water Cooling for IBM Larger Server Systems: Back to the Future”, IEEE, Piscataway, NJ, 2008 11th Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITHERM '08), pp. 266-274 (May 2008). |
| IBM Technical Disclosure, “Highly-Efficient Copper Rail Design for Optimized Cooling of DRAM Modules”, IPCOM000184053D (Jun. 9, 2009). |
| Geisler et al., “Passive Immersion Cooling of 3-D Stacked Dies”, IEEE Transactions on Components and Packaging Technologies, vol. 32, No. 3, pp. 557-565 (Sep. 2009). |
| Forrest et al., “Pressure Effects on the Pool of Boiling of the Fluorinated Ketone”, Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm), pp. 1-9 (2010) (No further date information is available). |
| Rubenstein et al., “Hypbrid Cooled Data Center Using Above Ambient Liquid Cooling”, IEEE (2010) (No further date information is available). |
| Chainer et al., “Thermoelectric-Enhanced, Liquid-Based Cooling of a Multi-Component Electronic System”, U.S. Appl. No. 13/102,195, filed May 6, 2011. |
| Eckberg et al., “Multi-Rack, Door-Mounted Heat Exchanger”, U.S. Appl. No. 13/115,404, filed May 25, 2011. |
| Campbell et al., “Multi-Fluid, Two-Phase Cooling of Electronic Component(s)”, U.S. Appl. No. 13/281,945, filed Oct. 26, 2011. |
| Chainer et al., “Directly Connected Heat Exchanger Tube Section and Coolant-Cooled Structure”, U.S. Appl. No. 13/283,933, filed Oct. 28, 2011. |
| Campbell et al., “Multi-Rack Assembly With Shared Cooling Unit”, U.S. Appl. No. 13/285,116, filed Oct. 31, 2011. |
| Chainer et al., “Dynamically Limiting Energy Consumed by Cooling Apparatus”, U.S. Appl. No. 13/305,967, filed Nov. 29, 2011. |
| Number | Date | Country | |
|---|---|---|---|
| 20130333865 A1 | Dec 2013 | US |
