Patent Grant
9591792
This document relates to data center cooling.
Computer users often focus on the speed of computer microprocessors (e.g., megahertz and gigahertz). Many forget that this speed often comes with a cost—higher electrical power consumption. For one or two home PCs, this extra power may be negligible when compared to the cost of running the many other electrical appliances in a home. However, in data center applications, where thousands of microprocessors may be operated, electrical power requirements can be very important. Not only must a data center operator pay for electricity to operate its many computers, but the operator must also pay to cool the computers. For example, a pair of microprocessors mounted on a single motherboard can draw 200-400 watts or more of power. Multiply that figure by several thousand (or tens of thousands) to account for the many computers in a large data center, and one can readily appreciate the amount of heat that can be generated.
The cost of removing all of the heat can also be a major cost of operating large data centers. That cost typically involves the use of even more energy, in the form of electricity and natural gas, to operate chillers, condensers, pumps, fans, cooling towers, and other related components. Heat removal can also be important because, although microprocessors may not be as sensitive to heat as are people, increases in heat generally can cause increases in microprocessor errors and failures.
This specification describes technologies relating to evaporative cooling in data centers. For example, a number of misting nozzles may be placed at an outside air intake to a data center and outdoor air may be circulated past the misting nozzles, so that the air is cooled by the heat of evaporation in water or other liquids that may be evaporated into the airflow.
In an example aspect, a data center cooling system includes a multiple fans configured to circulate outside air at ambient conditions through an entry zone of a data center and multiple atomizers positioned upstream of the entry zone configured to spray atomized water into the circulating outside air. An evaporation zone is provided where the atomized water evaporates and cools the outside air to produce cooled air. A rack zone is positioned downstream of the evaporation zone. The rack zone includes computer racks arranged in substantially parallel rows. The fans are further configured to circulate the cooled air from the evaporation zone into the rack zone.
Implementations may include some, all, or none of the following features, alone or in combination. The cooling system may include a cool air plenum in fluid communication with the evaporation zone and configured to capture the cooled air and direct the cooled air into the rack zone at predetermined discrete locations. The cool air plenum may include one or more conduits positioned above and substantially sealed from the rack zone and can be bordered by one or more diffusers configured to direct the cooled air downwardly into the rack zone. The cool air plenum may include one or more conduits positioned below and substantially sealed from the rack zone and may be bordered by one or more diffusers configured to direct the cool air upwardly into the rack zone.
The system may include a hot air plenum in fluid communication with the rack zone and configured to capture heated air that has been circulated past electronic equipment in the rack zone. The hot air plenum may include one or more conduits positioned above a top level of the rack zone, such that hot air vents upwardly into the hot air plenum. The hot air plenum may include one or more conduits positioned beneath a bottom level of the rack zone. The system can further include one or more fans configured to draw hot air from the rack zone downwardly into the hot air plenum.
The entry zone may be an opening in a wall of the data center and can include louvers extending across the opening. The system may include a filter system positioned inside the entry zone upstream of the fans, to filter outside air entering the entry zone. The system may also include a cool air plenum configured to capture the cooled air and to direct the cooled air into the rack zone. The cool air plenum can include a supply conduit in fluid communication with the evaporation zone, which conduit can extend substantially the width of the rack zone and capture the cooled air from the evaporation zone. Multiple delivery conduits can be connected to the supply conduit, each delivery conduit projecting in a substantially perpendicular direction from the supply conduit and extending substantially the length of the rack zone. The cool air plenum may be positioned above the rack zone and the delivery conduits can be in sealed fluid communication with diffusers configured to direct the cool air downwardly into the rack zone. The cool air plenum may be positioned below a floor that supports equipment in the rack zone, and the delivery conduits can be in sealed fluid communication with diffusers configured to direct the cool air upwardly through perforations in the floor into the rack zone.
The system may include one or more exhaust fans configured to draw into a hot air plenum heated air that has been circulated past electronic equipment in the rack zone. The hot air plenum may be configured to capture the heated air and can include an exhaust conduit that extends substantially the width of the rack zone. The exhaust conduit can be configured to receive the hot air from return conduits and to vent the hot air outside the data center. The return conduits can be connected to the exhaust conduit, and each return conduit can project in a substantially perpendicular direction from the exhaust conduit and extend substantially the length of the rack zone. The return conduits can include one or more apertures, such that the hot air is drawn from the rack zone into the return conduits through the apertures and directed to the exhaust conduit. The delivery conduits of the cool air plenum can be arranged in parallel to the return conduits of the hot air plenum, and alternate with each other across the width of the rack zone.
The entry zone may include an opening in a wall of the data center and a louvers extending across the opening. The fans may be a wall of fans that is substantially the same size as the opening in the wall of the data center. The atomizers may include atomizers spaced at intervals in rows extending across the wall of fans, and/or above and downstream of the fans.
The system may also include a control system. The control system can include; a thermometer configured to measure a wet or dry bulb temperature, or both, of the outside air; a flow meter configured to measure a rate of flow of cooled air into the rack zone; a humidistat configured to measure a humidity level of the cooled air; and a controller. The controller can be configured to receive the measurements from the thermometer, flow meter, and humidistat, and to control the flow rate of water to the atomizers based on the measurements and a temperature and humidity set point for the rack zone.
The system may also include a supplemental cooling system for the data center. The supplemental cooling system can include a cooling tower, a water-to-water heat exchanger in fluid communication with the cooling tower, an air-to-water heat exchanger in fluid communication with the water-to-water heat exchanger. The supplemental cooling system can receive hot air from the rack zone and be arranged to cool the hot air from the rack zone and to recirculate the cooled air back into the rack zone. The controller can be further configured to selectively activate the supplemental cooling system to cool the rack zone, e.g., based on the measurements and on the temperature and relative humidity set point for the rack zone. The supplemental cooling system may also include a chiller in fluid communication with the air-to-water heat exchanger, and the controller may be further configured to selectively activate the chiller based on the measurements and on the temperature and relative humidity set point for the rack zone.
The system may also include a control system including a thermometer configured to measure the temperature of the cool air, a humidistat configured to measure the relative humidity of the cool air, a controller configured to receive the measurements from the thermometer and humidistat and to control the flow rate of water to the plurality of atomizers based on the measurements and a maximum allowable temperature and relative humidity for the rack zone.
In another aspect, a data center cooling system includes fans configured to draw outside air at ambient conditions into an entry zone of a data center, a atomizers positioned downstream of the entry zone configured to spray atomized water into the outside air, and an evaporation zone where the atomized water evaporates and cools the outside air to produce cooled air. A rack zone is positioned downstream of the evaporation zone and includes computer racks arranged in substantially parallel rows. An air-to-air heat exchanger is positioned between the evaporation zone and the rack zone.
Various implementations can include some, all or none of the following features, either alone or in combination. The air-to-air heat exchanger may include one or more heat wheels. A first conduit can be positioned on the evaporation zone-side of the one or more heat wheels and configured to receive the cooled air and direct the cooled air through the one or more heat wheels. A second conduit can be positioned on the rack zone-side of the one or more heat wheels and configured to receive hot air from the rack zone, direct the hot air through the one or more heat wheels and redirect the now cooled air back into the rack zone. A second set of atomizers may be positioned downstream of the heat exchanger and configured to spray atomized water into the cooled air before the cooled air enters the rack zone. A second evaporation zone may be provided where the atomized water from the second set of atomizers evaporates and further cools the air before the air enters the rack zone.
The system may also include a control system that includes: a thermometer configured to measure a temperature of the cooled air redirected back to the rack zone; a flow meter configured to measure a rate of flow of the now cooled air into the rack zone; a humidistat configured to measure a relative humidity of the now cooled air; and a controller. The controller can be configured to receive the measurements from the thermometer, flow meter, and humidistat, and to control the flow rate of water to the atomizers based on the measurements and on a temperature and relative humidity set point for the rack zone. The system may also include a supplemental cooling system for the data center. The supplemental cooling system can include a cooling tower, a water-to-water heat exchanger in fluid communication with the cooling tower, an air-to-water heat exchanger in fluid communication with the water-to-water heat exchanger and positioned to receive hot air from the rack zone. The supplemental cooling system can be activated to cool hot air from the rack zone and to recirculate cooled air back into the rack zone. The controller may be further configured to selectively activate the supplemental cooling system to cool the rack zone, e.g., based on the measurements and on the temperature and relative humidity set point for the rack zone. The supplemental cooling system may also include a chiller in fluid communication with the air-to-water heat exchanger, and the controller may be further configured to selectively activate the chiller, e.g., based on the measurements and on the temperature and relative humidity set point for the rack zone.
In another aspect, a method for cooling a data center includes drawing outside air at ambient conditions into an entry zone of the data center, spraying atomized water into the outside air downstream of the entry zone, providing an evaporation zone where the atomized water evaporates and cools the outside air to cool air, directing the cool air into a rack zone positioned downstream of the evaporation zone that includes computer racks arranged in substantially parallel rows. The rack zone is in fluid communication with the evaporation zone.
Various implementations may include some, all, or none of the following features, either alone or in combination. Directing the cool air into the rack zone may include directing the cooled air into a cool air plenum in fluid communication with the evaporation zone and configured to capture the cooled air and directing the cooled air from the cool air plenum into the rack zone. Hot air from the rack zone may be captured in a hot air plenum in fluid communication with the rack zone and venting the hot air out of the data center. The outside air may be filtered upstream of spraying atomized water into the outside air. A temperature of the outside air may be determined, the rate of flow of cooled air into the rack zone and a humidity level of the cool air, and a flow rate of water to the atomizers may be controlled to generate the spray based on the determinations and on a temperature and humidity set point for the rack zone.
A supplemental cooling system for the data center may be activated based on the determinations and on the temperature and humidity set point for the rack zone. The supplemental cooling system may include a cooling tower, a water-to-water heat exchanger in fluid communication with the cooling tower, an air-to-water heat exchanger in fluid communication with the water-to-water heat exchanger and positioned to receive hot air from the rack zone. The supplemental cooling system may also a chiller in fluid communication with the air-to-water heat exchanger.
The method may also include selectively activating the chiller based on the determinations and on the temperature and humidity set point for the rack zone. The method may also include determining a temperature and a humidity level of the cool air and controlling the flow rate of water to atomizers generating the spray based on the determinations and on a maximum allowable temperature and humidity level for the rack zone. The method may also include selectively activating a supplemental cooling system for the data center based on the determinations and on the maximum allowable temperature and humidity level for the rack zone, wherein the supplemental cooling system may include a cooling tower, a water-to-water heat exchanger in fluid communication with the cooling tower, an air-to-water heat exchanger in fluid communication with the water-to-water heat exchanger and positioned to receive hot air from the rack zone.
In another aspect, outside air at ambient conditions may be drawn into an entry zone of the data center, atomized water may be sprayed into the outside air downstream of the entry zone, an evaporation zone may be provided where the atomized water evaporates and cools the outside air to cool air, the cool air may be directed into an air-to-air heat exchanger that is positioned between the evaporation zone and a rack zone. Hot air may be directed from a rack zone of the data center into the air-to-air heat exchanger, which cools the hot air to cooled air. The cooled air may be recirculated from the heat exchanger back into the rack zone. The rack zone includes computer racks arranged in a substantially parallel rows. The air may be sprayed with atomized water to further cool the air before the cooled air is recirculated back into the rack zone.
The air-to-air heat exchanger may include one or more heat wheels. A first conduit can be positioned on the evaporation zone-side of the one or more heat wheels and configured to receive the cool air and direct the cool air through the one or more heat wheels. A second conduit can be positioned on the rack zone-side of the one or more heat wheels and configured to receive hot air from the rack zone and the direct the hot air through the one or more heat wheels to cool the air. The method may also include determining the temperature, humidity level and/or flow rate of the cooled air being recirculated back to the rack zone and controlling the flow rate of water to the atomizers generating the spray based on the determinations and on a temperature and humidity set point for the rack zone.
The systems and techniques described herein can include the following advantages. Facility cooling can be achieved using less energy thereby conserving resources and reducing costs.
The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features and advantages will be apparent from the description and drawings, and from the claims.
Like reference symbols in the various drawings indicate like elements.
The large number of computers operating in a computer data center can generate a great deal of heat. This heat can be damaging to the computers, and therefore should be removed. Generally, computers are air-cooled wherein relatively cool air is drawn across the computers to absorb the generated heat (though water cooling may be provided in spot applications such as on microprocessors and other high-heat-generating components). In a typical data center, the warmed air is cooled actively (e.g., by air conditioning), but such active cooling can consume a great deal of power in the process.
Methods and systems are described herein that can conserve power while cooling a data center using evaporative cooling. The evaporative cooling can be used, in some implementations, in combination with other cooling systems and techniques, as is described below.
Referring to
In the example implementation shown, a collection of fans 116 are positioned downstream of an opening in an exterior wall of the data center. The opening in the wall is covered by a set of louvers 110, e.g., architectural louvers that may be paired with powered dampers. The fans 116 draw outside air at outdoor ambient conditions into an entry zone 113 of the data center. A collection of atomizers 140, 142 are positioned upstream of the entry zone 113 and are arranged to spray atomized water into the outside air drawn into the data center. An evaporation zone 143 is provided where the atomized water evaporates and, while doing so, cools the outside air to a temperature that is closer to the wet bulb temperature for the air. The rack zone 141 is positioned downstream of the evaporation zone 143 and the cool air is directed into the rack zone 141. In the implementation shown, a filter section 115 is included downstream of the louvers 110 and before the fans 116 to filter the outside air before it is directed any further into the data center.
In the particular implementation shown, the cool air is directed (by the fans 116) into a cool air plenum 109 that includes several conduits 150 as shown in the plan view of the data center in
The cool air becomes heated air after exposure to the rack zone 141, such as by being circulated across motherboards or other components in the racks (e.g., from front sides of each rack to back sides of each rack). Referring to
In some implementations, the hot air plenum 105 may be served by one or more conduits positioned beneath, rather than above, the rack zone 141, wherein one or more fans may be configured to draw hot air from the rack zone 141 downwardly into the hot air plenum 105. Such conduits may be located in a raised floor, basement, or other appropriate space. The hot air may be gathered from the hot air plenum 105 by air handling units that include, for example, one or more fans appropriately sized for the task. Both the cool air plenum and the hot air plenum 105 may be located below the floor, each may be located on opposite sides of the rack zone, with one below the floor and the other above the ceiling, or vice-versa. Where the cool-air plenum is below the floor, outside air may be drawn in from a sub-basement located below the data center 100, and then the cool-air plenum is above the ceiling, outdoor air may be drawn in through the roof of the data center 100.
As such, the cooled outside air is provided to the work space 106, drawn through the racks 102a-102d such as by fans mounted on the many trays that are mounted in racks 102a-102d. This air may be heated as it passes over the trays and through power supplies running the computers on the trays, and may then be exhausted into a number of hot exhaust spaces 104a-104d. Each tray may have its own power supply and fan, with the power supply at the back edge of the tray, and the fan attached to the back of the power supply. All of the fans may be configured or programmed to deliver air at a single common temperature, such as at a set 113° F. (45° C.), so that the individual fans control their respective speeds to maintain an essentially equal temperature rise across the computers throughout the data center 100. The heated air is then routed through the hot exhaust spaces 104a-104d and the chimney plenums 108 to the hot air plenum 105, and exhausted to atmosphere through an exhaust 114, so that the outside air to be supplied to the computer does not mix with, and get diluted by, the warmed air from the computers.
Air is drawn through the louvers 110 and filters 115 by a collection of fans 116. In some implementations, the collection of fans 116 may cover an area that is substantially the same size as the opening in the outside wall of the data center 100. The fans may be propeller fans arranged in a grid where a large pressure rise is not needed, and high volumetric flow rate is more important. Other types of fans (e.g., centrifugal and in-line or axial flow) may also be used.
A number of water supply lines 117 carry water to a collection of water misters, or atomizers 140-142 that are spaced across the wall of fans 116. The atomizers 140 are located downstream from the fans 116 to spray a fine mist of water into the flow of air when the outside air is determined to be sufficiently warm and/or dry enough to evaporate the atomized water (when there is a sufficient distance between its wet bulb and dry bulb temperatures). The atomizers 142 are located above the flow of air, such that gravity will pull the mist into the flow of air below. As the atomized water evaporates, heat energy is drawn out of the air. The flow of water to the water supply lines 117 is controlled by a collection of atomizer flow controllers 119. Various examples of processes for controlling the atomizer flow controllers 119 and the evaporative cooling system 112 in general, will be discussed in the descriptions of
In some implementations, fresh air may be provided to the work space 106 by various other mechanisms. For example, a supplemental air-conditioning unit, such as a standard roof-top unit may be provided to supply necessary exchanges of outside air. Also, such a unit may serve to dehumidify the work space 106 for the limited latent loads in the data center 100, such as human perspiration (though direct evaporative cooling would not be used during such a time). In some implementations, the cool air may be provided to the work space 106 through plenums or other types of conduits that run below the floor of the work space 106, wherein the diffusers 111 may be configured to direct the cool air upwardly into the areas surrounding the racks 102a-102d (e.g., through a perforated floor).
The work space 106 may include heat loads other than the trays, such as from people in the space and lighting. Where the volume of air that passes through the various racks is very high and picks up a very large thermal load from multiple computers, the small additional load from other sources may be negligible, apart from perhaps a small latent heat load caused by workers, which may be removed by a smaller auxiliary air conditioning unit as described above.
In some implementations, the rack zone 141 may be cooled using the evaporative cooling system 112 (as described above), by cooling units cooled by a chiller system as will be described next, or a combination of both.
Referring again to the racks 102a-102d in
Other arrangements may also be used. For example, trays may be mounted vertically in groups, such as in the form of computer blades. The trays may simply rest in a rack and be electrically connected after they are slid into place, or they may be provided with mechanisms, such as electrical traces along one edge, that create electrical and data connections when they are slid into place.
Air may circulate from the work space 106 across the trays and into the hot exhaust spaces 104a, 104b, 104c, and 104d behind the trays. The air may be drawn into the trays by fans mounted at the back of the trays (not shown). The fans may be programmed or otherwise configured to maintain a set exhaust temperature for the air into the warm air plenum, and may also be programmed or otherwise configured to maintain a particular temperature rise across the trays. Where the temperature of the air in the work space 106 is known, controlling the exhaust temperature also indirectly controls the temperature rise. The regions between the racks 102a-102d exposed to the cool air of the work space 106 may, in certain circumstances, be referenced as “cold aisles,” and the hot exhaust spaces 104a-104d as “warm aisles.”
In operation, then, circulating air may enter the fronts of the racks 102a-102d from the work space 106, may pass across heated components in the racks 102a-102d, such as microprocessors and memory chips, and may be exhausted out of the back of the racks into cooling units (not shown), so that each cooling unit receives heated exhaust air from racks on opposed sides. (Where cooling units proximate to the rows of racks are used, cooled air may be recirculated through the work space and may be blended with outside air that has received direct evaporative cooling.) The cooling units may be provided with air circulating equipment such as fans and cooling equipment such as cooling coils, so that air is drawn through the coils and exhausted in a cooled state back into the data center work space 106 or into a collection of chimney plenums 108.
In some examples, warm air that passes through the racks may be drawn upward into the hot air plenum 105. In another implementation, air may enter a warm air plenum behind the computers and be drawn by fans down into an under-floor space (e.g., below a real or false floor), where it may be gathered, circulated through cooling coils, and delivered back into the workspace, such as through perforated floor plates, or may not be cooled and may be exhausted form the data center 100, or some may be cooled and blended with outdoor make-up air that has been direct evaporative cooled.
The cooling units may be individual units that are configured in a row and brought into close contact with the computing racks. In some examples, two rows of computers may be installed in the data center 100, where each row contains computing racks sandwiched on each side of corresponding cooling units. In one example, each rack may be approximately five feet wide (having three bays of computers per rack), and six to eight feet, or 14 or more feet high.
Also where cooling units are provided adjacent to the racks, a collection of cooling loops 152, in the form of additional piping, can also be included and connected to headers 154, 156 in preparation for installing computing racks above the respective loops. Although vertical positioning is not discernible in this figure, the cooling loop 152 may be located below the level of the computer racks, as may cooling loops for the particular computing racks that are shown here, where those additional cooling loops are not visible in the figure. For example, the cooling loops may be located below a raised floor in a facility, or may be located in a ceiling space of a level that is below the level where the computing racks are located. Similarly, the cooling loop may be located above the computer units, though such location may raise issues with respect to leaking water impinging on the computer racks. Cooling loop 152 may also be located within or adjacent to the cooling units.
The temperature rise of the air flowing across the trays can be large. For example, the work space 106 temperature may be about 77° F. (25° C.) and the exhaust temperature into the hot exhaust spaces 104a-104d may be set to 113° F. (45° C.), for a 36° F. (20° C.)) rise in temperature. The exhaust temperature may also be as much as 212° F. (100° C.) where the heat generating equipment can operate at such elevated temperature. For example, the temperature of the air exiting the equipment and entering the warm-air plenum may be 118.4, 122, 129.2, 136.4, 143.6, 150.8, 158, 165, 172.4, 179.6, 186.8, 194, 201, or 208.4° F. (48, 50, 54, 58, 62, 66, 70, 74, 78, 82, 86, 90, 94, or 98° C.). Such a high exhaust temperature generally runs contrary to teachings that cooling of heat-generating electronic equipment is best conducted by washing the equipment with large amounts of fast-moving, cool air. Such a cool-air approach does cool the equipment, but it also uses lots of energy.
Cooling of particular electronic equipment, such as microprocessors, may be improved even where the flow of air across the trays is slow, by attaching impingement fans to the tops of the microprocessors or other particularly warm components, or by providing heat pipes and related heat exchangers for such components.
Cooling water for the cooling loops 152 may be provided from a cooling water circuit powered by pump 124. The cooling water circuit may be formed as a direct-return, or indirect-return, circuit, and may generally be a closed-loop system. Pump 124 may take any appropriate form, such as a standard centrifugal pump. Heat exchanger 122 may remove heat from the cooling water in the circuit. Heat exchanger 122 may take any appropriate form, such as a plate-and-frame heat exchanger or a shell-and-tube heat exchanger.
Heat may be passed from the cooling water circuit to a condenser water circuit that includes heat exchanger 122, pump 120, and cooling tower 118. Pump 120 may also take any appropriate form, such as a centrifugal pump. Cooling tower 118 may be, for example, one or more forced draft towers or induced draft towers. The cooling tower 118 may be considered a free cooling source, because it requires power only for movement of the water in the system and in some implementations the powering of a fan to cause evaporation; it does not require operation of a compressor in a chiller or similar structure. The cooling by the towers may be considered indirect evaporative cooling because the evaporation does not occur in air that has or will enter the data center 100 building, while the other evaporative cooling may be referred to as direct evaporative cooling. By using an electronic control system, and depending on the conditions (e.g., outdoor wet and dry bulb temperatures, and heat load in the building), either of the evaporative cooling sources may be used, or the two may be used simultaneous and in varying proportions to each other.
The cooling tower 118 may take a variety of forms, including as a hybrid cooling tower. Such a tower may combine both the evaporative cooling structures of a cooling tower with a water-to-water heat exchanger. As a result, such a tower may be fit in a smaller face and be operated more modularly than a standard cooling tower with separate heat exchanger. An additional advantage may be that hybrid towers may be run dry, as discussed above. In addition, hybrid towers may also better avoid the creation of water plumes that may be viewed negatively by neighbors of a facility.
As shown, the fluid circuits may create an indirect water-side economizer arrangement. This arrangement may be relatively energy efficient, in that the only energy needed to power it is the energy for operating several pumps and fans. In addition, this system may be relatively inexpensive to implement, because pumps, fans, cooling towers, and heat exchangers are relatively technologically simple structures that are widely available in many forms. In addition, because the structures are relatively simple, repairs and maintenance may be less expensive and easier to complete. Such repairs may be possible without the need for technicians with highly specialized knowledge.
Alternatively, direct free cooling may be employed, such as by eliminating heat exchanger 122, and routing cooling tower water (condenser water) directly to cooling coils (not shown) Such an implementation may be more efficient, as it removes one heat exchanging step. However, such an implementation also causes water from the cooling tower 118 to be introduced into what would otherwise be a closed system. As a result, the system in such an implementation may be filled with water that may contain bacteria, algae, and atmospheric contaminants, and may also be filled with other contaminants in the water. A hybrid tower, as discussed above, may provide similar benefits without the same detriments.
Control valve 126 is provided in the condenser water circuit to supply make-up water to the circuit. Make-up water may generally be needed because cooling tower 118 operates by evaporating large amounts of water from the circuit. The control valve 126 may be tied to a water level sensor in cooling tower 118, or to a basin shared by multiple cooling towers. When the water falls below a predetermined level, control valve 126 may be caused to open and supply additional makeup water to the circuit. A back-flow preventer (BFP) may also be provided in the make-up water line to prevent flow of water back from cooling tower 118 to a main water system, which may cause contamination of such a water system.
Optionally, a separate chiller circuit may be provided. Operation of system 100 may switch partially to this circuit, which can be used in combination with the evaporation cooling system 112. During times of extreme atmospheric ambient (i.e., hot and humid) conditions or times of high heat load in the data center 101 the chiller circuit may be used exclusively. Controlled mixing valves 134 are provided for electronically switching to the chiller circuit, or for blending cooling from the chiller circuit with cooling from the condenser circuit. Pump 128 may supply tower water to chiller 130, and pump 132 may supply chilled water, or cooling water, from chiller 130 to the remainder of system 100. Chiller 130 may take any appropriate form, such as a centrifugal, reciprocating, or screw chiller, or an absorption chiller.
The chiller circuit may be controlled to provide various appropriate temperatures for cooling water. In some implementations, the chilled water may be supplied exclusively to a cooling coil, while in others, the chilled water may be mixed, or blended, with water from heat exchanger 122, with common return water from a cooling coil to both structures. The chilled water may be supplied from chiller 130 at temperatures elevated from typical chilled water temperatures. For example, the chilled water may be supplied at temperatures of 55° F. (13° C.) to 65 to 70° F. (18 to 21° C.) or higher. The water may then be returned at temperatures like those discussed below, such as 59 to 176° F. (15 to 80° C.). In this approach that uses sources in addition to, or as an alternative to, free cooling, increases in the supply temperature of the chilled water can also result in substantial efficiency improvements for the data center 100.
Pumps 120, 124, 128, 132, may be provided with variable speed drives. Such drives may be electronically controlled by a central control system to change the amount of water pumped by each pump in response to changing set points or changing conditions in the data center 100. For example, pump 124 may be controlled to maintain a particular temperature in the rack zone 141, such as in response to signals from a thermostat or other sensor in the rack zone 141.
The system can include a controller 170. The controller 170 can be a computer system configured to receiving readings from various sensors placed about the data center 100 and respond to the sensed values by altering the operation of the evaporative cooling system (e.g., changing atomizer flow rates, louver positions, fan speeds), For example, in operation, the controller 170 can be in controllable communication with cooling systems of the data center 100, and can respond to signals from various sensors placed in the data center 100. The sensors may include, for example, thermostats, humidistats, flowmeters, and other similar sensors. In one implementation, one or more thermostats may be provided in the hot exhaust spaces 104a-104d, and one or more thermostats may be placed in the rack zone 141. In addition, air pressure sensors may be located in work space 106, and in the conduits 160. The sensors can be used by the controller 170 to determine whether to use the evaporative cooling system 112 exclusively, in combination with one or more of the other cooling systems described (or others), or not at all. For example, a thermostat 172 may be used by the controller 170 to sense the temperature differential at the boundary of the evaporation zone 143 and the work space 106 to control the speed of associated pumps, so that if temperature begins to rise, the pumps turn faster to provide additional cooling waters. Pressure sensors may also be used by the controller 170 to control the speed of various items such as fans 113 and 116 to maintain a set pressure differential between two spaces, such as that attic 105 and the rack zone 141, and to thereby maintain a consistent airflow rate. Where mechanisms for increasing cooling, such as speeding the operation of pumps, are no longer capable of keeping up with increasing loads, the controller 170 may activate the chiller 130 and associated pumps 128, 132, and may modulate control valves 134 accordingly to provide additional cooling.
Various values for temperature of the fluids in data center 100 may be used by the controller 170 in the operation of data center 100. In one exemplary implementation, the temperature setpoint in the hot exhaust spaces 104a-104d may be selected to be at or near a maximum exit temperature for trays in racks 102a-102d. This maximum temperature may be selected, for example, to be a known failure temperature or a maximum specified operating temperature for components in the trays, or may be a specified amount below such a known failure or specified operating temperature. In certain implementations, a temperature of 45° C. may be selected. In other implementations, temperatures of 25° C. to 125° C. may be selected. Higher temperatures may be particularly appropriate where alternative materials are used in the components of the computers in the data center, such as high temperature gate oxides and the like.
In one implementation, supply temperatures for cooling water may be 68° F. (20° C.), while return temperatures may be 104° F. (40° C.). In other implementations, temperatures of 50° F. to 84.20° F. or 104° F. (10° C. to 29° C. or 40° C.) may be selected for supply water, and 59° F. to 176° F. (15° C. to 80° C.) for return water. Chilled water temperatures may be produced at much lower levels according to the specifications for the particular selected chiller. Cooling tower water supply temperatures may be generally slightly above the wet bulb temperature under ambient atmospheric conditions, while cooling tower return water temperatures will depend on the operation of the data center 100.
Using these parameters and the parameters discussed above for entering and exiting air, relatively narrow approach temperatures may be achieved with the data center 100. The approach temperature, in this example, is the difference in temperature between the air leaving a coil and the water entering a coil. The approach temperature will always be positive because the water entering the coil is the coldest water, and will start warming up as it travels through the coil. As a result, the water may be appreciably warmer by the time it exits the coil, and as a result, air passing through the coil near the water's exit point will be warmer than air passing through the coil at the water's entrance point. Because even the most-cooled exiting air, at the cooling water's entrance point, will be warmer than the entering water, the overall exiting air temperature will need to be at least somewhat warmer than the entering cooling water temperature.
Keeping the approach temperature small permits a system to be run on free, or evaporative, cooling for a larger portion of the year and reduces the size of a needed chiller, if any is needed at all. To lower the approach temperature, the cooling coils may be designed for counterflow rather than for self-draining. In counter-flow, the warmest air flows near the warmest water and the coolest air exits near where the coolest water enters.
In certain implementations, the entering water temperature may be 64° F. (18° C.) and the exiting air temperature 77° F. (25° C.), as noted above, for an approach temperature of 12.6° F. (7° C.). In other implementations, wider or narrower approach temperature may be selected based on economic considerations for an overall facility.
Referring now to
Outside air is drawn by the fans 116 into an evaporation zone 202. Within the evaporation zone 202, atomized water is introduced into the air by the atomizers 142 and allowed to evaporate thereby cooling the air. In some embodiments, the evaporation zone 202 may provide a volume of space that is sufficient to allow the atomized water to evaporate substantially completely, such that liquid water does not enter the rack zone 204.
The cooled air is allowed to flow substantially freely (e.g., the air is generally not routed through plenums) into the rack zone 204. The cooled air flows among, through, and out of a rack zone 204 that includes a collection of rows of racks 206 where the air is warmed. The warmed air is then returned to the outside through an exhaust 114. In some implementations, the warmed air may be drawn into an exhaust plenum located beneath the floor of the data center, where the warmed air may be routed outdoors.
In some embodiments, the data center 200 may be oriented such that the evaporative cooling system 112 faces in the general direction of naturally prevailing winds. For example, by orienting the data center 200 substantially along the direction of the prevailing winds, naturally occurring outdoor breezes may help cause cool air to flow into, through and out of the data center, and reduce the need for power to operate the fans 113 and 116.
It should be understood that the evaporative cooling system described in reference to
Atomized water is sprayed (320) into the drawn outside air at a location downstream of the entry zone. For example, a mist of water may be produced by the atomizers 140 and/or 142 at or near the outlets of the fans 116. The atomized water is allowed to evaporate in an evaporation zone and thereby cool the drawn outside air (330). In some implementations, the evaporation zone may be a space in which the sprayed atomized water and drawn air can mix long enough to allow the water to evaporate. For example, the evaporation zone may be the evaporation zone 143 or 202.
Optionally, a supplemental cooling system may be activated (335) as needed. For example, some outdoor environmental conditions (e.g., outdoor air with very high relative humidity) may limit how effectively the evaporative cooling system 112 may be able to cool the data center 100. To supplement the cooling effect provided by the evaporative cooling system 112, a supplemental cooling system including the components 118-134 (e.g., chillers, cooling towers, heat exchangers, pumps) may be optionally engaged to further cool the air in the data center 100.
The cool air is directed into a rack zone that is downstream of the evaporation zone (340). In some implementations, the cool air may be drawn through the racks to absorb heat that is generated by the computers in the racks, and is then exhausted outside of the data center. In some implementations, e.g., as described in reference to
In some implementations, the process 300 may be used in addition to, or in conjunction with, processes for controlling data center temperatures using a supplemental cooling system. For example, cooling towers, water-to-water, air-to-air and/or air-to-water heat exchangers, chillers, and/or other equipment may be used to supplement the cooling provided by an evaporative cooling system controlled by the process 300. In some implementations, a controller executing the steps of the process 300 may selectively activate a supplemental cooling system based on the measurements and on the temperature and relative humidity set points for the rack zone.
The process 400 starts (410) and the temperature of the outside air supply is measured (420). The temperature measurement is used to determine (430) if the temperature exceeds the cold rack zone setpoint. If the temperature does not exceed the cold rack zone setpoint, then the atomizer flow is turned off, (440) and the outside air temperature is measured (420) again. If the temperature exceeds the cold rack zone setpoint, then the relative humidity of the supply air entering the rack zone, as well as the air flow rate, is measured (450). For example, if the air outside is already cooler than what is needed to cool the data center, then no evaporation is used to cool the air any further. In some implementations, the temperature and humidity may be measured using devices such as thermometers and humidistats.
The smaller of (A) the atomizer flow to reduce the outside air temperature to the setpoint, or (B) the atomizer flow to raise the relative humidity to the maximum allowed, is calculated (460). The calculated value is then used to update (470) the atomizer flow rate. Under some conditions, such as very high outside temperatures and/or relative humidities, the evaporative cooling system may not provide sufficient cooling even when the atomizer rate is at its substantially maximum flow rate. In some implementations, additional cooling beyond the capacity of the evaporative cooling system may be provided by activating a supplemental cooling system. The flow rate is based on the atomizer flow rate, the measured air temperature, the measured relative humidity, and the rack zone temperature and humidity set points. Once the flow rate is updated (470), the outside air supply temperature is measured (420) once again, and the cycle continues.
The process 500 starts (510) and the temperature of the air entering the rack zone is measured (520), for example with a thermometer, and if the temperature is determined (530) to not exceed the rack zone temperature maximum, then the atomizer flow rate is reduced (540). Otherwise, if the temperature is determined (530) to exceed the rack zone temperature maximum, then the humidity of the air entering the rack zone is measured (550). For example, a humidistat may be used to measure the humidity of the air.
If the humidity is determined (560) to be above the maximum humidity level allowed, then the atomizer flow rate is reduced (540). Otherwise, if the humidity is determined (560) not to be above the maximum humidity level allowed, then the atomizer flow rate is increased (570). Under some conditions, such as very high outside temperatures and/or relative humidities, the evaporative cooling system may not provide sufficient cooling even when the atomizer flow rate is at its substantially maximum flow rate. In some implementations, additional cooling beyond the capacity of the evaporative cooling system may be provided by activating a supplemental cooling system. The atomizer flow rate is varied based on the previous atomizer flow rate, the measured air temperature, the measured relative humidity, and the rack zone temperature and humidity set points. The temperature of the air entering the rack zone is then measured (520) again.
Referring now to
The data center includes an evaporative cooling system 602 controlled by a controller 670. In some implementations, the evaporative cooling system 602 may be the evaporative cooling system 112. In some implementations, the controller 670 may be a computer configured to receiving readings from various sensors placed about the data center 600 (e.g., thermostats, humidistats, flow sensors, pressure transducers) and respond to the sensed values by altering the operation of the evaporative cooling system 602 (e.g., changing atomizer flow rates, louver positions, fan speeds), In some implementations, the controller 670 may be substantially similar in function to the controller 170 of
A set of louvers 604 are selectively opened to permit outside air to flow through the louvers 604 and a filter 606 into an entry zone 608. A collection of fans 610 draw the air from the entry zone 608 to an evaporation zone 612. Within the evaporation zone 612, a collection of atomizers 614 controllably sprays atomized water into the drawn air. As the atomized water evaporates, heat energy is drawn out of, and thereby cools, the air within the evaporation zone 612. The cooled outside air is passed through an outside air plenum 616 that substantially intersects with one or more heat exchangers 620, and is finally exhausted back to the outside by an exhaust fan 622.
A workspace 630 is defined around a number of parallel rows of computers mounted in vertical racks, such as racks 632a, 632b, 632c, 632d included in a rack zone 636. Air may circulate from the workspace 630 across the trays and into hot exhaust spaces behind the trays (not shown). As the air flows across the trays, the air absorbs heat that is generated by the computers thereby cooling the trays and heating the air. The heated air may be routed upward into a hot air plenum 640, by chimney plenums 634, or into a raised floor or basement, or other appropriate space, and may be gathered there by air handling units that include, for example, one or more fans appropriately sized for the task.
A circulation fan 626 draws the heated air through the hot air plenum to an indoor air plenum 618 that substantially intersects with the heat exchangers 620. As the heated indoor air passes though the heat exchangers 620, heat energy from the heated air is transferred to the cooled outdoor air. In other words, the cool outdoor air is heated by cooling the hot indoor air without substantial mixing of the indoor and outdoor air.
The cooled indoor air is then drawn through the fan 626 into a cool air plenum 650. The cool air plenum 650 routes the cooled indoor air to one or more diffusers, e.g., diffusers 624 such that the cooled indoor air is distributed into the rack zone 636 and throughout the work space 630.
In some implementations, the heat exchangers 620 can be air-to-air heat exchangers. In a particular example, the air-to-air heat exchangers are heat wheel heat exchangers. In the example shown, there are three heat exchangers 620 arranged in series. In other implementations, more or less heat exchangers can be used and they can be positioned in series or in other configurations relative to each other.
The cooled outside air flows into an outside air plenum 708 that substantially intersects with the heat exchangers 750. The heat exchangers 750 also substantially intersect an indoor air plenum 710. The indoor air plenum 710 carries hot air from the data center and through the heat exchangers 750. The heat exchangers 750 transfer heat energy from the hot indoor air to the cooled outside air, thereby cooling the indoor air and heating the outside air. In some implementations, since the outside air may be substantially isolated from a work space (e.g., the work space 630 of
The heated outside air is exhausted to the outdoors by a vent 712 that, in some embodiments, may include a vent fan 714. The cooled indoor air is drawn out to the data center by a fan 716. In some implementations, the flow of air to the data center may be measured by a flow meter. In some implementations, the cooled air may be cooled further by evaporatively cooling the air with a controllable water mist provided by a collection of atomizers 718 within a misting evaporation zone 720.
In the example of the evaporative cooling system 700, the heat exchangers 750 implement a technology referred to as “heat wheels”. Generally speaking, a heat wheel is a rotary heat exchanger that operates on the air-to-air principle of heat transfer. It can provide a way of recovering air conditioning energy in hot, humid climates. Heat wheels can improve air-side economization (e.g., free cooling, the use of outside air to cool servers in the data center). Rather than introducing exterior air directly into the data center, the heat wheel briefly mixes the outside air and exhaust air to create an “air-to-air” heat exchanger. Some examples of heat wheels that may be used in various implementations of the evaporative cooling system 700 are the “TC” and “TF” series “Thermowheel” heat wheel products available from Thermotech Enterprises, Inc. of Tampa, Fla.
Referring now to
Referring now to
The process 800 begins when outside air is drawn (810) at ambient conditions into an entry zone of a data center (e.g., the entry zone 608). For example, the ambient conditions may be the naturally occurring temperature and humidity of the outdoor air that surrounds the data center. Atomized water is sprayed (820) into the outside air at a location, an evaporation zone, downstream of the entry zone (e.g., the evaporation zone 612). The atomized water is allowed (830) to evaporate and cool the outside air in the evaporation zone.
The cool air is directed (840) to one side of a heat exchanger positioned between the evaporation zone and a rack zone. For example, the heat exchangers 620 and 750 are positioned at the boundary between the outside air supply and the indoor air that is supplied to the work space 630 and a rack zone occupied by the racks 632a-632d. The cool air is directed (840) through one side of the heat exchangers 620 by the outside air plenum 616.
Hot air from the rack zone is directed (850) into an opposite side of the heat exchanger to cool the air. For example, the indoor air plenum 618 directs air heated by the computers over the other side of the heat exchangers 620.
Optionally, a supplemental cooling system may be activated (855) as needed. For example, some outdoor environmental conditions (e.g., outdoor air with very high relative humidity) may limit how effectively the evaporative cooling system 602 may be able to cool the data center 600. To supplement the cooling effect provided by the evaporative cooling system 602, a supplemental cooling system including the components 118-134 of
The cooled air from the heat exchangers is then recirculated (860) back into the rack zone. For example, the cool air plenum 650 carries air cooled by the heat exchangers 620 back to the work space 630 where the air is dispersed by the diffusers 624.
In some implementations, the process 800 may be used in addition to, or in conjunction with, processes for controlling data center temperatures using a supplemental cooling system. For example, cooling towers, water-to-water and air-to-water heat exchangers, chillers, or other equipment may be used to supplement the cooling provided by an evaporative cooling system controlled by the process 800. In some implementations, a controller executing the steps of the process 800 may selectively activate the supplemental cooling system based on the measurements and on the temperature and relative humidity set points for the rack zone.
The process 900 starts (910) and the temperature of the air entering a rack zone is measured (920). The temperature measurement is used to determine (930) if the temperature exceeds the cold rack zone setpoint. If the temperature does not exceed the cold rack setpoint, then the atomizer flow is maintained off (940) and the temperature is measured (920) again.
Otherwise, the relative humidity and air flow rate of the supply air entering the rack zone is measured (950). The smaller of (A) the atomizer flow needed to reduce the outside air temperature to the setpoint, or (B) the atomizer flow needed to saturate the humid exhaust, is calculated (960). For example, the flow rate (A) may be implemented when the outdoor air is sufficiently dry to allow evaporative cooling to reduce the air temperature to the cool rack zone setpoint. In another example, the flow rate (B) may be implemented to cool the outside air as much as possible while still substantially evaporating the atomized water, thereby possibly preventing liquid water from precipitating out of the outside air or condensing within the plenum or heat exchangers. The calculated (960) flow rate is then used to update (970) the atomizer flow rate (e.g., to turn on the atomizer flow if off, or adjust the atomizer flow if already on). Once the flow rate has been updated (970) the air temperature is measured (920) once again. As such, the flow rate is varied based on the previous atomizer flow rate, the measured air temperature, the measured relative humidity, and the rack zone temperature and humidity set points. Under some conditions, such as very high outside temperatures and/or relative humidities, the evaporative cooling system may not provide sufficient cooling even when the atomizer flow rate is at its substantially maximum flow rate. In some implementations, additional cooling beyond the capacity of the evaporative cooling system may be provided by activating a supplemental cooling system.
The process 1000 starts (1010) and the temperature of the air entering a rack zone is then measured (1020) to determine (1030) if the air temperature exceeds a rack zone maximum temperature. If the temperature is determined (1030) to not exceed the rack zone maximum temperature, then the atomization flow rate is reduced (1040) and the air temperature is measured (1020) again. Otherwise, the humidity of the humid exhaust is measured (1050). For example, if the temperature does not exceed the rack zone's upper operating temperature, then no additional cooling of the outside air may be needed thereby conserving water.
If the humidity of the humid exhaust is determined (1060) to be above saturation, then the atomization flow rate remains off or is reduced (1040). For example, the flow rate may be reduced (1040) since saturated air cannot evaporate additional liquid water. As such, the atomized mist may pool as liquid water within the evaporation zone 612, condense within the plenum 616 or the heat exchangers 620, or be vented outside without substantially cooling the air. Otherwise, the atomization flow rate is increased (1070) and the air temperature is measured (1020) again. As such, the flow rate is varied based on the previous atomizer flow rate, the measured air temperature, the measured relative humidity, and the rack zone temperature and humidity set points. Under some conditions, such as very high outside temperatures and/or relative humidities, the evaporative cooling system may not provide sufficient cooling even when the atomizer flow rate is at its substantially maximum flow rate. In some implementations, additional cooling beyond the capacity of the evaporative cooling system may be provided by activating a supplemental cooling system.
A number of embodiments have been described. Nevertheless, it will be understood that various modifications can be made without departing from the spirit and scope of the invention.
In addition, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. In addition, other steps can be provided, or steps can be eliminated, from the described flows, and other components can be added to, or removed from, the described systems. Accordingly, other embodiments are within the scope of the following claims.
This application is a continuation of, and claims priority under 35 U.S.C. §120 to, U.S. patent application Ser. No. 12/761,981, filed Apr. 16, 2010 now U.S. Pat. No. 8,974,274 and entitled “Evaporative Induction Cooling.” The entire contents of this prior application is hereby incorporated by reference as if fully set forth herein.
| Number | Name | Date | Kind |
|---|---|---|---|
| 1887938 | Lewis | Nov 1932 | A |
| 2118949 | Scott | May 1938 | A |
| 2256940 | Crawford | Sep 1941 | A |
| 3997635 | Hallgren | Dec 1976 | A |
| 4103508 | Apple | Aug 1978 | A |
| 4118945 | Boochever et al. | Oct 1978 | A |
| 4313310 | Kobayashi et al. | Feb 1982 | A |
| 4474027 | Kaya et al. | Oct 1984 | A |
| 4526011 | Logan et al. | Jul 1985 | A |
| 4761965 | Viner | Aug 1988 | A |
| 4910971 | McNab | Mar 1990 | A |
| 5529536 | Sizemore et al. | Jun 1996 | A |
| 5555742 | Kelley | Sep 1996 | A |
| 5643082 | Furukawa | Jul 1997 | A |
| 5664995 | O'Keefe | Sep 1997 | A |
| 5729983 | Garrett et al. | Mar 1998 | A |
| 5806762 | Herr et al. | Sep 1998 | A |
| 5943211 | Havey et al. | Aug 1999 | A |
| 6141986 | Koplin | Nov 2000 | A |
| 6202434 | Hearne, Jr. | Mar 2001 | B1 |
| 6374627 | Schumacher et al. | Apr 2002 | B1 |
| 6386969 | O'Brien | May 2002 | B1 |
| 6412292 | Spinazzola et al. | Jul 2002 | B2 |
| 6418728 | Monroe | Jul 2002 | B1 |
| 6557357 | Spinazzola et al. | May 2003 | B2 |
| 6786056 | Bash et al. | Sep 2004 | B2 |
| 6923111 | Kiefer et al. | Aug 2005 | B2 |
| 7219714 | Heydari | May 2007 | B1 |
| 7299647 | Tilton et al. | Nov 2007 | B2 |
| 7500911 | Johnson et al. | Mar 2009 | B2 |
| 7864530 | Hamburgen et al. | Jan 2011 | B1 |
| 8047904 | Yates et al. | Nov 2011 | B2 |
| 8162293 | Goldman et al. | Apr 2012 | B2 |
| 20010002541 | Patel et al. | Jun 2001 | A1 |
| 20020102936 | Daumler | Aug 2002 | A1 |
| 20030173688 | Koo | Sep 2003 | A1 |
| 20040020224 | Bash et al. | Feb 2004 | A1 |
| 20040177613 | DePenning | Sep 2004 | A1 |
| 20040206101 | Bash et al. | Oct 2004 | A1 |
| 20050103032 | Pierson | May 2005 | A1 |
| 20060065000 | Belady | Mar 2006 | A1 |
| 20070183126 | Tilton et al. | Aug 2007 | A1 |
| 20080029250 | Carlson et al. | Feb 2008 | A1 |
| 20080055850 | Carlson et al. | Mar 2008 | A1 |
| 20080204999 | Clidaras et al. | Aug 2008 | A1 |
| 20090046427 | Noteboom et al. | Feb 2009 | A1 |
| 20090201645 | Kashirajima et al. | Aug 2009 | A1 |
| 20090310300 | Chrysler | Dec 2009 | A1 |
| 20100110626 | Schmitt et al. | May 2010 | A1 |
| 20110009047 | Noteboom et al. | Jan 2011 | A1 |
| 20110023506 | Day et al. | Feb 2011 | A1 |
| 20110175498 | Bash et al. | Jul 2011 | A1 |
| Number | Date | Country |
|---|---|---|
| 1546911 | Nov 2004 | CN |
| Entry |
|---|
| Data Centers; Open Compute Project; Apr. 7, 2011; Jul. 18, 2011; opencompute.org; 42 pages. |
| Facebook “Environmental Practices”; Jan. 11, 2010; Jul. 18, 2011; Prineville Data Center; 2 pages. |
| International Search Report and Written Opinion; Eun, Park Ji; Oct. 26, 2011; World Intellectual Property Organization (WIPO) (International Bureau Of); PCT/US2011/032568; 9 pages. |
| Authorized Officer Simin Baharlou, Notification Concerning Transmittal of International Preliminary Report on Patentability for related international application PCT/US2011/032568, mailed Oct. 26, 2012, 6 pages. |
| U.S. Appl. No. 12/060,130, filed Mar. 31, 2008, Cooling Diversity in Data Centers, Carlson et al. |
| U.S. Appl. No. 12/631,644, filed Dec. 4, 2009, Modular Data Center Cooling, Hamburgen et al. |
| Number | Date | Country | |
|---|---|---|---|
| 20130010423 A1 | Jan 2013 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 12761981 | Apr 2010 | US |
| Child | 13617349 | US |
