Patent Application
20070139883
This invention relates to systems and methods for providing operational resources to data centers. More particularly, this invention relates to providing operational resources such as cooling and back-up power to the vast array of racks of electronics that populate data centers. These racks often include processing devices such as servers that require high quality, dependable sources of electric power, and reliable cooling due to the heat generated by the electronics.
The information age is upon us, driven by the ability of anyone with a computer or computer-like device, such as a wireless hand-held device, to access the numerous information networks that are spread around the globe. These networks include the Internet as well as the private networks used by companies to provide round-the-clock access to office and company information for its employees, customers and clients. Access to these networks can be obtained from the home, office, or virtually anywhere.
In order to provide access to these networks, network providers utilize dedicated sites that are often referred to as data centers. These data centers include numerous racks of electronics that provide various capabilities to its users such as: data storage, data access, e-mail, network applications, Internet access, etc. The electronics can include servers (i.e., computers that may be accessed by multiple users “simultaneously” from remote locations—servers typically do not provide conventional users with keyboard/mouse/monitor interfaces (such access, however, may be provided to a system administrator)), power supplies and access hardware/software (such as interfaces for T1 lines, broadband access or dial-up access).
Often, the racks of electronics are configured to be somewhat stand-alone, whereby each rack includes at least one server, a power supply and interface hardware to provide access to that server. In this manner, the operational capacity of the data center can be more easily managed, often by simply adding one or more new server racks as required. Server racks can also be configured such that they accept multiple servers in the same rack, such as by way of example, “blade servers.” With the increased density of the electronics associated with each server in any given rack, the power requirements and heat generation of the racks (and ultimately the data centers) increases. In addition to the stand-alone racks of electronics, these data centers often must also have various support systems to maintain normal operations. These support systems can include uninterruptible power supplies, air conditioning systems to provide vast quantities of cooling air and back-up energy power systems to provide back-up power in the event of a loss or degradation of primary power (which is often utility power).
While the use of individual, stand-alone racks provides flexibility and ease of management, these advantages are inherently limited by the amount of power and cooling that are available. For example, a data center might be initially designed with the capability for thirty racks of servers even though the initial system will only include twenty servers. In that case, the air conditioning system and power system would have to be significantly upgraded if the system grew beyond thirty servers.
In addition, one of the most significant expenses incurred during the operation of the data center is the cost of electricity. This is due not only to the cost of supplying power to the racks of electronics, but to the cost of supplying power to the vast number of compressors that are included in the air conditioning system needed to provide cooling air to the server racks. In most instances, the data center includes a raised floor having openings for each of the server racks (which are themselves open on the bottom, such that the openings in the floor are aligned with the servers). The server racks also include a number of blowers installed at the top of the racks. The cooling air is provided underneath the raised floor and pulled through the server racks by the blowers (which also adds to the electricity demands of the facility), which blow the hot air from the server racks into the room. The air conditioning system must then extract the excess heat from the server room and, for example, exhaust it to the atmosphere.
In some instances where somewhat precise cooling would be advantageous, water cooling systems have been employed. These systems typically utilize water or some other coolant that is maintained in close proximity to, if not direct contact with, the components being cooled. For example, personal computers that use water and/or liquid cooling systems to maintain the temperature of critical electronic components include the Power Mac G5 sold by Apple Computer. The liquid cooling subsystems provide a high degree of cooling such that, for example, the quantity of forced cooling air can be reduced (so that the noise generated by the fans is reduced). One of the inherent risks of such systems, however, is the potential for disaster if the cooling system leaks fluid onto the powered electronics.
In view of the foregoing, it is an object of the present invention to provide improved cooling for data centers.
It is also an object of the present invention to provide improved support systems for data centers which can reduce the demands of the data center for electric power.
It is an additional object of the present invention to provide data centers with improved back-up electrical power.
It is still a further object of the present invention to promote data centers with integrated cooling and back-up power capabilities.
These and other objects of the invention are accomplished by replacing conventional air conditioning systems in a data center with a closed loop of compressed air and heat exchangers. In addition, the electronics in the server racks are configured with heat sinks in thermal contact with the electronics. The present invention may be practiced by either direct thermal contact or by locating the closed loop in close proximity to the electronics such that heat is removed from the electronics by conduction and/or convection (and the term “thermal contact” shall refer to either configuration for the purposes described herein). A portion of the closed loop is passed through each of the server racks and through the heat sinks. The heat from the electronics is transferred to the compressed air, which is then directed to the heat exchangers. The heat exchangers are preferably installed with direct access to the atmosphere, so that heat extracted from the closed loop can be easily exhausted. Once the heat is extracted, the, cooler compressed air is once again returned to the servers racks.
In order to further improve the performance of the closed loop to extract heat from the server racks, persons skilled in the art may include one or more subsystems to cool the compressed air before passing it through the server racks. For example, it may be advantageous to include a chilled water system in the data center and to run the closed loop through the chilled water prior to the server racks. It may also be desired to utilize water tanks, kept at a location away from the electronics, to store water at a temperature lower than that of the compressed air after the heat exchangers, and to pass the closed loop through the water tanks prior to the server racks (or, it may be desirable to utilize both the chilled water subsystem and the water storage tanks).
The present invention may also include an air turbine, generator and support electronics in each server rack to provide short term back-up power in the event of a loss or degradation of power from the primary source. In this configuration, the closed loop would include routing to the turbine after exit from the heat sinks (through valves that would be normally closed). In the event of a disruption in primary power (either quality or quantity), the valve would open and the heated compressed air would drive the turbine. A generator coupled to the turbine would produce the back-up electricity. In addition, it may be advantageous to include a capacitor in each server rack to provide bridging power for the brief instant that primary power is failing and prior to the turbine being up to speed.
In another aspect of the present invention, a control system can be used to monitor the temperature of the various heat sinks. In the event that hot spots occur (i.e., instances where one server rack might be hotter than others), the control system could direct additional compressed air to the affected heat sinks so that all of the server racks are maintained at a relatively constant temperature.
The present invention provides many advantages over conventional systems. For example, by eliminating or reducing the size of the conventional air conditioning systems, overall cost for electricity is significantly reduced. In addition, eliminating the forced air approach reduces the manufacturing cost of the server racks, because the blowers are no longer needed, as well as the associated reduction in electric demands from the use of the blowers.
Another advantage of the present invention is that it eliminates some of the inherent problems related to expansion of the data center because each server rack is independently cooled. When a new rack is added, compressed cooling air may be brought into the rack without concern for exceeding the existing air conditioning capacity of the center. In addition, because each server rack is independently cooled, the data center can be more densely populated with server racks, and each rack may be more densely populated with servers and electronics.
A further advantage of the present invention is the elimination of the batteries that are often used to provide short-term back-up power to the system. These batteries usually are accompanied by a complex system to charge and maintain the batteries (and safely exhaust the potentially explosive gases emitted by the batteries). By integrating the back-up power system with the improved cooling system, the present invention provides a highly efficient, low cost and relatively simple solution to many of the problems faced by data centers operators.
The above and other features of the present invention, its nature and various advantages will become more apparent upon consideration of the following detailed description, taken in conjunction with the accompanying drawings, in which like reference characters refer to like parts throughout, and in which:
Data center 100 requires two primary resources to operate: electricity and cooling. Under normal operation, electric power is provided to data center 100 via primary power, such as utility power. The majority of electric power used by data center 100 is used by the processors in the individual servers mounted in racks 102 and by air conditioning units 106. In addition, blowers 112, which are constantly running, consume most of the remaining power used by the data center. The electric power input to data center 100 is monitored by UPS 116, which may include conditioning and switching circuitry that operates to control the quantity and quality of power provided to the servers in racks 102.
In the event of a fluctuation in primary power, UPS 116 operates to maintain the delivered power as a relatively constant supply such that data center 100 does not generally experience the effects of the fluctuation. The fluctuation may include extremely short-term losses in power (such as for durations of less than one second), short-term losses in power (such as for durations of less than 15-20 minutes), long term power loss (which may extend indefinitely), and spikes or variations in the quality of input power. UPS 116 may include circuitry to ride-through or bridge extremely short-term power losses, spikes and variations, while utilizing energy from lead-acid batteries 114 for short term power losses. Long term power outages are often dealt with by external fuel-driven generators (not shown) that can operate indefinitely provided that they are refueled on a regular basis.
In addition to electricity, data center 100 must be provided with cooling due to the high levels of heat generated by the electronics in each of the servers in racks 102. In particular, the individual processors that run each of the servers in racks 102 generate a substantial amount of the excess heat that must be removed from server racks 102 to prevent overheating that would invariably lead to system shutdowns.
Data center 100 receives its cooling resources from a series of compressor-based air conditioning units 106 that may be mounted on roof 118 of data center 100. Air conditioners 106 operate on a vapor-compression refrigeration cycle which lowers the air temperature and removes moisture (excess humidity) from the air. The cooled air is provided to racks 102 via ducts 108 which direct the cooling air under raised floor 104. Server racks 102 are typically constructed such that most, if not all, of the bottom of the units are open. Raised floor 104 is installed with openings in the floor that correspond to the size of server racks 102, which are aligned over the openings. The cooled air that was provided under raised floor 104 travels as indicated by arrows 110 into the bottom of server racks 102. In addition, blowers 112 operate to pull the cooled air through racks 102.
Under normal conditions, the compressors on air conditioners 106 and blowers 112 on racks 102 are constantly running to cycle freshly-cooled air through data center 100. One deficiency in this approach, however, is the quantity of required cooling air due to the broad, relatively general application of the cooling air to the source of heat (i.e., the processors in the servers (in addition to the large demands for electric power to run the compressors and blowers).
Data center 200 is provided with cooling resources in a more directed and efficient manner than for conventional data centers, such as data center 100. Cooling is provided to data center 200, and specifically to server racks 202 and 204, via a closed-loop air-based system (versus the general application cooling via open-loop systems in conventional data centers). The air is constantly maintained in a pressurized state so there is no need for compressors. The cooling system operates by varying the temperature of the pressurized air in various locations throughout the closed-loop such that relatively cool pressurized air is provided to the processors, which heats the air. It is understood that the air used in embodiments according to the invention need not be limited to atmospheric air, but can be any type of working fluid such as, for example, nitrogen, helium, or hydrogen.
The heated pressurized air is then passed through heat exchangers 208 which remove the excess heat and direct the cooler pressurized air back to the server racks again. Heat exchangers 208 may be any type of conventional heat exchangers, including, for example, a conventional chilled water system. Thus, while heat exchangers 208 are shown on roof 216 of data center 200, if a chilled water system is utilized as the system heat exchanger the closed-loop will pass through chilled water 212 and roof-top heat exchangers may not be necessary. In any event, air flow throughout the closed-loop system is maintained by one or more hermetically sealed (or semi-hermetically sealed) gas pumps (see element 306 of
While the cooling system is, in and of itself, a closed-loop system, it may be possible to utilize heated pressurized air from the closed-loop system to drive an energy backup system in accordance with the principles of the present invention (and which is described more fully below with respect to
System performance is primarily dependent upon four factors: the difference in the temperature of the air in the closed-loop just prior to entering and leaving server racks 202, the difference in temperature of the pressurized air in the closed loop entering and leaving heat exchangers 208, the average or nominal pressure of the air in the closed loop, and fluid flow pressure losses that determine pumping power requirements. For example, the system will operate more effectively if the air in the closed-loop is maintained at 6000 PSI than if it were maintained at 1000 PSI, but air at 1000 PSI is likely to be more readily available or easier to provide than 6000 PSI air. In addition, while the present invention may be practiced with air-to-air heat exchangers to reduce the temperature of the air in the closed-loop, it may be more effective to pass the conduits carrying the pressurized air through air-to-liquid heat exchangers within water holding tank 210 and/or chilled water container 212.
Water holding tank 210 may simply be a tank of water that acts to cool the air passing through the submersed conduit. In addition to, or instead of water holding tank 210, chilled water container 212 (and the associated conventional chilled water system (not shown)) may be used to reduce the temperature of the air passing through the conduit in the closed-loop. Under these circumstances, a chilled water system that may be located external to the data center building produces chilled water that is circulated through container 212. The closed-loop conduit is submersed in the chilled water and air passing through the submersed portion of the conduit is thereby further cooled.
It should be noted that there may be circumstances in which the water in tank 210 might not be cooler than the pressurized air in the closed-loop, such as when the system has been running all day and the outside air is relatively hot. In this instance, once the sun has set and the external temperature goes down, the pressurized air may act to reduce the temperature of the water in tank 210 (so that the water acts, in essence, as a thermal capacitor that stores cooling energy at night and discharges it during the day).
Assuming that optional water tank 210 and water container 212 are included, cooling occurs in the following manner. Pressurized air is driven by a sealed pump (see element 306 of
The server includes an input port and output port (see, for example,
Data center 200 of the present invention provides many advantages over data center 100, including a significantly reduced demand for electricity (due, in large part, to the elimination or reduction of the compressors in the air conditioners and the blowers in the server racks), significantly reduced noise (due to the elimination of the blowers) and more effective method of providing the necessary cooling. The closed-loop pressurized air system is capable of removing more heat from the data center more effectively because it is in direct thermal contact with the main heat generating components, the processor(s), as opposed to passing a large quantity of cooling air through server rack 102, most of which has virtually no contact with the heat generated by the processor(s).
Persons skilled in the art will appreciate that the present invention may also be accomplished, albeit in a somewhat less effective manner, by utilizing the closed loop pressurized air cooling system shown in
While this configuration may be less effective at removing heat from the processors in server racks 202 than heat sink in thermal contact with the closed loop, this configuration does provide the advantages of eliminating the need for a water loop and/or vapor-compression air conditioning system used in conventional data centers. This configuration may be more practical for use with existing data centers in which servers cannot be easily modified for use with processor heat sinks. Moreover, by utilizing this configuration of the present invention, data center operators would be able to migrate to the heat sink/thermal contact configuration as such servers became commercially available, because the closed loop pressurized air system would already be in operation.
Data center 300, which is constructed and operated in accordance with the principles of the present invention, includes server 302 (mounted in server rack 318), heat exchanger 304, sealed pump 306, air supply 308, optional water holding tank 310, optional chilled water container 312, processor heat sink 314 and optional turbine-based back-up energy subsystem 316. Cooling resources are provided to server 302 in a manner similar to that described previously with respect to data center 200.
In the event of a disruption in primary power, in which case energy subsystem 316 draws off a portion of the heated pressurized air, the closed-loop of pressurized air is maintained by air supply 308. In all circumstances, the pressurized air is circulated through the system by sealed pump 306. Pressurized air having a relatively ambient temperature is passed through conduit submerged in water tank 310 and then through submerged conduit in chilled water container 312, after which it enters server rack 318 and is directed to an input port (not shown) on server 302. The pressurized air then passes through heat sink 314 (which is in thermal contact with the processor(s) and other heat generating components, such as power converters, in server 302) which raises the temperature of the pressurized air and lowers the temperature of the processor(s) and associated components.
The heated pressurized air then passes through valve 320 and on to heat exchanger 304 which is preferably installed exterior to physical building 322 that houses most of the components of data center 300. Moreover, while sealed pump 306 is shown exterior to building 322, persons skilled in the art will appreciate that sealed pump 306 may be installed at any location in the closed-loop, regardless of whether the location is inside or outside building 322.
Persons skilled in the art will appreciate that valve 320 could instead be located on the input-side of heat sink 314, in which case turbine 316 would be driven by relatively cool air instead of heated air. During a fluctuation in primary power, the otherwise closed-loop circulates pressurized air to cool server 302 while air in tank 308 is supplied to compensate for any system air used to drive turbine 316. Under these circumstances, the exhaust air from turbine 316 would be particularly cool which would help to cool server 302 and any other heat generating components in server rack 318.
In the event of a fluctuation in primary power to data center 300, valve 320 may be opened to direct some portion of the heated pressurized air to turbine-based back-up energy system 316. The heated pressurized air causes the turbine to operate which drives a generator (which is a part of system 316) to produce short-term back-up energy. In addition, energy system 316 may also include an extremely short-term energy storage device, such as a capacitor (once again, within system 316), which can provide bridging power to server 302 for the interim period prior to the turbine reaching operational status.
The advantages of data center 300, in addition to the previously described advantages with respect to data center 200, are related to the addition of an energy back-up system that needs no additional system resources to operate. The “fuel” to drive the turbine—hot pressurized air—is a normal by-product of the cooling system of data center 300. Moreover, as shown in
This also provides the additional advantage of ease of scalability of the data center. Instead of having to add new air conditioning units and additional back-up batteries, etc., each new server rack is installed with its own back-up energy system. In addition, because each additional server uses significantly less cooling resources, cooling for many more servers can be provided by the system before an additional heat exchanger is needed (and the addition of a heat exchanger is significantly less expensive to purchase and operate than an air conditioner).
Energy system 500 includes rack-mount drawer 502, turbine 504, generator 506, electronics 508, input port 510 and wires 512. Rack-mount drawer 502 should be constructed to be the width and depth of a standard server rack so that it may be easily installed therein. It may be preferable to mount system 500 in the lower portion of the server rack, such as the bottom “drawer,” since the height of the rack will likely be larger than a standard server drawer.
System 500 operates in accordance with the principles of the present invention by receiving heated pressurized air via input port 510. The heated air enters port 510 once the valve (shown in
Once turbine 504 is spinning at an appropriate speed, generator 506 will produce short-term back-up power that is supplied to electronics 508 for distribution to the server rack via wires 512. In the event of a long-term failure of primary power, electronics 508 can send a signal to the data center which causes external, fuel-burning back-up generators (not shown) to come on-line. These generators can run indefinitely provided that they are refueled on a regular basis.
Thus it is seen that a closed-loop pressurized air system can be used to provide an improved level of cooling resources to a data center. In addition, the heated pressurized air of the closed-loop can be used as “fuel” to drive a turbine-based energy back-up system. Persons skilled in the art will appreciate that the present invention can be practiced by other than the described embodiments, which are presented for purposes of illustration rather than of limitation, and the present invention is limited only by the claims which follow.
