The present disclosure relates generally to modular electronic systems, and more particularly, maintaining cooling of modular electronic systems during module replacement.
Modular electronic systems are designed to provide flexibility to configure systems as per user needs. These systems typically have multiple slots to accommodate a variety of modules (e.g., line cards, service cards, fabric cards, and the like). Most of these modules can be replaced with the latest product upgrades without disturbing normal operation of the system (i.e., hot swappable). It is desirable to replace faulty modules without powering down the system or impacting the performance of other modules in the system, such as by disrupting system cooling.
Corresponding reference characters indicate corresponding parts throughout the several views of the drawings.
In one embodiment, a method generally comprises receiving an indication at a modular electronic system of initiation of online removal for a module removably inserted into a slot of the modular electronic system, increasing a fan speed at the modular electronic system before the module is removed, monitoring an internal temperature at the modular electronic system, and providing an indication that the module is ready for removal upon reaching a specified cooling state at the modular electronic system based on the temperature monitoring. A panel on an adjacent module is opened and extends into the slot upon removal of the module to substantially block airflow bypass from the slot and maintain cooling within the modular electronic system.
In another embodiment, an apparatus generally comprises a frame supporting a plurality of slidably removable modules, at least a portion of the modules each comprising a panel movable between a closed position and an open position in which the panel extends into an open slot from which a module is removed, a cooling fan for cooling said modules, a thermal sensor for sensing an internal temperature at the apparatus, and a processor configured to receive an indication of an online removal process for the module, increase a speed of the cooling fan, and provide an indication that the module is ready for removal upon identifying that the internal temperature has reached a cooling state. The panel on an adjacent module is configured to open and extend into the slot upon removal of the module to substantially block airflow bypass from the slot and maintain cooling within the modular electronic system.
In yet another embodiment, logic is encoded on one or more non-transitory computer readable media for execution and when executed by a processor operable to identify initiation of an online removal process for a module removably inserted into a slot of a modular electronic system, increase a fan speed at the modular electronic system before the module is removed, monitor an internal temperature at the modular electronic system, provide an indication that the module is ready for removal upon reaching a specified cooling state at the modular electronic system based on said monitored temperature, and unlock a panel on an adjacent module so that the panel is free to open and extend into the slot upon removal of the module to substantially block airflow bypass from the slot and maintain cooling within the modular electronic system.
The following description is presented to enable one of ordinary skill in the art to make and use the embodiments. Descriptions of specific embodiments and applications are provided only as examples, and various modifications will be readily apparent to those skilled in the art. The general principles described herein may be applied to other applications without departing from the scope of the embodiments. Thus, the embodiments are not to be limited to those shown, but are to be accorded the widest scope consistent with the principles and features described herein. For purpose of clarity, details relating to technical material that is known in the technical fields related to the embodiments have not been described in detail.
Network devices such as switches, routers, server racks, or other electronic devices may be configured as a modular electronic system with a plurality of removable modules (e.g., service cards, line cards, fabric cards, or other cards, components, or modules). In a modular configuration system, Online Insertion & Removal (OIR) refers to a process of replacing a module for repair or replacement (e.g., product upgrade, failed module replacement) without disrupting the performance of the modular system. During this process, the system continues to be functional and a faulty module is replaced with a new or repaired module.
In an air-cooled modular electronic system, cooling air is supplied to each of the modules to maintain a normal cool state of each module. When a module is removed from the system, a large amount of cooling air flows into a space (slot) from which the module has been removed and the cooling balance among the remaining modules is impacted. The open slot causes the system airflow to be unbalanced as the empty space provides a path of least resistance and airflow is allowed to bypass through this space resulting in other areas or components failing to receive sufficient airflow for cooling.
Depending on the complexity of the system design, the OIR process may take some time, during which the module slot would remain open to ambient air rushing in the now open slot. In a fan cooled system that is optimized for uniform flow through each of the slots, airflow would be negatively impacted as the slot opening would offer the least resistance airflow path and most of the air would start flowing through that open slot. This would reduce the airflow for other modules, thereby causing their component temperatures to rise, which often leads to overheating, and may potentially result in shorter life span or catastrophic failure of one or more of the operational modules remaining in the system.
In one example, it is desired to extend the OIR process to at least five minutes. This means that the open slot could be allowing air inrush for up to five minutes, starving air flow from the other cards, and possibly causing system failure. Meeting five minutes OIR times with hardware components operating above 80 Watts, for example, is increasingly difficult, if not impossible with conventional systems typically only allowing for replacement times of two minutes or less before overheating occurs.
The embodiments described herein provide for maintaining of cooling of modular electronic systems during module removal. As described in detail below, one or more embodiments provide cooling logic and an airflow panel to prevent the loss of cooling to modular components when modules are removed. In one or more embodiments, cooling fan logic may be used to increase cooling prior to removal of an OIR module to prevent the loss of cooling to the active modules during OIR. In one or more embodiments, in order to avoid airflow bypass from the open slot, a panel (airflow panel, flap, door) is provided, which is operational during OIR conditions and configured to prevent airflow bypass during module OIR. One or more embodiments may improve the OIR time limit for an extended period of time and possibly indefinitely, thereby providing high reliability and availability of modular electronic systems.
The embodiments described herein may operate in the context of a data communications network including multiple network devices. The network may include any number of network devices in communication via any number of nodes (e.g., routers, switches, gateways, controllers, edge devices, access devices, aggregation devices, core nodes, intermediate nodes, or other network devices), which facilitate passage of data within the network. One or more of the network devices may comprise a modular electronic system as described herein. The network devices may communicate over one or more networks (e.g., local area network (LAN), metropolitan area network (MAN), wide area network (WAN), virtual private network (VPN) (e.g., Ethernet virtual private network (EVPN), layer 2 virtual private network (L2VPN)), virtual local area network (VLAN), wireless network, enterprise network, corporate network, data center, Internet, intranet, radio access network, public switched network, or any other network).
Referring now to the drawings, and first to
The modular electronic system 10 shown in
One or more removable modules 16 comprise an OIR button 19a used in removal of the module 16. As described in detail below, the OIR buttons 19a may be in communication (e.g., mechanically, electronically, or both) with cooling logic 18a to initiate increased cooling of the modules 16 prior to removal of the OIR module, and with panel logic 18b to unlock an airflow panel on a module adjacent to the module to be removed to prevent airflow bypass once the module is removed.
The term OIR or online removal as used herein refers to removal of a module from the modular electronic system without removing power from the system (i.e., one or more remaining modules are operational). The module 16 removed from the system for replacement, repair, or upgrade is referred to herein as the “OIR module”. The module 16 (or modules) that are thermally impacted due to airflow bypass from the open slot left by the OIR module is referred to herein as the “impacted module”.
In one or more embodiments, an indication may be provided to notify a user when it is acceptable to remove the OIR module after initiation of the online removal process. For example, the modular electronic system may comprise a display or light on the chassis itself or each module. In the example shown in
As described below, a fan speed may be increased once the OIR button 19a is pressed. One or more internal thermal sensors 18c in the modular electronic system 10 may be monitored to determine when the internal temperature has stabilized or reached a specified temperature, for example. When this cooling state is reached, the OIR indicator 19b may change color to indicate that the OIR module can be safely removed. Once the OIR module is removed, an airflow panel located on a module adjacent to the OIR module may be deployed to prevent airflow bypass through the open slot, as described below with respect to
In the example shown in
In order to start the OIR process, maintenance personnel may select the OIR button 19a (e.g., button mechanically depressed, tab pressed, lever moved or mechanically engaged, or other input at a selectable interface) on the module to be removed (FC2-S2 card in the example of
In one or more embodiments, the modules 16 may be physically locked in placed (e.g., ejector locked) until the temperature has stabilized after the fan speed has increased. Once the modules are sufficiently cooled, the OIR module may be physically unlocked or released (e.g., ejector unlocked) at approximately the same time (or slightly before) the OIR indicator 19b changes state to indicate that the OIR module is ready for removal.
As described in detail below, after the OIR process is initiated, the airflow panel 20 may be unlocked by panel logic 18b so that it is ready to deploy into the open slot when the OIR module is removed (
It is to be understood that the modular electronic system 10, modules 16, OIR button 19a, and OIR indicator 19b shown in
Memory 34 may be a volatile memory or non-volatile storage, which stores various applications, operating systems, modules, and data for execution and use by the processor 32. For example, components of the cooling/panel element 38 (e.g., code, logic, software, firmware, etc.) may be stored in the memory 34. The network device 30 may include any number of memory components.
Logic may be encoded in one or more tangible media for execution by the processor 32. For example, the processor 32 may execute codes stored in a computer-readable medium such as memory 34. The computer-readable medium may be, for example, electronic (e.g., RAM (random access memory), ROM (read-only memory), EPROM (erasable programmable read-only memory)), magnetic, optical (e.g., CD, DVD), electromagnetic, semiconductor technology, or any other suitable medium. In one example, the computer-readable medium comprises a non-transitory computer-readable medium. The processor 32 may be operable to perform one or more steps shown in the flowcharts of
The cooling/panel logic 38 may comprise one or more components (software, code, logic) operable to monitor module presence, receive input from OIR button 19a, receive input from thermal sensors 18c, control fan speed based on ambient temperature sensor and thermal sensor of critical modules 16, activate unlocking of panel 20, and lock/unlock module ejectors (
The network interfaces 36 may comprise any number of interfaces (connectors, line cards, ports) for receiving data or transmitting data to other devices. The network interface 36 may include, for example, an Ethernet interface located on one of the modules 16 for connection to a computer or network.
It is to be understood that the network device 30 shown in
When the temperature has stabilized to the specified cooling state, the OIR module 16 is ready to be removed and an indication is provided (step 44). An indication that the modular electronic system (e.g., module, portion, or section of modular electronic system) has reached its cooling state may comprise, for example, a visible indication (e.g., a change in color of a light (e.g., LED turning off, changing color, or blinking light turning solid)), or unlocking of a physical lock (e.g., module ejector unlocked or released). A lock on the airflow panel 20 may also be released. The lock may be released at the same time that the LED 19b indicates the OIR module 16 is ready to be removed or may be released as soon as the OIR button 19a is pressed. Once the OIR module 16 is removed, the airflow panel on an adjacent module is free to be released from the adjacent module and deployed into the open slot 22 to prevent air bypass from the slot and maintain cooling within the modular electronic system (step 46).
At step 53, the temperature sensors have stabilized and the OIR LED 19b on the OIR module 16 changes (e.g., turns green or solid blue light) and the module ejector is unlocked or released. The lock on airflow panel 20 on an adjacent module is also released. The OIR module 16 may then be pulled out thereby releasing the panel on the adjacent module so that it covers the opening in the slot left behind by the removed OIR module (step 54). The system may then run for an extended period (or possibly indefinitely). At step 55, a replacement module 16 is inserted into the open slot 22. The panel 20 is closed and locked in place when the replacement module is inserted into the slot due to inward movement of replacement module (step 56). The module power is turned on and the module status LED turns green (step 57). The fan speed may be adjusted based on the ambient temperature (step 58). The system returns to normal operation, the ejector on the module is locked, and the OIR LED on the module is turned off (step 59).
It is to be understood that the processes shown in
As previously described with respect to
In one embodiment, a servo motor 100 engages with the spring lever 97 to release the locking pin 98 from the spring lever. In one example, the miniaturized servo controlled motor 100 may be connected to the spring lever 97 to provide rotation needed for the spring lever (held in place by spring 99) to release locking pin 98. The servo motor 100 provides angular rotation to move the spring lever 97 thereby allowing the locking pin 98 and connected hinge lever 93 to move along the slotted bracket 96 and open the panel 90 to cover the open slot left by the removed module in the adjacent slot.
As shown in the cut-out view, the locking pin and hinge lever 93 are released and move along with the panel 90 when the panel is free to move to its open position when the adjacent module is removed. As previously described, the panel 90 is spring loaded by torsion spring 95 to move to its open position. In one embodiment, the opening of the panel 90 may be automated with the action of the servo motor 100 such that the panel is resting against an adjacent module when unlocked, opening fully when the module 16 is removed. The spring lever 97 may return to its spring biased position after the locking pin 98 is released.
The servo motor 100 may be activated as soon as the OIR button 19a on the module to be replaced is pressed (
The panel 90 is pushed back into its locked position when the adjacent module 16 is inserted back into the open slot 22 (
Additionally, filament 102 may also provide a thermo-mechanical “backup” protection feature where by setting a suitable metallurgical phase transition temperature filament 102 may be caused to pull lever 97 if local system operating temperature reaches a predetermined temperature point, typically within the waiting period shown in
It is to be understood that the mechanisms for unlocking the panel 90 shown in
The following describes results of experiments showing the thermal impact on the impacted module when the OIR module is removed. Without a waiting period or airflow panel 20 deployed, the temperature on the impacted module (FC6-S2 card in the example of
As can be observed from the foregoing, the embodiments described herein may provide numerous advantages. For example, one or more embodiments allow more time for module replacement and allow for replacement times beyond conventional OIR times. In one or more embodiments, the airflow panel avoids air bypass through the slot opening of the removed module, thereby improving thermal management of the system. In one or more embodiments, cooling logic helps to maintain cooling within the modular electronic system by pre-cooling adjacent modules before removal of the OIR module.
Although the method and apparatus have been described in accordance with the embodiments shown, one of ordinary skill in the art will readily recognize that there could be variations made to the embodiments without departing from the scope of the invention. Accordingly, it is intended that all matter contained in the above description and shown in the accompanying drawings shall be interpreted as illustrative and not in a limiting sense.
The present application is a divisional application of U.S. patent application Ser. No. 16/819,399, filed Mar. 16, 2020, titled “METHOD AND APPARATUS FOR MAINTAINING COOLING OF MODULAR ELECTRONIC SYSTEM DURING MODULE REMOVAL”, which is a divisional application of U.S. patent application Ser. No. 15/639,968, filed Jun. 30, 2017, now U.S. Pat. No. 10,631,424, the contents of which are incorporated by reference herein in their entireties and for all purposes.
Number | Name | Date | Kind |
---|---|---|---|
4642044 | Ishikawa | Feb 1987 | A |
5722981 | Stevens | Mar 1998 | A |
5773755 | Iwatare | Jun 1998 | A |
6307738 | Tran | Oct 2001 | B1 |
6816975 | Sasaki | Nov 2004 | B1 |
7344439 | Henry | Mar 2008 | B2 |
7843683 | Suffern | Nov 2010 | B2 |
8678524 | Green | Mar 2014 | B2 |
9301431 | Esmaily | Mar 2016 | B2 |
10631424 | Gupta et al. | Apr 2020 | B2 |
11172587 | Gupta | Nov 2021 | B2 |
20010006453 | Glorioso | Jul 2001 | A1 |
20030033393 | Larson | Feb 2003 | A1 |
20060048001 | Honda | Mar 2006 | A1 |
20060101054 | Dempski | May 2006 | A1 |
20070132550 | Avraham | Jun 2007 | A1 |
20090056359 | Germagian | Mar 2009 | A1 |
20110273835 | Katakura | Nov 2011 | A1 |
20130083481 | Goto | Apr 2013 | A1 |
20130155612 | Korikawa | Jun 2013 | A1 |
20140244881 | Stewart | Aug 2014 | A1 |
20140360232 | Al-Kahwati | Dec 2014 | A1 |
20150098180 | Berghe | Apr 2015 | A1 |
20150300047 | Leong et al. | Oct 2015 | A1 |
20160205812 | Curtin | Jul 2016 | A1 |
20170076575 | Kunnathur Ragupathi et al. | Mar 2017 | A1 |
20170318707 | Shabbir | Nov 2017 | A1 |
20180046226 | Bowers et al. | Feb 2018 | A1 |
20180082343 | Gordon | Mar 2018 | A1 |
20180148173 | Schwartz et al. | May 2018 | A1 |
20180217644 | Dan | Aug 2018 | A1 |
20180223573 | Wolfe | Aug 2018 | A1 |
20180310437 | Anderl | Oct 2018 | A1 |
Number | Date | Country | |
---|---|---|---|
20210410312 A1 | Dec 2021 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 16819399 | Mar 2020 | US |
Child | 17470465 | US | |
Parent | 15639968 | Jun 2017 | US |
Child | 16819399 | US |