The present invention relates to system management of AdvancedTCA and microTCA compliant communication and computing equipment architectures. In particular, the present invention relates to fault tolerant, multiprotocol shelf management controller architectures that are extensible, programmable, interface indifferent and reconfigurable.
The telecom marketplace is being shaped by exploding data rates, near-convergence of video, data, voice, wireless and VoIP and a global demand for networks and telecom applications characterized by high availability, optimum performance and reduced capital expenditures. One of the hallmarks of this marketplace is the vast array of disruptive technologies that rapidly obsolete competing legacy technologies. In the wake of the recent telecom retrenchment after the burst of the Internet boom, telecom equipment manufacturers have responded to this new wave of disruptive technologies by adopting standardized architectures over proprietary solutions.
The AdvancedTCA (Advanced Telecom Computing Architecture, hereinafter “ATCA”) is one of the new standards gaining widespread acceptance for design and deployment of next generation of telecommunication and computer networking equipment. The core ATCA specification adopted by the PCI Industrial Computer Manufacturers Group (PICMG 3.0) is a chassis, backplane, power, shelf management, clock and base interface specification targeted to the next generation of telecommunications and networking equipment. The ATCA specification covers the details of the mechanical construction, the system management, power distribution, data transport, thermal loading and power connectors to provide an open communication architecture framework for building next generation high-availability, high density systems.
The ATCA specification supports high-speed interconnections, active monitoring and control of the various modules with reliability exceeding “five-nines” (99.999%) availability. One of the features that makes it possible to achieve such a level of reliability, is a mandatory shelf management component that addresses monitoring and control of thermal load, power handling, and system health in general at a shelf level. The shelf management component is governed by extensive requirements set forth in the PICMG and Intelligent Platform Management Interface (IPMI) specifications well known in the art. The IPMI comprises a main controller called the Baseboard Management Controller (BMC) and remote controllers. The remote controllers within the same chassis are connected to the BMC via the system interface called IPMB (Intelligent Platform Management Bus/Bridge). IPMB is an enhanced implementation of I2C (hereinafter “I2C” is an acronym for Inter Integrated Circuit bus). The BMC connects to remote controllers or another BMC in another chassis via IPMC (Intelligent Platform Management Chassis) bus/bridge.
I2C (hereinafter “I2C”) is an acronym for Inter Integrated Circuit bus. I2C is a 2-wire serial interface standard defined by Philips Semiconductor to provide an easy way to connect a CPU to peripheral chips. The bus physically consists of 2 active, bi-directional wires (the serial data line and the serial clock line) and a ground connection that allows full duplexed communication between a plurality of devices. The I2C interface is a simple master/slave type interface wherein a device that controls signal transfers on the line as well as the clock frequency is designated the master and a device that is controlled by the master is designated the slave. The master can transmit or receive signals to or from a slave, respectively, or control signal transfers between two slaves, where one is the transmitter and the other is the receiver. I2C bus supports more than one master connected to one bus. In typical cases, the microcontroller acts as the master and the external peripheral devices act as the slaves.
A shelf manager is mandated by the AdvancedTCA specification and therefore, must be capable of being integrated with other modular components to construct platform-independent, telecommunication and networking equipment solutions. However, it will readily be evident to one skilled in the art that a shelf may include diverse features and functions provided by a combination of proprietary and open communication standards based devices, cards, blades or other similar components. Typically, the proprietary features and functions are provided by using dedicated hardware components/systems or software specifically tailored for the particular shelf configuration. Conventional shelf manager solutions lack the flexibility to accommodate changes in the standard of either the shelf manager or the components that are part of the shelf without extensive redesign. It would be advantageous to provide a low cost system capable of being configured for transparent porting of legacy as well as non-legacy applications. In particular, these approaches do not offer a cost advantage for MicroTCA standard compliant applications because the MicroTCA standard is geared toward smaller scale and more price sensitive applications.
The present invention is a low-cost, fault tolerant shelf management controller used for shelf management of AdvancedTCA and the MicroTCA specification compliant shelf architectures that can be reconfigured for legacy as well as non-legacy applications resulting from multiple versions of an evolving specification.
The shelf management controller is arranged in a dual redundant configuration in an active-standby mode, alternately referred to as the master-slave, server-client. In accordance with the AdvancedTCA specification, the shelf management controller architecture is specified by the Intelligent Platform Management Interface (IPMI) specification as extended by PICMG 3.0. In one embodiment, each shelf management controller includes at least one bit stream processor comprising sequenced stage machines implementing one or more finite state machines associated with one or more devices that are under control of the shelf management controller. The finite state machines may be hardware or software based.
In another embodiment, the shelf management controller is modeled as a layered architecture which includes at least one IPMI API layer. The IPMI API layer enables the shelf manager to adapt to legacy and future IPMI specifications. The API also insulates the controller code from the need to know the details of the interfaces used to communicate with other Baseboard Management Controllers (BMC)s.
In another embodiment, the shelf management controller architecture is configured into a layered architecture with the layered functionality provided by a combination of reconfigurable hardware and software architecture that lends itself to a plurality of controllers including, for example, the modular management controller (MMC), the Carrier Intelligent Platform Management Controller (Carrier IPMC), the Shelf Management Controller (ShMC) and the MicroTCA Carrier Management Controller.
The invention may be more completely understood in consideration of the following detailed description of various embodiments of the invention in connection with the accompanying drawings, in which:
While the invention is amenable to various modifications and alternative forms, specifics thereof have been shown by way of example in the drawings and will be described in detail. It should be understood, however, that the intention is not to limit the particular embodiments described. On the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention as defined by the appended claims.
The present invention will be described in conjunction with the figures. In the various figures, like reference numerals refer to the same feature of the invention.
Referring now to
The IPMI is a standardized, abstracted, message-based interface to intelligent platform management hardware. IPMI protocol is defined in detail in Intelligent Platform Management Interface (IPMI) specification, Version 1.5, and Version 2.0 developed by a consortium including Intel Corporation, Hewlett-Packard Company, NEC Corporation, and Dell Computer Corporation, the disclosures of which are hereby incorporated herein in their entirety. One embodiment of the present invention is configured for adapting to the IPMI upgrade path including, for example, IPMI version 1.5 and version 2.0 and the IPMI standard as extended by PICMG 3.0 including the Intelligent Platform Management Interface 210, the Intelligent Platform Management Bus (IPMB) 215, the Intelligent Chassis Management Bus (ICMB) 220, the Shelf Management Controller (SHMC) 230, and the Intelligent Platform Management Controllers (IPMC) 235. Alternate embodiments of the present invention, envisage adaptation to other standards and specifications.
In one embodiment, each FRU 130 includes an IPMC 235 which implements a standardized interface based on IPMI v 1.5 for example. In a secondary embodiment, each FRU 130, such as a voltage or a thermal sensor, maybe managed by a IPMC 235. The IPMC 235 on the shelf manager is referred to as the shelf management controller (SHMC) 240. One of skill in the art will recognize that the term “shelf manager controller (ShMC)” is ATCA terminology designator for a device that hosts the IPMI firmware and resides within an ATCA shelf 120. Using the terminology of the ATCA specification, the BMC controller 240 would be termed an IPMC when it resides on an ATCA carrier board 250 or an MMC (Modular Management Controller when it resides on an Advanced Mezzanine Card (AMC) 255 hosted on the ATCA carrier board 250 without departing from the scope of the present invention. The ICMB 260 is a specialized bus that is used for chassis-to-chassis communication.
The IPMB 215 interconnects the IPMCs 235 with the ShMC 240. In one embodiment, this bus is physically a I2C-Bus, which is a two wire serial interface used for intra-chassis communication between the ShMC 240 and the IPMCs 235. Shelf-to-Shelf communication maybe obtained through the ICMB 260. In an alternate embodiment, the ShMC 240 provides the interface between the Shelf-External System Manager software 265, the IPMCs 235 and the ICMB 260 to enable management of the shelf as a single system. This includes autonomous monitoring, event logging and recovery control. In an exemplary embodiment, the IPMB 215 (called the IPMB-0) is constructed dual redundant with every IPMC 235 connected to the SHMC 230 via a IPMB-A 270 and an IPMB-B bus 275. Each of the IPMI/IPMB-A and IPMI/IPMB-B topologies maybe configured as either radial or bussed as shown in
The ATCA specifications to which the present invention is compatible, require the shelf manager to be implemented in a dual redundant configuration for conformance to the specification i.e. two shelf management controllers and dual redundant management bus to ensure fault tolerance. The outside-the-shelf system management software 265 interacts with the SHMC 230 using the Internet Protocol (IP) which is the primary network interface to outside-the-shelf entities. The Remote Management and Control Protocol (RMCP) is the messaging mechanism that provides IPMI over LAN functionality thereby enabling the system managers to communicate with ShMCs and the FRUs. The IPMI standard provides a specification for the IPMC (and ShMC) command sets, including command sets for sensors, event logs, and sensor data record access along with the specification for the data formats. The data forms utilized by IPMI include sensor data records, event log entries, and field replaceable unit (FRU) inventory entries. In one embodiment of the present invention, the layered architecture model provides an IPMI API layer that interfaces with a I2C device and in another embodiment, the IPMI API layer interfaces with a RMCP device (IPMI over LAN). Other protocols besides the I2C and RMCP may be used without digressing from the scope of the present invention.
As depicted to
Referring again to
In the illustration embodiment of
In an alternate embodiment, instead of the Bit Stream Processor there may be provided a CPU or other processor including an Ethernet controller (and/or an I2C controller) and equipped with a software stack that implements IPMI and RMCP. When a RMCP message packet is received from the system manager, the packet is opened and examined for UDP Port #. If the UDP Port # matches the IPMI message, the packet is stripped of its network header and a send message header (if any) is encapsulated. Then the message is sent to the appropriate interface. The SHMC kernel can request a “Copy Back” in a specific embodiment of the present invention. An IPMI message to the System Manager is encapsulated and sent over the System Manager Physical Port.
In one embodiment, the HFSM 475 includes the BSP 440 configured with a selected sequence of pipelined stage engines. Each stage engine may have a different, extensible and reprogrammable architecture that causes an instantiation of a device finite state machine (DFSM) 480 for each IPMC 235 transmitting a message (e.g. system health, temperature, fan revolution etc) to the HFSM 475. The DFSMs 480 are advantageously configured for data flow communication to a stage engine of the BSP 440 adapted to instantiate a messaging finite state machine (MFSM) 485. Generally, the HFSM (as well as the DFSMs and the MFSM) uses three basic constructs. The HFSM maintains an action table that contains the action to perform when a given event is received while the FSM is in a given state, a next state table which contains the next state to enter when a given event is received while the FSM is in a given state and an event handler which drives the event processing when presented with an event, looks up and performs the necessary actions and updates the current state information. The stage machine (or the BSP or the FPGA) control and status register files are accessible through a Register Access Control (RAC) 487 mechanism whereby IPMI encapsulated messages are directed to the microprocessor in the stage machine (or BSP or FPGA) who then performs the actual register read or write. The microprocessor acts as a Register Access Controller (RAC) who interprets the RAC message, determines which forwarding logic element/Sub-module Access Controller (SAC) 489 the message is addressed and facilitates the register access with the SAC. Resulting status/response is return to the message originator. The RAC/SAC 487/489 provides a means to set or change the messaging methods per device (i.e. IPMC 235) on-the-fly, thus providing one mechanism that implements the level of programmability and flexibility of the present invention.
In one embodiment, the HFSM 475 is adapted to detect I2C bus failure as well as a device failure. If the failure is determined to be on a device monitored by one of the IPMCs 235, the SHMC 310 (315) isolates that device from accessing the backplane.
Referring again to
In a secondary embodiment, the ShMC 310(315) is augmented by a thin hardware assisted protocol stack. Another embodiment of the system implements an OS bypass scheme to assure a tiny and manageable ShMC implementation. The primary embodiment includes a EEPROM to execute instructions, such as for example an EEPROM with a TINY CHIP using system-on-chip (SOC) concepts, that would enable cost wise scaling of the capabilities of the ShMC processor 320.
In the primary embodiment, the dual redundant SHMC 310(315) configuration is used to introduce fault tolerant operation of the shelf management controller. In a first embodiment, checkpoints are inserted by adding an additional checkpoint state in the HFSM 475. When a current state in the HFSM 475 is the checkpoint state, a checkpoint process maybe initiated. On errors being indicated, the HFSM 475 may initiate a failover to ShMC 315 over the exclusive-use bus 335 and a recovery process initiated on ShMC 310 without introducing an abnormality in the ATCA shelf. The recovery process may be done by restoring faulty states internal to the SHMC 310 by replaying the logged states stored on SHMC 315 in their original order to recreate ShMC 310's pre-failure state. In a secondary embodiment, an additional ShMC (not illustrated) may be used to augment ShMC 310(315) and the correct state is obtained by voting among the three or more copies of the states held between the three ore more ShMCs. In an exemplary embodiment, the voted results are loaded into the registers of each of the HFSMs 475 for purposes of resolving any conflicting votes.
In an alternate embodiment, the IPMI API 510 is used to provide an IPMI v1.5 management infrastructure compliant with PICMG 3.0 within the shelf, but presenting the functionality of one or more of IPMI 1.0, 1.5, 2.0 and other evolving versions at its external management interface thus allowing an upgrade path to users whose needs cannot be met by the existing standards. In an exemplary embodiment, the aforementioned functionality is provided by IPMI middleware 610 that represents a software agent running in the OS 325. The middleware is a software-provided abstraction of the I2C/IPMI (or other interfaces such as JTAG, Update Channel, RAC/SAC) interface from the device IPMCs 235 that takes a IPMI version x based message and translates it into a consistent internal-to-the-shelf format (e.g. IPMI v. 1.5) compatible for communicating with the SHMC 310 and vice versa for communications emanating from the SHMC 310. In one embodiment, the middleware is packaged into a portable library. The ShMC controller processor executes the API software in an exemplary embodiment.
One of skill in the art will appreciate that the ability to utilize dissimilar interfaces (e.g. I2C, JTAG, Update Channel, RAC/SAC and Free Space) for interfacing devices (i.e. IPMC) to the SHMC, as well as the ability to utilize different software algorithms to translate a message from one interface to another, improves the detection of errors because it allows factoring out contributions (if any) by a particular interface or algorithm to the detected error. In this regard, the present invention contemplates the use of Remote Management Control Protocol (RMCP), Remote Procedure Calls (RPC), Simple Network Management Protocol (SNMP) v1 and v3 Gets, Sets, and Trap, IPMI 1.5, CLI over Serial Port, Telnet, and SSH Secure Shell.
The above description of illustrated embodiments of the invention, including what is described in the Abstract, is not intended to be exhaustive or to limit the invention to the precise forms disclosed. While specific embodiments of, and examples for, the invention are described herein for illustrative purposes, various equivalent modifications are possible within the scope of the invention, as those skilled in the relevant art will recognize. These modifications can be made to the invention in light of the above detailed description. The terms used in the following claims should not be construed to limit the invention to the specific embodiments disclosed in the specification and the claims. Rather, the scope of the invention is to be determined entirely by the following claims, which are to be construed in accordance with established doctrines of claim interpretation.
The present invention claims the benefit of U.S. Provisional Application No. 60/761,129, filed Jan. 23, 2006, and entitled “SHELF MANAGEMENT CONTROLLER WITH HARDWARE/SOFTWARE IMPLEMENTED DUAL REDUNDANT CONFIGURATION,” and U.S. Provisional Application No. 60/743,761, filed Mar. 24, 2006, entitled “MODULAR CHASSIS-PROVIDING SCALABLE MECHANICAL, ELECTRICAL AND ENVIRONMENTAL FUNCTIONALITY FOR MICROTCA CARRIER BOARDS,” the disclosures of each of which are hereby incorporated by reference. The present invention is also related to the U.S. patent application Ser. No. 11/466,367, entitled “OMNI-PROTOCOL ENGINE FOR RECONFIGURABLE BIT-STREAM PROCESSING IN HIGH-SPEED NETWORKS,” filed Aug. 22, 2006, which is assigned to the assignee of the present invention and the disclosure of which is hereby incorporated by reference.
Number | Name | Date | Kind |
---|---|---|---|
3868158 | Laboue | Feb 1975 | A |
3874444 | Perce et al. | Apr 1975 | A |
4126269 | Bruges | Nov 1978 | A |
4187058 | Fish | Feb 1980 | A |
4453785 | Smith | Jun 1984 | A |
5031070 | Hsu | Jul 1991 | A |
5136464 | Ohmori | Aug 1992 | A |
5268637 | Liken et al. | Dec 1993 | A |
5409419 | Euchner et al. | Apr 1995 | A |
5412534 | Cutts et al. | May 1995 | A |
5528454 | Niklos | Jun 1996 | A |
5557506 | Wood et al. | Sep 1996 | A |
5680294 | Stora et al. | Oct 1997 | A |
5751549 | Eberhardt et al. | May 1998 | A |
5949646 | Lee et al. | Sep 1999 | A |
5982634 | Wronski | Nov 1999 | A |
6166917 | Anderson | Dec 2000 | A |
6181570 | Ellebrecht et al. | Jan 2001 | B1 |
6253266 | Ohanian | Jun 2001 | B1 |
6320760 | Flamm et al. | Nov 2001 | B1 |
6370605 | Chong, Jr. | Apr 2002 | B1 |
6456498 | Larson et al. | Sep 2002 | B1 |
6466449 | Sheen et al. | Oct 2002 | B1 |
6496376 | Plunkett et al. | Dec 2002 | B1 |
6504717 | Heard | Jan 2003 | B1 |
6522539 | Ota et al. | Feb 2003 | B2 |
6607402 | Cohen et al. | Aug 2003 | B2 |
6661673 | Brooks et al. | Dec 2003 | B2 |
6693901 | Byers et al. | Feb 2004 | B1 |
6704196 | Rodriguez et al. | Mar 2004 | B1 |
6708372 | Stewart | Mar 2004 | B2 |
6795314 | Arbogast et al. | Sep 2004 | B1 |
6805560 | Budny et al. | Oct 2004 | B1 |
6900387 | Gravell et al. | May 2005 | B2 |
6934786 | Irving et al. | Aug 2005 | B2 |
6935868 | Campini et al. | Aug 2005 | B1 |
6944020 | Wintersteen et al. | Sep 2005 | B2 |
6968958 | Lauchner et al. | Nov 2005 | B2 |
6985967 | Hipp | Jan 2006 | B1 |
7013352 | Garnett | Mar 2006 | B2 |
7083422 | Campini et al. | Aug 2006 | B2 |
7099160 | Ice | Aug 2006 | B1 |
7101188 | Summers et al. | Sep 2006 | B1 |
7120739 | Fujimoto et al. | Oct 2006 | B2 |
7159062 | Byers et al. | Jan 2007 | B2 |
7172432 | Campini et al. | Feb 2007 | B2 |
7200003 | Hood et al. | Apr 2007 | B2 |
7239509 | Roeske | Jul 2007 | B1 |
7259961 | Lucero et al. | Aug 2007 | B2 |
7293090 | Saleh et al. | Nov 2007 | B1 |
7324348 | Berg et al. | Jan 2008 | B2 |
7370223 | Olmstead et al. | May 2008 | B2 |
7388757 | Moakes et al. | Jun 2008 | B2 |
7509375 | Christian et al. | Mar 2009 | B2 |
7516263 | Sandy et al. | Apr 2009 | B2 |
7603580 | Zohar et al. | Oct 2009 | B2 |
20030130969 | Hawkins et al. | Jul 2003 | A1 |
20030152074 | Hawkins et al. | Aug 2003 | A1 |
20030182483 | Hawkins | Sep 2003 | A1 |
20040073834 | Kermaani et al. | Apr 2004 | A1 |
20040264145 | Miller et al. | Dec 2004 | A1 |
20050141207 | Campini | Jun 2005 | A1 |
20050280986 | Coglitore et al. | Dec 2005 | A1 |
20050281014 | Carullo et al. | Dec 2005 | A1 |
20060002098 | Berg, Jr. et al. | Jan 2006 | A1 |
20060023384 | Mukherjee et al. | Feb 2006 | A1 |
20060036793 | Sandy et al. | Feb 2006 | A1 |
20060044756 | Wong | Mar 2006 | A1 |
20060114923 | Overgaard | Jun 2006 | A1 |
20060221559 | Campini et al. | Oct 2006 | A1 |
20060221581 | DeNies et al. | Oct 2006 | A1 |
20070038732 | Chandwani et al. | Feb 2007 | A1 |
20070121306 | Moakes et al. | May 2007 | A1 |
20070217172 | Bisbikis et al. | Sep 2007 | A1 |
20080037218 | Sharma et al. | Feb 2008 | A1 |
20080062667 | Campini et al. | Mar 2008 | A1 |
20090097200 | Sharma et al. | Apr 2009 | A1 |
Number | Date | Country | |
---|---|---|---|
20070255430 A1 | Nov 2007 | US |
Number | Date | Country | |
---|---|---|---|
60761129 | Jan 2006 | US | |
60743761 | Mar 2006 | US |