The invention relates generally to processes for configuring and installing products in a data center or warehouse environment.
Companies and other large entities increasingly rely on distributed computing where many user terminals connect to one or more servers that are centrally located. These locations called “data centers” may be facilities owned by the company or may be supplied by a third-party. These data centers house not only computers, but may also have persistent connections to the Internet and thus, conveniently house networking equipment such as switches and routers. Web servers and other servers that need to be network accessible are often housed in data centers. Where a third-party owns the data center, the entity in question rents a “cage” or enclosure that has racks upon which assembled/standalone units, such as computers and routers, can be installed. The entity may also simply lease the units that are rack-mountable from the third-party. In any case, the data center is usually divided into a number of predefined areas, including a shipping/docking area, assembly area, and area where enclosures and their constituent racks are kept.
Typically, the business process of installing and configuring new computer or networking systems involves a series of independent stages. First, based on determined requirements, components of the systems are ordered through a vendor or supplier. Once the components for these systems are received, inventory logs the “asset” tag for the component which identifies it for future reconciliation/audits. While the order for the components themselves may identify a number of attributes that each component should have (i.e. amount of memory, number of ports, model number etc.), the inventory systems often do not, and may only be concerned with the fact that the item was in fact received, and what the serial number or other distinguishing identifier is. Conventional asset records track accounting information such as depreciation, but not other attribute information.
Once a component or set of components is received it is installed in the data center. Installation and assembly of components that make up a deployable “asset” is not typically performed by those employed in the receiving/warehousing department or by those who track inventory. After the component is physically assembled or installed, it will need to attain a “soft” configuration. The soft configuration includes attributes such as the IP (Internet Protocol) address, operating environment and so on. This soft configuration information frequently depends upon the attributes of the component. For instance, when installing software applications on a computing system asset (“compute node”), the operating system image to be deployed may depend on the size of the disk in the asset. Similarly, the MAC (Media Access Control) address of the network interface card may be needed to give the asset a correct IP address. The current environment relies on highly skilled employees for all aspects of component assembly and configuration. Because such skilled workers are in short supply, the assembly and configuration of new components in a data center can take weeks.
The management system is the vehicle and charge of the administrative or Information Technology (IT) departments within a large entity such as a corporation. The management system must identify, once products are received, what they consist of, and how to configure or install them. This information must be either discovered by the management system or re-entered into the management system by the skilled workers who configure and install the component. As is often the case, the skilled assembler must take the received components and inspect/test them to find out its attributes and configuration because the original order data and the received physical component cannot be easily correlated.
There is thus needed a more efficient configuration process that requires less use of skilled workers and increases the reliability of the configuration job and time-to-deployment of components.
The invention includes a method, system, and article to automatically soft configure a node, such as a compute node, in a data center. The data center may have several racks and a unit may be installed in one of the racks as the node. Each rack may be identified by a unique rack location. The data center may include various servers, devices, and rack locations tied together through a Local Area Network (LAN) mechanism. A new unit deployed within the data center may be discovered. A configuration template for the discovered unit may then be found. Based on the configuration template, software automatically may be installed no the discovered unit.
Referring to the figures, exemplary embodiments of the invention will now be described. The exemplary embodiments are provided to illustrate aspects of the invention and should not be construed as limiting the scope of the invention. The exemplary embodiments are primarily described with reference to block diagrams or flowcharts. As to the flowcharts, each block within the flowcharts represents both a method step and an apparatus element for performing the method step. Depending upon the implementation, the corresponding apparatus element may be configured in hardware, software, firmware or combinations thereof.
The invention primarily consists of utilizing a management system to control the configuration and installation of software on a compute node. The management system maintains a database of asset records, and for each node, when the node is first requested or ordered, it creates an asset record and asset ID unique to that asset. The asset record is associated with the node based upon a certain parameter such the MAC address of the node's NIC. Once a node is deployed it sends out a network request. Based on this request, the management system proceeds with a new unit discovery process. The management system then finds a configuration template suitable for the node. Finally, using the configuration template, software is automatically installed on the node.
At this point, the node has been bolted into a rack, has been plugged to power and networking and has been powered on. By using network messaging (described in detail with respect to FIG. 2), the new unit will undergo a discovery process (block 150). In the new unit discovery, the unit will broadcast a message on the network requesting the management system to provide it with configuration data. The management system uses the information provided by the unit to find a configuration template for the discovered unit (block 160). The configuration templates are a series of configuration parameters and instructions that are stored/created for different classes or types of units. Depending upon the type, model or class of the unit, the management system or other specialized system (e.g., see software configuration system, described below) will find an appropriate configuration template (block 160).
Once a configuration template is found, the management system or other specialized system (e.g., see software configuration system, described below) will install software on the unit based on the parameters given by the template (block 170). Alternatively, the management system may provide the unit with instructions on how to install this software. This automatic installation of software is made possible in a data center environment partially because the management system database contains information about the attributes (such as the MAC address of the network interface card (NIC) in the unit). Once the software is installed, the unit can signal to the management system that it is ready for use (block 180).
The MAC (Media Access Control) address of the NIC is a device signature unique to the NIC. The MAC uniquely identifies the NIC to the management system. MAC addresses are assigned at the time of manufacture and are guaranteed to be globally unique. All network messages sent by the NIC contain its MAC address to allow other nodes to communicate back to it. When a primary NIC sends out a network request message, the management system will compare the MAC sent by the node with all the MACs that are known (block 230). The known MACs will be those of devices that are in inventory or have been received by the company and thus, are present in the management system database. If the MAC is not known, then one possible explanation is that an intruder has penetrated the network. Thus, in this case of an unknown MAC, the management system will begin intruder diagnostics (block 235). Each node with network access in a data center must connect to a known good switch, determining the switch of origin will allow the management infrastructure to determine the location of the intruder. All unknown MACs are assumed to be intruders until verification is complete and the management infrastructure is updated.
If the MAC is known, then using the MAC as a key (or indexing parameter) the asset ID of the node is found (block 240). The next test is to see whether the state information (associated by and stored along with the asset ID) for the node indicates that the node is in the initial state (block 250). The initial state is when the node is first installed in a rack. If it is not in the initial state, then a further check is performed to see whether the node's state information indicates that it is in a reinstall state (block 260). If the node is neither in reinstall nor initial states, then it indicates that the node is undergoing a reboot. In this case, the node is allowed to proceed with its normal boot process (block 270). If the node is either in reinstall state (checked at block 260) or in the initial state (checked at block 250), then software needs to be installed. When in a reinstall state, the node is configured in a like manner to the initial state with the exception that a node needs to be scrubbed (i.e. have its hard drive erased). Hence, to determine which software to install and the parameters thereof, the management system finds an appropriate configuration template for the discovered unit (block 280).
Next, the node is associated with the order's corresponding asset record (block 380). This allows the management system to associate other attributes of the node (e.g., processor type, amount of memory or internal disk) with the MAC address. The management system then waits for the node to be deployed in a rack on the data center floor (block 390). At this point the asset ID for the specific node has been associated with all MACs that will be accessing the network from that node. The asset record contains the configuration information (or a pointer to the configuration template) so that the process of installing and configuring software on the newly deployed node can be automatically carried out by the management system (or other dedicated system such as a software configuration system, detailed below) when it requests configuration information over the network as it is powered up.
LAN mechanism 430 allows other systems such a software configuration system 440 and a management system 450 to be connected to each other and to new compute node 400. The software configuration system 440 serves applications and performs installs of applications to nodes. The management system 450 has database server software, which manages asset records that can be stored in a datastore 460 (e.g., a database). During new unit discovery, the management system 450 responds to a network request from the new compute node 400, once deployed in its rack. The management system 450 then compares the MAC of the primary NIC of compute node 400 with a list of MACs for known devices which may be stored in datastore 460. If known, the management system 450 finds the appropriate asset ID (and, consequently, asset record) associated with the node 400. It then sends a message to compute node 400 with pointers (contained in the asset record) to the correct software in the software configuration system 440. In one embodiment of the invention, the software configuration system may be a tftp (Trivial File Transfer Protocol) server. The compute node then requests the software configuration system for the software and loads it. Depending on the configuration, the node may also request other software from the software configuration system, or alternatively, the software configuration system may install other software on node 400.
The management system 450 is also responsible for tracking and maintaining state information regarding the new compute node 400. This state information can be stored in datastore 460 in an asset record corresponding to the new compute node 400. If the management system 450 determines, for instance, that the new compute node 400 is in an initial state, it will initiate software configuration system 440. The management system 450 will find a configuration template that corresponds to the asset class/type of the new compute node 400 which would be designated in its asset record. The configuration template that is found will then form the basis by which the software configuration system 440 decides how and what software will be installed onto new compute node 400. The software configuration system 440 then installs, automatically, the desired software onto the new compute node 400.
The management system 450 also initially creates the asset record at the time the new compute node 400 is requested or ordered, and maintains in that asset record any post-deployment information that would be desirable for further installation, monitoring or maintenance of the new compute node 400. The software configuration system 440 will contain installable versions of the software that is to be installed on nodes and application software that controls the installation process.
In accordance with the invention, the compute node 500 may be assembled of the components—such as CPU 510, RAM 520, disk 530, primary NIC 540 and secondary NIC 550. Prior to assembly, the bar-code information for these components may be scanned and used to create asset record. When finally deployed, the compute node 500 will send a network request message through either NIC 540 or NIC 550. The management system will located the correct soft configuration information for the node using the MAC address of the NIC that sent the request. Next, the management system and software configuration system will install applications onto disk 530 of node 500 through one or both of the two NICs 540 and/or 550. If the MAC address of the NIC is not known to the management system, the management system may flag the request as a possible intrusion, and start appropriate security measures. Once these applications, such as operating system software, are configured on the node 500, it is then completely deployed as an operational part of its rack and of the data center in which its rack is housed. The CPU 510, RAM 520 and/or disk 530 may be of such a type, speed and capacity that would warrant installing only certain software or only certain optimized or un-optimized versions of the same software. The management system would be able to determine such parameters of the install based upon the asset information about the node 500 that is contained in its asset record.
When the compute node 500 boots, the components attached to the internal bus 580 become active in a specific order. Ordinarily, the primary NIC 540 being in the primary slot becomes active and can communicate with the LAN 590 before the compute node 500 is fully booted. This allows for the primary NIC 540 to act as a gateway for a new soft configuration for the node 500 to be done (soft configuration includes network identity, operating system, applications, etc.).
According to one or more embodiments of the invention, the system 607 or systems similar to it, would be programmed to perform the following functions when implemented as a software configuration system server:
In either role, system 607 has a processor 612 and a memory 611, such as RAM, which is used to store/load instructions, addresses and result data as desired. The implementation of the above functionality in software may derive from an executable or set of executables compiled from source code written in a language such as C++. The instructions of those executable(s), may be stored to a disk 618, such as a hard drive, or memory 611. After accessing them from storage, the software executables may then be loaded into memory 611 and its instructions executed by processor 612. The result of such methods may include calls and directives in the case that the asset records (and related information such as software configuration templates) are stored on disk 618, or a simple transfer of native instructions to the asset records database via network 600 if it is stored remotely. The asset records base may be stored on disk 618, as mentioned, or stored remotely and accessed over network 600 by system 607. Also, installable versions of software applications that are to be installed on deployed nodes may be stored on disk 618, as mentioned, or stored remotely and accessed over network 600 by system 607.
Computer system 607 has a system bus 613 which facilitates information transfer to/from the processor 612 and memory 611 and a bridge 614 which couples to an I/O bus 615. I/O bus 615 connects various I/O devices such as a network interface card (NIC) 616, disk 618 and to the system memory 611 and processor 612. The NIC 616 allows software, such as server software, executing within computer system 607 to transact data, such as requests for network addressing or software installation, to nodes or other servers connected to network 600. Network 600 is also connected to the data center or passes through the data center, so that sections thereof, such as deployed nodes placed in racks and management and software configuration systems, can communicate with system 607.
The exemplary embodiments described herein are provided merely to illustrate the principles of the invention and should not be construed as limiting the scope of the invention. Rather, the principles of the invention may be applied to a wide range of systems to achieve the advantages described herein and to achieve other advantages or to satisfy other objectives as well.
Number | Name | Date | Kind |
---|---|---|---|
5717930 | Imai et al. | Feb 1998 | A |
5978590 | Imai et al. | Nov 1999 | A |
6067582 | Smith et al. | May 2000 | A |
6304892 | Bhoj et al. | Oct 2001 | B1 |
6304906 | Bhatti et al. | Oct 2001 | B1 |
6366876 | Looney | Apr 2002 | B1 |
6499115 | Wiedeman et al. | Dec 2002 | B1 |
6640278 | Nolan et al. | Oct 2003 | B1 |
6651093 | Wiedeman et al. | Nov 2003 | B1 |
6651141 | Adrangi | Nov 2003 | B1 |
6708187 | Shanumgam et al. | Mar 2004 | B1 |
6842749 | Zara et al. | Jan 2005 | B1 |
6857012 | Sim et al. | Feb 2005 | B1 |
6859882 | Fung | Feb 2005 | B1 |
Number | Date | Country | |
---|---|---|---|
20040015957 A1 | Jan 2004 | US |