The present disclosure relates generally to the field of networks, and, more particularly, to a system and method for recovering from a failure in a network.
As the value and use of information continues to increase, individuals and businesses continually seek additional ways to process and store information. One option available to users of information is an information handling system. An information handling system generally processes, compiles, stores, and/or communicates information or data for business, personal, or other purposes thereby allowing users to take advantage of the value of the information. Because technology and information handling needs and requirements vary between different users or applications, information handling systems may also vary with regard to the kind of information that is handled, bow the information is handled, how much information is processed, stored, or communicated, and how quickly and efficiently the information may be processed, stored, or communicated. The variations in information handling systems allow for information handling systems to be general or configured for a specific user or specific use, including such uses as financial transaction processing, airline reservations, enterprise data storage, or global communications. In addition, information handling systems may include a variety of hardware and software components that may be configured to process, store, and communicate information and may include one or more computer systems, data storage systems, and networking systems.
Computers, including servers and workstations, are often grouped in clusters to perform specific tasks. A server cluster is a group of independent servers that is managed as a single system and is characterized by higher availability, manageability, and scalability, as compared with groupings of unmanaged servers. A server cluster typically involves the configuration of a group of independent servers such that the servers appear in the network as a single machine or unit. Server clusters are managed as a single system, share a common namespace on the network, and are designed specifically to tolerate component failures and to support the addition or subtraction of components in the cluster in a transparent manner. At a minimum, a server cluster includes two or more servers, which are sometimes referred to as nodes, that are connected to one another by a network or other communication links.
A high availability cluster is characterized by a fault tolerant architecture cluster architecture in which a failure of a node is managed such that another node of the cluster replaces the failed node, allowing the cluster to continue to operate. In a high availability cluster, an active node hosts an application, while a passive node waits for the active node to fail so that the passive node can host the application and other operations of the failed active node. To restart the application of the failed node on the passive node, the application must typically reaccess resources and data that was previously held by and accessible to the application on the failed active node. These resources include various data structures that describe the run-state of the application, the address space occupied and accessible by the application, the list of open files, and the priority of the process, among other resources. The process of reaccessing application resources at the passive node produces an undesirable period of downtime during the failover of the affected application from the active node to the passive or backup node. During the period in which the affected application is being established on the passive node, a user cannot access the affected application. In addition, all incomplete transactions being processed by the application at the time of the initiation of the failover process are lost and will have to be resubmitted and reprocessed.
In accordance with the present disclosure, a system and method for recovering from a failure in a cluster node is disclosed. When a node of a cluster fails, a second instance of a software application running on the first node is created on another cluster node. The software application running on the second node is provided with and begins operation on the basis of a data structure that includes data elements representative of the operating state of the software application running on the first node of the cluster. The data structure is a snapshot of the operating state of the first node and is saved to a storage location accessible by all of the nodes of the cluster.
A technical advantage of the disclosed system and method is a failure recovery technique that provides for the rapid initiation and operation in a second node of a software application running on the failed first node. Because the software application of the second node has access to a data structure representative of the operating environment of the software application of the first node, the software application of the second node need not recreate these resources as part of its application initiation sequence. Because of this advantage, the software application of the second node can begin operation with reduce downtime. Because the system and method disclosed herein results in less downtime, fewer transactions are missed during the transition from the software application of the first node to the software application of the second node.
Another technical advantage of the system and method disclosed herein is the disclosed system and method may be implemented such that the saved data structure is stored in multiple locations in the network. In this manner, because the data structure can be stored in multiple locations, the failure of both the first node together with another storage location need not compromise the failure recovery methodology disclosed herein. Another technical advantage is that the system and method disclosed herein may be implemented so that the snapshot of the representative data structure is recorded or captured on a periodic basis or on an event-drive basis in connection with changes to the operating environment of the software application of the first node. Other technical advantages will be apparent to those of ordinary skill in the art in view of the following specification, claims, and drawings.
A more complete understanding of the present embodiments and advantages thereof may be acquired by referring to the following description taken in conjunction with the accompanying drawings, in which like reference numbers indicate like features, and wherein:
For purposes of this disclosure, an information handling system may include any instrumentality or aggregate of instrumentalities operable to compute, classify, process, transmit, receive, retrieve, originate, switch, store, display, manifest, detect, record, reproduce, handle, or utilize any form of information, intelligence, or data for business, scientific, control, or other purposes. For example, an information handling system may be a person computer, a network storage device, or any other suitable device and may vary in size, shape, performance, functionality, and price. The information handling system may include random access memory (RAM), one or more processing resources such as a central processing unit (CPU) or hardware or software control logic, ROM, and/or other types of nonvolatile memory. Additional components of the information handling system may include one or more disk drives, one or more network ports for communication with external devices as well as various input and output (I/O) devices, such as a keyboard, a mouse, and a video display. The information handling system may also include one or more buses operable to transmit communications between the various hardware components. An information handling system may comprise one or more nodes of a cluster network.
Shown in
As indicated in
A flow diagram of the cluster failover method is shown in
With respect to frequency and timing of the capture of the snapshot of the application descriptor. A snapshot of the application descriptor may be taken periodically or according to a predefined schedule. As an example of a period snapshot capture, a snapshot may be taken every thirty seconds during any period in which the associated application is active. In addition to or as an alternative to a periodic capture of the application descriptor, the capture of a snapshot of the application descriptor may be event driven. A snapshot of the application descriptor may be taken when any or certain predefined elements of the application descriptor are modified. In this event-driven mode, a change to the application description would result in an updated snapshot of the application descriptor being saved to the memory location.
At step 32 of
Following the completion of the steps of
The failure recovery technique disclosed herein has been described with respect to a single instance of application software that is being replicated upon the failure of an active node to a passive node. The technique described herein may be employed with any number of instances of application software present in the active node. In the case of multiple instances of application software present on the active node, an application descriptor is created for each instance of application software and, as described with respect to
The recovery failure techniques disclosed herein is not limited in its use to clusters having only two nodes. Rather, the technique described herein may be used with clusters having multiple nodes, regardless of their number. Although a dual node example of the technique is described herein, the failure recovery system and method of the present disclosure may be used in cluster networks having any combination of single active nodes, single passive nodes, multiple active nodes, and multiple passive nodes. Although the present disclosure has been described in detail, it should be understood that various changes, substitutions, and alterations can be made hereto without departing from the spirit and the scope of the invention as defined by the appended claims.