Patent Application
20060020618
1. Field of the Invention
This invention is related in general to the field of information handling systems. In particular, the invention consists of a method of preserving operations on persistent data during power interruptions.
2. Description of the Prior Art
In a computing system, digital information is often stored within storage server systems that include hard disk drives, magnetic tapes, optical disks, or other memory storage device. These memory storage devices are usually segregated into ranks or redundant arrays of independent disks (“RAIDS”). This allows for striping information over numerous memory storage devices to provide redundancy in case of a device failure. Each memory storage device may be divided into a plurality of segments. Additionally, each memory storage device may include a reserved area separate from the segments. Both the reserved area and the segments include continuous areas for writing information to and reading information from referred to as tracks.
A memory storage device may include several types of tracks, with each type of track being used for a specific purpose. In a complex storage server system, these types of tracks may include status information tracks such as global Copy Services Meta Data (“CSMD”) tracks residing within the reserved areas and volume-level Meta Data (“VLMD”) tracks residing within the segments.
A storage server system can be logically ordered so that the memory storage devices are used to form a logically continuous memory space. For example, one or more segments may be used to create a volume, e.g., a logical memory space. Volumes are ordered into one or more logical subsystems (“LSS”) and one or more logical subsystems form the logical memory space of the storage server system. This logical memory space is used to hold customer or client information.
Operations on reserved area tracks and segment tracks may take many forms, including writing information to them, reading information from them, and clearing them. In a clearing process, information residing on a track is destroyed or erased. An example of an instruction to erase a track is a Clear Status Tracks (“CST”) command. When executed, a CST command erases information on the CSMD and VLMD tracks. However, a problem occurs is a memory storage device is inaccessible at the time the CST command is issued.
One method of ensuring that a CST command has been performed on CSMD is to utilize redundant copies of CSMD within the storage server system. If a memory storage device holding CSMD is offline, it may simply be updated from other copies of the CSMD. However, the vast majority of information residing within the storage server system is client or customer information. Because it would not be efficient to store multiple copies of this information over numerous memory storage devices, if a memory storage device is offline when a CST command is issued, there may be no mechanism for ensuring that VLMD is cleared when the device becomes accessible again. If the CST is not performed on these VLMD tracks, stale and potentially damaged data would remain on the tracks.
To solve this problem, information pertaining to whether CST must be performed on VLMD tracks must be logged and maintained in a persistent manner. In a storage server system including logical subsystems, this data must track each volume for each LSS.
In U.S. Pat. No. 5,592,675, Hiromichi Itho et al. disclose saving work states for multiple users on a shared system. A work state is saved as persistent data at power off and restored at power on. However, there is no discussion of how to handle operations, such as a deletion, that could be performed on persistent work states. It would be advantageous to have a system to maintain persistence of operations which are performed on persistent data.
In U.S. Pat. No. 6,351,751, Bernard Traversat et al. describe a persistent data manager client/server interface but does not teach how persistence of operations performed on non-volatile data is performed. Rather, the invention teaches who handles the operations to be done on persistent data. A client sends a request to read, modify, or delete non-volatile data and the server performs the action. There is no mention of whether or not the request to modify or delete the non-volatile data is kept persistent in any way. It would be advantageous to have a system that maintains persistent operations to be performed on non-volatile memory.
In U.S. Pat. No. 6,424,641, Michael Haupt et al. teach how to defer read and write accesses for addresses in memory until a time when such data is allowed access. Deferment is implemented by creating a linked list of access requests in a temporary storage structure. Because the nature of this storage structure is temporary, there is no way of remembering these deferred requests across power cycles. It would be advantageous to maintain a deferred operation on inaccessible persistent data even in the event of a power loss.
While some of the above patents and publications may provide a method of managing persistent data, none address the problem of maintaining persistence of operations that must be performed on persistent data. None of these documents indicate how to handle persistent data when the data is inaccessible. Furthermore, these inventions do not offer a method for maintaining operations on persistent data across power cycles. Accordingly, it is desirable to have a method for managing operations on persistent data, even over power cycles and during periods when the persistent data is unavailable.
The invention disclosed herein is a method of maintaining operations on persistent data when volumes of information become unavailable. This persistent information may include volume-level Meta Data residing on tracks within data segments that form the volumes. A command to Clear Status Tracks is executed, even if the memory storage device containing the VLMD is inaccessible, when a memory storage device containing one or more volumes comes back on line.
One aspect of this invention is a bitmap is created for each logical subsystem in the storage server system. Each bit in the bitmap represents a volume that is subject to a persistent operation, such as a CST. These bitmaps are made persistent by storing them on Meta Data tracks that will be read during volume-online notification.
Each LSS has its own corresponding persistent operation bitmap. These persistent operation bitmaps are anchored in a global control structure. When a system-wide command is issued, all bits of a persistent operation bitmap corresponding to an LSS are set.
Each persistent operation bitmap has a corresponding operation-in-progress bitmap to track volumes that are in the process of having their VLMD cleared. Initially, all the bits of the operation-in-progress bitmap are not set. When a persistent operation is dispatched to a volume, the corresponding bit for that volume within the operation-in-progress bitmap is set. When the persistent operation is completed on the volume, the corresponding bits in the operation bitmap and the operation-in-progress bitmap are cleared. However, if the persistent operation failed on the volume (due to communication or power failure), only the corresponding bit in the operation-in-progress bitmap is cleared.
When a system-wide persistent operation has been executed on the storage server system, the operation bitmaps are copied from the global control structure onto one or more global Meta Data (“GMD”) tracks. The GMD resides within the reserved areas of the memory storage devices.
Every time a volume that was inaccessible during a persistent operation becomes available, a volume-online preparation procedure will first check to see if a persistent operation is still pending for that volume. If so, the persistent operation is performed before the volume goes online.
Various other purposes and advantages of the invention will become clear from its description in the specification that follows and from the novel features particularly pointed out in the appended claims. Therefore, to the accomplishment of the objectives described above, this invention comprises the features hereinafter illustrated in the drawings, fully described in the detailed description of the preferred embodiments and particularly pointed out in the claims. However, such drawings and description disclose just a few of the various ways in which the invention may be practiced.
This invention is based on the idea of using a plurality of bitmaps to track the implementation of a system-wide operation on persistent data within a storage server system. The invention disclosed herein may be implemented as a method, apparatus or article of manufacture using standard programming or engineering techniques to produce software, firmware, hardware, or any combination thereof. The term “article of manufacture” as used herein refers to code or logic implemented in hardware or computer readable media such as optical storage devices, and volatile or non-volatile memory devices. Such hardware may include, but is not limited to, field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), complex programmable logic devices (CPLDs), programmable logic arrays (PLAs), microprocessors, or other similar processing devices.
Referring to the figures, wherein like parts are designated with the same reference numerals and symbols,
Each LSS 24 includes one or more volumes 20 (
The global MDCS 30 is illustrated in the block diagram of
Each persistent operation bitmap 32 has a corresponding operation-in-progress bitmap 34 to track volumes 20 that are in the process of having their VLMD 23 modified. Initially, all the bits of the operation-in-progress bitmap 32 are not set. When a persistent operation is dispatched to a volume 20, the corresponding bit 36 within the operation-in-progress bitmap 32 is set. When the persistent operation is completed on the volume, the corresponding bits 36,38 in the persistent operation bitmap 32 and the operation-in-progress bitmap 34 are cleared. However, if the persistent operation failed on the volume 20 (due to communication or power failure), only the corresponding bit 38 in the operation-in-progress bitmap is cleared.
When a system-wide persistent operation has been executed on the storage server system 10, the persistent operation bitmaps 32 are copied from the MDCS 30 onto one or GMD 26 tracks. The GMD 26 resides within the reserved areas 18 of the memory storage devices 14.
Every time a volume 20 that was inaccessible during a persistent operation becomes available, a volume-online preparation procedure will first check to see if a persistent operation is still pending for that volume. If so, the persistent operation is performed before the volume goes online.
A system-wide persistent operation command, such as clear status tracks, is issued in step 44 resulting in the setting all the bits 36 in the persistent operation bit map 32 corresponding to all the volumes 20 (step 46) and clearing all the bits 38 in the operation-in-progress bitmap 34 (step 48).
Instruction are then dispatched to implement the persistent operation command on a volume in step 50 and setting the corresponding bit 38 in the operation-in-progress bitmap 34 in step 52. If the persistent operation command successfully executes (step 52), the corresponding bit 36 in the persistent operation bit map 32 and the corresponding bit 38 in the operation-in-progress bitmap 34 are cleared (step 54). Once the system-wide persistent operation command has been applied to all volumes 20, the persistent operation bitmaps 32 and the operation-in-progress bitmaps 34 are stored (step 56) in the GMD 26. When a volume becomes accessible, the stored persistent operation bitmaps 32 are checked to see if the volume has outstanding persistent operation in step 58 and outstanding persistent operations are implemented in step 60.
Those skilled in the art of maintaining persistent operations on persistent data may develop other embodiments of the present invention. For example, a single bitmap may be implemented for tracking both outstanding persistent operations and operations in progress. Additionally, a bitmap may be created for the entire logical memory space, rather than for each logical subsystem. The terms and expressions which have been employed in the foregoing specification are used herein as terms of description and not of limitation, and there is no intention in the use of such terms and expressions of excluding equivalents of the features shown and described or portions thereof, it being recognized that the scope of the invention is defined and limited only by the claims which follow.
