Data stores, such as, for example, disk drives and flash-based storage, are most efficient when writing data sequentially and are very inefficient when writing randomly scattered data. In addition, disk drives are most efficient when reading sequential data and are very inefficient when reading randomly scattered data. A typical modern disk drive is capable of moving approximately 700 kilobytes (KB) of data in an amount of time taken for the disk drive to seek an arbitrary location on a disk. As technology advances, disk drives may be capable of moving larger amounts of data during a same time period.
Most data transfers are much smaller than 700 KB. As a result, disk drives may spend a significant amount of time seeking locations on disks during non-idle time periods.
This Summary is provided to introduce a selection of concepts in a simplified form that is further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
A processing device and a machine-implemented method may be provided for sequentially aggregating data, originally intended to be written to a volume of a random access medium, to a write log. The processing device may determine whether the data is suitable for writing to the write log. If the data is determined to be suitable for writing to the write log, then the processing device may sequentially aggregate, or write, the data to the write log. Otherwise, the data may be written to an originally intended destination. Entries of the write log may be drained to originally intended destinations upon an occurrence of at least one of a number of conditions. The conditions may include, but not be limited to: the write log being in a state in which new log entries are not permitted; an absence of input or output activity with respect to the random access medium upon which the write log resides; the write log being at the least a first predetermined amount full; the write log being at least a second predetermined amount full regardless of the input or output activity of the random access medium; and a remapping tree being larger than a predetermined size. The remapping tree may map an address range of a random access medium to one or more entries of the write log.
In some embodiments, respective checkpoints may be written to the write log at time intervals. Each of the respective checkpoints may include information describing contents of the write log, such that the remapping tree may be rebuilt after a dirty shutdown. The information included in the respective checkpoints may be obtained from nodes of the remapping tree.
In order to describe the manner in which the above-recited and other advantages and features can be obtained, a more particular description is described below and will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings. Understanding that these drawings depict only typical embodiments and are not therefore to be considered to be limiting of its scope, implementations will be described and explained with additional specificity and detail through the use of the accompanying drawings.
Embodiments are discussed in detail below. While specific implementations are discussed, it is to be understood that this is done for illustration purposes only. A person skilled in the relevant art will recognize that other components and configurations may be used without parting from the spirit and scope of the subject matter of this disclosure.
A method and a system are provided for receiving a request to write to a random access medium, such as, for example, a disk, flash-based storage, or other random access medium. The request to write may then be analyzed to determine whether the request is suitable for sequentially aggregating to a write log. The request may be determined to be suitable for sequentially aggregating to the write log when the request is for writing less than a predetermined amount of data to the random access medium. In some embodiments a request for writing, to the random access medium, an adjustable threshold amount of data, which may be less than 700 KB, may be determined to be suitable for sequentially aggregating to the write log.
The write log may reside within a same volume of the random access medium as an originally intended destination, a different volume of the random access medium as the originally intended destination, or a different volume of a different random access medium as the originally intended destination. The write log may include a number of entries, each having metadata describing a respective entry and corresponding payload data. When a request to write data to the random access medium is determined to be suitable for writing to the write log, the data may be added to an end of the write log. A remapping tree may map locations of the random access medium, outside of the write log, to corresponding entries of the write log and vice versa.
Draining is a process by which entries in the write log are moved to intended destinations of the volume of the random access medium. The write log may be drained to the intended locations of the random access medium under a number of conditions. For example, in some embodiments, the conditions may include: the write log being in a state in which new entries to the write log are not permitted; the write log being more than a first predetermined amount full; no input/output activity with respect to the random access medium and the write log being more than a second predetermined amount full, where the second predetermined amount is less than the first predetermined amount; and the remapping tree being larger than a third predetermined size. During draining, adjacent writes may be merged into a single drain write and drain writes may be ordered to minimize seek time.
In some embodiments, a checkpoint may be written to the write log at adjustable time intervals. A checkpoint may include information summarizing contents of the write log. A dirty shutdown may be an unexpected shutdown or system failure. After a dirty shutdown or a normal shutdown of a system, information included in a last checkpoint written to the write log may be used to recover from the dirty shutdown or the normal shutdown by rebuilding a remapping tree.
Processor 120 may include at least one conventional processor or microprocessor that interprets and executes instructions. Memory 130 may be a random access memory (RAM) or another type of dynamic storage device that stores information and instructions for execution by processor 120. Memory 130 may store temporary variables or other intermediate information used during execution of instructions by processor 120. ROM 140 may include a conventional ROM device or another type of static storage device that stores static information and instructions for processor 120. Storage device 150 may include a hard disk and corresponding drive, a flash-based storage device, or other type of random access data storage device or medium for storing data and/or instructions for processor 120.
Processing device 100 may perform functions in response to processor 120 executing sequences of instructions contained in a tangible machine-readable medium, such as, for example, memory 130, ROM 140, storage device 150 or other medium. Such instructions may be read into memory 130 from another machine-readable medium or from a separate device via a communication interface (not shown).
In embodiments consistent with the subject matter of this disclosure, a write log, for sequentially aggregating writes intended for a random access medium, may be a circular storage area within a random access medium, such as, for example, a disk or other random access medium.
Write log 304 may include a first control area 306 and a second control area 308, which may be located at a beginning and an end, respectively, of write log 304. The first and the second control area may include information about write log 304 including, but not limited to, information about a starting entry of write log 304 and information about a physical end of write log 304. In some embodiments, the information about a starting entry may include a starting log sequence number (LSN) of write log 304, and the information about a physical end of write log 304 may include a wrapping LSN. Some embodiments may include information, such as a pointer or other information regarding a location of a last checkpoint within the first control area 306 and the second control area 308.
Each entry of write log 304 corresponds to a LSN. A LSN may include a wrap count portion and an offset portion. The wrap count portion may be a number of occurrences of wrapping back to a physical beginning of write log 304. The offset portion may be an offset in sectors or some other unit of measurement, from a physical beginning of write log 304. A wrapping LSN may be a LSN at which wrapping to a physical beginning of write log 304 and incrementing of the wrap count may occur. For example, if the wrapping LSN is <wrap count>ยท400, where 400 is an offset from a physical beginning of write log 304, then when a current LSN has reached, for example, 2.400, then wrapping to a physical beginning of write log 304 may occur and the current LSN may become 3.0.
Write log 304 may include a number of entries. As shown in
A remapping tree may be maintained in dynamic storage, such as, for example, memory or RAM. The remapping tree may map intended destinations, such as, for example, LBAs in a volume of a random access medium, to entries in a write log and vice versa. The remapping tree may be structured as an AVL tree. An AVL tree is a self-balancing binary search tree in which heights of child subtrees of any node may differ by no more than one.
If a LBA of a random access medium is written to twice, the second write invalidates the first write. Similarly, if the first write to the LBA is redirected to a write log and the second write to the LBA is also redirected to the write log, an entry in the write log corresponding to the first write may be superseded by an entry in the write log corresponding to the second write. This may be referred to as an internal overwrite.
However, if the first write is redirected to the write log while the second write is written directly to the LBA in the volume of the random access medium, the entry of the write log corresponding to the first write is no longer valid. An invalidation record may be appended to the write log indicating that the entry corresponding to the first write is no longer valid.
If, during act 802, the processing device determines that the received write request is not for a stream, then the processing device may determine whether data, or a data set, to be written, is less than a predetermined size, such as, 700 KB, or another size (act 804). If the data, or the data set, is not less than the predetermined size, then the processing device may indicate that the write request is not suitable for redirecting to the write log (act 808) and the process may be completed.
If, during act 804, the processing device determines that the data, or the data set, to be written, is less than the predetermined size, then the processing device may indicate that the write request is suitable for redirecting to the write log (act 806). The process may then be completed.
Returning to
If, the write to the write log completes successfully the processing device may update a remapping tree for mapping intended destinations in a volume of a random access medium to entries in the write log (act 716).
If, during act 704, the processing device determines that the data, or the data set, to be written, is not suitable for sequentially aggregating to the write log, then the data, or the data set, may be written to an intended destination in the volume of the random access medium (act 710). The processing device may then determine whether any of the LBAs of the intended destination correspond to one or more entries of the write log by referencing the remapping tree (act 712). If any of the LBAs of the intended destination corresponds to one or more entries of the write log, then one or more invalidation records may be appended to the write log to indicate invalidation of the intended destination corresponding to the one or more entries of the write log (act 714).
Writes to the write log may be completed out of order. For example,
A flushed LSN is a LSN corresponding to an entry of the write log, such that all entries of the write log preceding the corresponding entry have no pending writes. Start LSN is a LSN corresponding to a first entry of the write log to be considered for draining, or to be included in a checkpoint. Draining of the write log and creation of checkpoints, both of which are discussed below, may not consider entries beyond the flushed LSN. A flushing LSN is an LSN corresponding to a target entry for flushing. An end LSN is an LSN corresponding to a logical end of the write log.
Next, a write complete may be received with respect to a write to the write log (act 1010). The processing device may determine whether the write complete is for a write log entry before flushing LSN (act 1012). If not, then the processing device may decrement next flushing count, which may indicate a number of pending writes with respect to entries after flushing LSN (act 1014). The processing device may then repeat acts 1010-1012.
If, during act 1012, the processing device determines that the write complete is for a write log entry preceding the entry corresponding to flushing LSN, then the processing device may decrement current flushing count (act 1016).
The processing device may then determine whether current flushing count equals zero, indicating that all write log entries preceding the entry corresponding to flushing LSN are completed (act 1102;
If, during act 1302, the processing device determines that the processing device is not running on battery power, then the processing device may determine whether a status of the write log is invalidation only or disabled (act 1304). If the status of the write log is invalidation only or is disabled, then the processing device may drain the write log to intended destinations of a random access medium (act 1314).
If, during act 1304, the processing device determines that the status of the write log is not invalidation only and is not disabled, then the processing device may determine whether the write log is more than a first predetermined amount full (act 1308). In some embodiments, the first predetermined amount full may be 67% full. In other embodiments, the first predetermined amount full may be another suitable value. If the write log is more than a first predetermined amount full, then the processing device may drain the write log (act 1314).
If the write log is not more than the first predetermined amount full, then the processing device may determine whether there is no input/output with respect to the random access medium and the write log is greater than a second predetermined amount full, such that the second predetermined amount full is a less than the first predetermined amount full (act 1310). If the processing device determines that there is no input/output with respect to the random access medium and the write log is greater than the second predetermined amount full, then the processing device may drain the write log (act 1314).
If the processing device determines that there is input/output activity with respect to the random access medium or the write log is not greater than the second predetermined amount full, then the processing device may determine whether a remapping tree is larger than a third predetermined size (act 1312). If the remapping tree is larger than the third predetermined size, then the processing device may drain the write log (act 1314).
The processing device may then read multiple nodes of the remapping tree (for example, 20 nodes or another suitable number of nodes) and may generate a drain write plan (act 1408). Only entries of the write log corresponding to LSNs preceding the flushed LSN may be drained. The processing device may then determine whether the drain writes overlap with an active volume write (act 1410). The drain writes overlap with the active volume write when the drain writes and the active volume write are to at least one same LBA of the random access medium. When an overlap is detected, the processing device may wait for the volume write to complete (act 1412) and the processing device may begin the process again by re-performing act 1402.
If, during act 1410, the processing device determines that the drain writes do not overlap with the active volume write, then the processing device may merge adjacent drain writes into a single drain write to reduce inputs and outputs with respect to the random access medium (act 1502;
Next, the processing device may determine if the draining occurred because the write log was being disabled (act 1508). If so, then the processing device may determine whether the write log is empty (act 1510). If the write log is empty, then the process may be completed. Otherwise, the processing device may again perform act 1402.
If, during act 1508, the processing device determines that draining did not occur because the write log was being disabled, then the processing device may determine whether draining occurred because of no input/output activity with respect to the random access medium (act 1512). If so, the processing device may determine whether there was any input/output activity with respect to the random access medium other than activity related to draining of the write log (act 1514). If so, then the process may be completed. Otherwise, the processing device may determine whether the write log is relatively empty (act 1516). The write log may be determined to be relatively empty when the write log is less than a predetermined amount full, such as, for example, 10%/o or another suitable value.
If, during act 1516, the processing device determines that the write log is not relatively empty, then the processing device may again perform act 1402 (
If, during act 1512, the processing device determines that draining is not occurring because of no input/output activity with respect to the random access medium, then the processing device may determine whether a remapping tree size is less than a predetermined amount (act 1602;
The process may begin with the processing device waiting for a right moment to write a checkpoint to the write log describing a log range after a previous checkpoint yet before a flushed LSN (act 1702). For example, the right moment may be decided based on write log space not checkpointed since the previous checkpoint, whether a shutdown is initiated, bad sectors exist in non-checkpointed write log space, or other conditions. Next, the processing device may scan a remapping tree, which maps LBAs of a volume of a random access medium to entries in the write log (act 1704). The processing device may then create a checkpoint summarizing the remapping tree, may write the checkpoint to an entry of the write log corresponding to the end LSN, and may update the end LSN (act 1706). The processing device may then repeat acts 1702-1706. Note that only information regarding entries of the write log before the flushed LSN may be recorded in a checkpoint.
A dirty shutdown may be an unexpected shutdown or system crash.
The process may begin with the processing device obtaining a start LSN from either a first control area or a second control area of a write log and determining whether the start LSN is valid (act 1802). The processing device may determine whether the start LSN is valid by reading a write log entry corresponding to the start LSN and determining whether a checksum for the write log entry is valid, by checking another field of the write log entry for an expected value or a predictable value, or by other methods. If the start LSN is determined not to be valid, then the write log may be considered unrecoverable and a notification to that effect may be provided (act 1804).
If the start LSN is determined to be valid, then the processing device may determine whether any checkpoints exist in the write log (act 1806). In some embodiments, a pointer to a last checkpoint may be stored in the first control area and the second control area of the write log. In such embodiments, the pointer may be obtained and the last checkpoint of a chain of checkpoints may be read and validated. In other embodiments, the processing device may scan the write log, beginning at an entry corresponding to the start LSN, to search for checkpoint entries, until no other valid entries of the write log may be found.
If at least one checkpoint exists in the write log, then the processing device may find a last checkpoint of the chain of checkpoints in the write log (act 1808). The processing device may then rebuild a remapping tree based on the chain of checkpoints by traversing all the checkpoints until a write log entry corresponding to Start LSN is hit. (act 1810). Scan LSN may then be set to correspond to a first write log entry not described by any of the checkpoints (act 1811).
If, during act 1806, the processing device determines that no checkpoints exist in the write log, then the processing device may set Scan LSN to Start LSN (act 1820). After performing act 1810 or act 1820, the processing device may then read a next valid entry after corresponding to the scan LSN (act 1812). In some embodiments, one or more invalid entries of the write log may exist between valid entries of the write log. The processing device may then determine whether a valid entry was found (act 1814). If a valid entry was found, then the processing device may update the remapping tree (act 1816) and may again perform acts 1812-1814. If, during act 1814, the processing device determines that a valid entry of the write log was not found, then the processing device may set end LSN to correspond to the last found valid entry of the write log (act 1818). The process may then be completed.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms for implementing the claims.
Although the above descriptions may contain specific details, they are not be construed as limiting the claims in any way. Other configurations of the described embodiments are part of the scope of this disclosure. Further, implementations consistent with the subject matter of this disclosure may have more or fewer acts than as described with respect to
This application is a Continuation of, and claims benefit from, U.S. patent application Ser. No. 13/343,985 that was filed on Jan. 5, 2012, and that is Continuation of U.S. patent application Ser. No. 13/087,927, (U.S. Pat. No. 8,108,450) that was filed on Apr. 15, 2011 (Issued Jan. 31, 2012), and that is a Continuation of U.S. patent application Ser. No. 12/234,411, (Patent No. 7,953,774) that was filed on Sep. 19, 2008 (Issued May 31, 2011), each of which is incorporated herein by reference in its entirety.
Number | Date | Country | |
---|---|---|---|
Parent | 13343985 | Jan 2012 | US |
Child | 15229025 | US | |
Parent | 13087927 | Apr 2011 | US |
Child | 13343985 | US | |
Parent | 12234411 | Sep 2008 | US |
Child | 13087927 | US |