This application relates generally to a method and system for managing the storage of data in a data storage device.
Non-volatile memory systems, such as flash memory, are used in digital computing systems as a means to store data and have been widely adopted for use in consumer products. Flash memory may be found in different forms, for example in the form of a portable memory card that can be carried between host devices or as a solid state disk (SSD) embedded in a host device. These memory systems typically work with data units called “pages” that can be written, and groups of pages called “blocks” that can be read and erased, by a storage manager often residing in the memory system.
A non-volatile memory system may develop problems with efficiency as it becomes more filled with data. Over time, data associated with sequential logical addresses can become scattered over different physical locations in the memory. This fragmentation of the data within a memory system can lead to delays in response time for the memory system as the memory fills up because fewer free blocks may be available for incoming data and the memory system may then need to attend to housekeeping operations to free up more space. The housekeeping operations require more effort by the memory system when the data that is made obsolete by incoming updated data is scattered over multiple different blocks.
In order to address the problems and challenges noted above, a system and method for handling host write commands to reduce fragmentation is disclosed.
According to a first aspect, a method for controlling storage of content on a storage device is disclosed. The method includes, in a storage device having a controller in communication with non-volatile memory, receiving data for storage in the non-volatile memory and determining if a threshold amount of data has been received. If the threshold amount of data has been received, the non-volatile memory is scanned for sequentially numbered logical groups of data previously written in noncontiguous locations in the non-volatile memory. If a threshold amount of sequentially numbered logical groups previously written in noncontiguous locations are present, that threshold amount of sequentially numbered logical groups of data is contiguously re-written into an available new block in the non-volatile memory. In various alternative implementations, the threshold amount of data received may be a fixed, preset amount of data. The threshold amount of sequentially numbered logical groups previously written in noncontiguous locations may also be a fixed, preset amount of logical groups of data. Alternatively, one or both of the threshold amounts may be a variable amount, where the variability may be based on the state of the non-volatile memory, such as the fullness of the non-volatile memory. According to another aspect, a storage device is disclosed having a non-volatile memory and a controller in communication with the non-volatile memory that is configured to carry out the adaptive sorting of logical groups set out above.
In yet another aspect, method of controlling storage of content on a storage device is disclosed. The method includes, in a storage device having a non-volatile memory and a controller in communication with the non-volatile memory, receiving data to be written to the non-volatile memory, incrementing a data write counter an amount corresponding to the received data to be written, and scanning the non-volatile memory for sequentially numbered, but discontiguously located, logical groups when the data write counter reaches or exceeds a scan threshold amount. If a number of sequentially numbered, but discontiguously located, logical groups identified by scanning the non-volatile memory is at least equal to a contiguous write threshold, the identified logical groups are copied sequentially and contiguously into a new block in the non-volatile memory. The sequentially numbered logical groups are invalidated in their respective original blocks after copying the sequentially numbered logical groups contiguously into the new block. In alternative embodiments, one or both of the thresholds of data received or the contiguous write threshold may be increased or decreased based on a parameter relating to the non-volatile memory, such as fullness of the non-volatile memory.
A flash memory system suitable for use in implementing aspects of the invention is shown in
The storage device 102 contains a controller 106 and a memory 108. As shown in
As discussed in more detail below, the storage device 102 may include functions for memory management. In operation, the processor 110 may execute memory management instructions (which may be resident in instructions 116) for operation of the memory management functions. The memory management functions may control the assignment of the one or more portions of the memory within storage device 102, such as controller memory 112. For example, memory management functions may allocate a portion of controller memory 112 for a data cache. One, some, or all of the memory management functions may be performed by one or separate elements within the storage device 102. The controller RAM 114 may include one or more data cache areas for use in optimizing write performance. The controller 106 may also include one of more flash interface modules (FIMs) 122 for communicating between the controller 106 and the flash memory 108.
The flash memory 108 is non-volatile memory and may consist of one or more memory types. These memory types may include, without limitation, memory having a single level cell (SLC) type of flash configuration, also known as binary flash, and multi-level cell (MLC) type flash memory configuration. The flash memory 108 may be divided multiple parts, for example an SLC memory 118 and a main storage area 120, where the main storage area 120 may be further divided into multiple banks 124. Although the banks are preferably the same size, in other embodiments they may have different sizes. The storage device 102 may be arranged to have a different FIM 122 designated for each bank, or more than one bank 124 associated with a FIM 122. Each bank 124 may include one or more physical die, and each die may have more than one plane. The SLC memory 118 may contain a logical group bitmap 130 which may contain a list of valid and invalid logical groups of data stored in the flash memory 108, along with a group address table (GAT) 126 which may contain the physical location information of the logical groups. The GAT 126 and logical group bitmap 130 may be stored in the SLC Memory 118 or in another location in the storage device 102. The SLC memory 118 may also maintain a binary cache 132 and, as described further below, logical group sort data 128. Although the flash memory 108 is shown as including SLC memory 118 outside of the individual banks 124, in other implementations each bank may instead include some SLC memory where each bank would include in its SLC memory a portion of the GAT relevant to that bank.
Each bank 124 of the flash memory 108 may be arranged in blocks of memory cells. A block of memory cells is the unit of erase, i.e., the smallest number of memory cells that are physically erasable together. For increased parallelism, however, the blocks may be operated in larger metablock units. One block from each of at least two planes of memory cells may be logically linked together to form a metablock. Referring to
The individual blocks are in turn divided for operational purposes into pages of memory cells, as illustrated in
As used herein, a metablock is a unit of address space defined to have a range of logical addresses the same size as one or more physical blocks. Each metablock includes multiple logical groups (LGs) and each LG includes a range of logical block address (LBAs) that are associated with data received from a host 100. A logical group may refer to any of a number of logically related groups of data, such as a metapage or a certain number of metapages, a certain number of bytes of data or some other amount of data based on size that a data structure, such as the logical group bitmap 130 and the group address table (GAT) 126 in the SLC memory 118, or other location in the storage device 102, can track within a block.
As noted earlier, over time data in a storage device may become fragmented, where sequentially numbered logical groups become scattered into non-contiguous physical locations in the main storage of the storage device. When this happens, and especially when the main storage comes full, a subsequent write of updated data for a sequential run of logical groups corresponding to those fragmented throughout different locations in the main storage can lead to multiple housekeeping steps to remove the invalid data from each of the different locations to make a new free block for subsequent data writes. Referring to
An example of the type of situation that can result in long response times is shown in
Assuming that the memory is substantially full, in this example five separate compaction operations are needed to free up space for the next write (to free up a block with five logical groups of space). This sequence of compaction steps is illustrated in
In contrast, a logical group sorting technique as disclosed herein may greatly reduce the fragmentation of logical groups in the memory and thus reduce the number of housekeeping steps needed to free up additional blocks. An example of a set of blocks 502 in main storage, for example MLC flash memory, with pre-sorted logical groups is illustrated in
In order to achieve increased sorting of logical groups in memory, a method 600 of managing the memory of a storage device is illustrated in
Upon receipt of data for writing to the main storage, the controller 106 determines if a threshold amount of data directed to the main storage 120 has been received and, if the threshold amount of data has been received, scans the main storage to see how many sequentially numbered logical groups are present in the main storage that are discontiguously located (at 604, 606). In one embodiment, if a block is found having a range of sequentially numbered logical groups, but non-sequentially ordered within the block (for example logical group numbers 1-5 are present, but stored 1, 3, 5, 2, 4 in the block) that range would not be considered to contain discontiguous LGs and would not be counted in the discontiguous LG count. The threshold amount of data received may be a fixed, preset amount in one embodiment, or it may be an amount that varies depending on a state of the storage device in another embodiment. The threshold amount of data received may be tracked as a cumulative amount such that, after the threshold is reached and the memory is scanned, the controller then restarts a tally of received data until the next cumulative amount of received data reaches the threshold triggering the next scanning process. Accordingly, the threshold amount of data received affects the frequency at which the memory is scanned. If the threshold amount of data has not yet been received, the controller writes the received data to the main storage and continues to monitor the cumulative amount of received data (at 604, 602).
When scanning the main storage 120 to determine the number of fragmented logical groups, the controller 106 is looking for logical groups that are sequentially numbered but stored in discontiguous locations, such as in different blocks, in the main storage (at 606). If a threshold amount of sequential but discontiguous logical groups are identified, the controller 106 will then take that threshold number of logical groups and copy them into a new block (at 608, 610). In one embodiment, the sequential logical groups found as discontiguously located throughout the main storage 120 of the memory 108 are copied in sequential order contiguously into the new (free) block. If the scan of the main storage 120 does not turn up the threshold amount of fragmentation, then the process repeats once a next threshold amount of data is received (at 608, 602).
Another embodiment of the method of
After data is received and the counters updated, the received data threshold counter C may be compared to the current threshold X (at 708). The threshold X may be fixed as noted previously or may be dynamic as is described herein. If the received data threshold counter C is less than the threshold X, then the fullness of the main storage is determined and the threshold X may be adjusted based on the current fullness. Additionally, the threshold number of sequential but discontiguous logical groups (Y) may also be adjusted based on the drive fullness (at 710). The process of receiving data to be written to main storage, updating the main storage fullness parameters and incremented the received data threshold counter is then repeated until the counter C is equal to or greater than threshold X. The threshold X may be set at an initial default amount and then adjusted according to changes in fullness of the storage device. The adjustments may be based on an algorithm or simply looked up from a data structure, such as a table as shown in
If the received data threshold counter C is greater than the threshold X, then counter C is reset and the controller scans the main storage to detect the number of sequential, but discontiguously stored logical groups (at 712, 714). For example the controller 106 may scan the main storage by scanning the logical group bitmap 130 maintained in the SLC Memory 118 to determine if logical group data is valid and then scanning the GAT 126 to determine the physical location of each logical group in the main memory 120. A fixed or variable threshold number (Y) of sequentially numbered, but discontiguously stored logical groups may be compared against the number that were determined by scanning the main storage (at 714). The main storage may be formed of blocks of MLC flash arranged in one of more banks as illustrated in
The main storage area fullness may be calculated in a number of ways. In one embodiment, the main storage fullness may be calculated by subtracting the amount of invalidated (obsolete) data from the amount of data written to main storage, for example by subtracting counter B from counter A as described above. This amount may be calculated as a percentage fullness of the main storage and then used to reference a table or calculate an algorithm for adjusting one or more of the threshold amount of received data to trigger a scan of the memory (i.e. threshold X in
An example of a table implementation is illustrated in
In alternative embodiments, only one of the thresholds (X, Y) may be variable based on memory fullness and the other may be fixed. Also, criteria other than fullness, or in addition to fullness, may be used to alter one or both thresholds. In other implementations, the change in the threshold or thresholds may be non-linear, for example weighted to change more rapidly or at different increments as the memory becomes more or less full. The threshold amount of data received (X) may also be varied based on an algorithm or table that is completely different than the threshold number of sequential but discontiguous logical groups. Also, the number of sequential, but discontiguous logical groups re-written to a new block or blocks may be equal to or great than actual threshold number (Y) that triggers a re-write in sequential contiguous order. The controller may also look for all collections of sequential, but discontiguous logical groups that meet the current threshold (Y) during a scan or just the first such collection of sequential, but discontiguous logical groups that meets the threshold.
Depending on the size of the blocks used in the storage device and the current threshold number of logical groups that is being scanned for, a write of sequential logical groups to a new block may not completely fill up a new block or a whole number of new blocks. In one implementation, the controller may then write dummy data to fill up the remainder of the new block and avoid further fragmentation. In another implementation, the controller will not write dummy data and will instead use any remaining space in the new block for use in the next host write. Also, in other embodiments, the scanning of the main storage for fragmentation (sequential but discontiguously stored logical groups) and/or the act of re-writing the fragmented logical groups sequentially and contiguously in one or more new blocks, may be part of a larger memory management scheme that prioritizes garbage collection and other housekeeping operations to maintain a desired level of performance for the storage device.
The method and system described above for adaptively sorting logical groups to prevent or reduce fragmentation may be implemented on a removable or standalone storage device or memory embedded in a host. The techniques for implementing the scanning and sorting process may be implemented in hardware, or as software or firmware executable by a processor of the storage device.
An advantage of the disclosed method and system is that write performance may be improved where write amplification is reduced and a higher percentage of time may be spent on writing data rather than making room for data. For example, as illustrated in
A system and method has been disclosed for improving write performance in a storage device by reducing fragmentation in the memory. According to one embodiment a controller may, after a threshold amount of data has been received, scan the memory for sequentially numbered, but discontiguously stored logical groups. When a threshold number of sequential, but discontiguous logical groups are discovered in a memory scan, they may be re-written in sequence and contiguously into one or more free blocks. The disclosed system and method may adapt the rate of scanning and the trigger amount of fragmented data to account for a status of the storage device, such as the fullness of the main storage.
It is therefore intended that the foregoing detailed description be regarded as illustrative rather than limiting, and that it be understood that it is the following claims, including all equivalents, that are intended to define the spirit and scope of this invention.