1. Technical Field
This disclosure relates to non-volatile storage subsystems. More particularly, the disclosure relates to systems and methods for storing and loading system table data.
2. Description of the Related Art
Non-volatile storage systems can include system tables, such as logical-to-physical mapping table data. Such data may associate logical addresses used by a host system with corresponding physical storage locations in the non-volatile storage system. In certain circumstances, it may be desirable for system table data to be loaded from non-volatile memory, along with log data that indicates changes to a system table. However, loading and updating system table data can incur delays for memory accesses.
Systems and methods that embody the various features of the invention will now be described with reference to the following drawings, in which:
While certain embodiments are described, these embodiments are presented by way of example only, and are not intended to limit the scope of protection. Indeed, the novel methods and systems described herein may be embodied in a variety of other forms. Furthermore, various omissions, substitutions and changes in the form of the methods and systems described herein may be made without departing from the scope of protection.
As used in this application, “non-volatile memory” may refer to solid-state memory such as NAND flash. However, the systems and methods of this disclosure may also be useful in more conventional hard drives and hybrid drives including both solid-state and hard drive components. Solid-state memory may comprise a wide variety of technologies, such as flash integrated circuits, Chalcogenide RAM (C-RAM), Phase Change Memory (PC-RAM or PRAM), Programmable Metallization Cell RAM (PMC-RAM or PMCm), Ovonic Unified Memory (OUM), Resistance RAM (RRAM), NAND memory, NOR memory, EEPROM, Ferroelectric Memory (FeRAM), or other discrete NVM (non-volatile memory) chips. The solid-state storage devices may be physically divided into planes, blocks, pages, and sectors, as is known in the art. Other forms of storage (e.g., battery backed-up volatile DRAM or SRAM devices, magnetic disk drives, etc.) may additionally or alternatively be used.
As used herein, a “segment” refers broadly to any partition, division, or portion of a whole. Segments may or may not be of equal size and the size of each segment may vary over time. The term “page” is used herein according to its broad and ordinary meaning, and may include, among other things, a portion or segment of a storage array or storage device. Although certain embodiments may be described in the context of “pages,” embodiments of systems and methods disclosed herein may be implemented with respect to pages or segments. Therefore, reference to the copying or reading of a page should be understood to also potentially apply to the copying or reading of a segment of pages.
Overview
Data storage systems may include system tables having various sizes and/or characteristics. With respect to system tables of certain sizes, it may be desirable to store table data and related log data in separate structures. Such systems may take advantage of characteristics of certain non-volatile storage systems that allow for parallel storage of multiple table and/or log pages. However, storage of certain data in separate structures may make coherency of data a concern for purposes of reconstruction of table data. Furthermore, loading partial segments of system table data may require to some extent, or be facilitated by, data coherency.
Certain data storage systems implement serial copying, or “flushing” of system table and/or log data to non-volatile storage. Data coherency may be relatively easier to attain in such systems, though other considerations, such as load time, may make serial system table and log storage undesirable. In addition, the ability to load partial segments of table data efficiently may still be a concern.
In some embodiments of the present invention, system data and/or associated logs are stored in an interleaving manner. In certain embodiments, such a system may allow for satisfactory loading of partial segments of table data while maintaining satisfactory levels of data coherency. The various embodiments described in this disclosure increase the efficiency of table data reconstruction (e.g., mapping data, invalid table data, etc.) at power-up, the performance of which is often critical in applications such as solid-state storage devices.
System Overview
In non-volatile storage devices, such as solid-state storage devices, commands can be received from a host designating a logical memory address. The logical memory address can then be converted into a physical address corresponding to a storage location on the non-volatile media. This logical-to-physical address mapping may be maintained in a mapping table in volatile memory of a controller of the storage device. As changes are made to the mapping table, the controller may periodically save a copy of the mapping table, or pages of the mapping table, to the non-volatile storage array. Because the mapping table can comprise a significant amount of data (e.g. 256 Mb for a 256 Gb drive), changes to the mapping table made between saves to the non-volatile storage array can be recorded in a log of mapping table changes and the log can also be saved to non-volatile memory. In this way, the storage device is able to reload the mapping table to volatile memory from the non-volatile memory and reconstruct any changes to the mapping table by using the log data. On system power up, table data may be loaded prior to servicing any media access requests to ensure the logical addresses resolve to the proper locations on the storage array.
The controller 130 can be configured to receive data and/or storage access commands from a storage interface module 112 (e.g., a device driver) in a host system 110. Storage access commands communicated by the storage interface 112 can include write and read commands issued by the host system 110. Read and write commands can specify a logical block address in the storage system 120. The controller 130 can execute the received commands in the non-volatile storage module 150 or in the magnetic storage module 160. In one embodiment, the controller can include memory (e.g., DRAM) for storing data, such as system tables. In certain embodiments, one or more of the depicted devices or modules may be omitted, such as, for example, magnetic storage 160.
The non-volatile storage module 150 may be implemented using NAND flash memory devices. Other types of solid-state memory devices can alternatively be used. In one embodiment, the memory devices are MLC devices, SLC memory devices, or a combination of SLC and MLC devices, may be used in some embodiments.
Storage system 120 can store data communicated by the host system 110. That is, the storage system 120 can act as memory storage for the host system 110. To facilitate this function, the controller 130 can implement a logical interface that can present to the host system 110 storage system's 120 memory as a set of logical addresses (e.g., contiguous address) where data can be stored. Internally, the controller 130 can map logical addresses to various physical memory addresses in the magnetic media 164 and/or the non-volatile storage module 150.
In one embodiment, at least a portion of the non-volatile storage module 150 can be used as cache. The controller 130 can store data communicated by the host system 110 in the non-volatile storage module 150. To improve performance of the storage system 120 and/or host system 110, in some embodiments, various types of data can be stored in non-volatile memory cache, including frequently accessed data, data accessed at start-up (e.g., following a reset or power down), system data, sequentially accessed data, etc.
With continued reference to
The controller depicted in
Table Storage Structure
Certain embodiments disclosed herein provide for chronological and/or synchronized methods of writing system table and log pages to non-volatile memory.
In certain embodiments, log pages are flushed from volatile memory to non-volatile memory as changes to system table data are made or become necessary or desirable. After a predetermined number of log pages have been flushed to NVM (which may be include the non-volatile storage module 150 or the magnetic storage module 160 (e.g., in a shingled disk system where address indirection is used)), a system controller (e.g., system data manager 132) may be configured to flush one or more table pages to NVM prior to flushing further log pages. As will be described below with reference to subsequent figures, such a system data storage method may allow for improved synchronization of table and log pages for purposes of reconstruction of system table data. For example, storing table and log pages according to a fixed ratio may provide information relating to which of a group of previously flushed log pages has become obsolete, and therefore need not be loaded during reconstruction. Moreover, reconstruction time can thus be bounded and the storage system can power-up much more quickly.
As shown in the example embodiment of
When an adequate number of table pages have been stored to conform with the predetermined ratio of table pages to log pages, the method 300 progresses to block 360, where a log page Lm is copied to NVM. At block 370, it is determined whether to store additional log pages to meet the predetermined ratio of table pages to log pages. If more log pages are necessary or desired, m is incremented and the method 300 progresses back to block 360. The variable Lm may represent the next log page chronologically stored in volatile memory that is to be updated. Once enough log pages have been copied, the method 300 progresses back to block 350, from where the method is directed to copy one or more table pages. The method 300 may be performed indefinitely, or for a set amount of time, or set number of operations, such as write commands.
Although in
Full Reconstruction
It may be possible to calculate a location of an oldest valid table page based on the location of the most recently copied table/log page. This is performed at block 504. For example, it may be possible to make such a calculation based on information related to a ratio (R) of table pages to log pages to which the memory array portions 400A-400C conform. As illustrated in the figures, portions 400A-400C conform to a table page to log page ratio of one to four. That is, for every four log pages stored, one table page is stored. However, any suitable ratio may be implemented in accordance with embodiments disclosed herein, such as, for example, one to sixteen, or one to thirty-two. Based on a ratio of one to four, portions 400A-400C can be viewed as five-page segments, each including a table page and four log pages.
Information related to the size of the particular system table may also contribute to calculating the oldest valid table page. For example, the embodiments shown in
P=Pcurrent−((Tcnt−1)×SegSize+(Ptot% SegSize));
Therefore, in the embodiment of
The oldest valid table page is read at block 506. At block 508, it is determined whether the table page read is the most recently copied table page. This is performed at decision block 508. If there are one or more subsequently copied table pages to read, they are read in decision loop 508, 510. Once the most recently copied table page has been read, the oldest unapplied log page is located based at least in part on either the location of the oldest valid table page or the most recent table/log page at block 514. Log pages and table pages may be read from, or stored in, different sections of a memory device, or in different devices. In certain embodiments, log pages are applied to the system table in volatile memory as they are read. The loop represented by steps 516 and 518 involves reading subsequently copied log pages until the most recently copied log page is reached. The method 500A further includes updating the system table by applying the read log pages to the table. As described above, this may be done as the pages are read, or may be done at any other suitable time.
Once the log pages have been applied to the table, the table has been reconstructed in its entirety. As demonstrated by the example embodiments contained in
MaxPagesToLoad=Tcnt×(R+1)
Therefore, in the embodiments of
With reference to
At block 519, it is determined whether Lm is the most recently copied table/log page. If so, the process is terminated. If not, m is incremented and it is determined whether to read more log pages in order to meet the table page to log page ratio R. If so, the method 500B loops between steps 521 and 515 until either Lm is the most recently copied table/log page, or no more log pages are required to meet the ratio R. If it is determined at block 521 that no more log pages are required to meet the ration R, then the process is directed back to block 507 where the next table page is read.
Partial Reconstruction
As an example, with reference to
Full Reconstruction with Partial Reconstruction on Demand
If an update is required, full reconstruction is put on hold while the method carries out partial reconstruction of one or more pages, as described above with reference to
Segmented Mapping Table
Data storage systems may include system tables of various sizes and/or characteristics. With respect to system tables of a certain size, it may be desirable to store table data and related log data in separate structures. Such systems may take advantage of characteristics of certain non-volatile storage systems that allow for parallel storage of multiple table and/or log pages. However, storage of certain data in separate structures may make coherency of data a concern for purposes of reconstruction of table data. Furthermore, loading partial segments of system table data may require to some extent, or be facilitated by, data coherency.
Certain data storage systems implement serial copying, or “flushing” of system table and/or log data to non-volatile storage. Data coherency may be relatively more easy to attain in such systems, though other considerations, such as load time, may make serial system table and log storage undesirable. In addition, the ability to load partial segments of table data efficiently may still be a concern.
In some embodiments of the present invention, system data and/or associated logs are stored serially in an interleaving manner. In certain embodiments, such a system may allow for satisfactory loading of partial segments of table data while maintaining satisfactory levels of data coherency.
While various embodiments described herein are applicable to non-volatile storage module such as, but not limited to, NAND flash, the systems and methods of this disclosure may also be useful in more conventional hard drives (e.g., shingled hard drives) and hybrid hard drives including both solid-state and hard drive components. The solid-state storage devices (e.g., dies) may be physically divided into planes, blocks, pages, and sectors, as is known in the art. Other forms of storage (e.g., battery backed-up volatile DRAM or SRAM devices, magnetic disk drives, etc.) may additionally or alternatively be used.
Those skilled in the art will appreciate that in some embodiments, other types of table data storage and reconstruction can be implemented. In addition, the actual steps taken in the processes shown in certain figures may differ from those shown in the figures. Depending on the embodiment, certain of the steps described above may be removed, others may be added. Accordingly, the scope of the present disclosure is intended to be defined only by reference to the appended claims.
While certain embodiments have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of protection. Indeed, the novel methods and systems described herein may be embodied in a variety of other forms. Furthermore, various omissions, substitutions and changes in the form of the methods and systems described herein may be made. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the protection. Also, the features and attributes of the specific embodiments disclosed above may be combined in different ways to form additional embodiments, all of which fall within the scope of the present disclosure. Although the present disclosure provides certain preferred embodiments and applications, other embodiments that are apparent to those of ordinary skill in the art, including embodiments which do not provide all of the features and advantages set forth herein, are also within the scope of this disclosure. Accordingly, the scope of protection is intended to be defined only by reference to the appended claims.
Number | Name | Date | Kind |
---|---|---|---|
5835955 | Dornier et al. | Nov 1998 | A |
6728826 | Kaki et al. | Apr 2004 | B2 |
6856556 | Hajeck | Feb 2005 | B1 |
7126857 | Hajeck | Oct 2006 | B2 |
7212440 | Gorobets | May 2007 | B2 |
7430136 | Merry, Jr. et al. | Sep 2008 | B2 |
7447807 | Merry et al. | Nov 2008 | B1 |
7502256 | Merry, Jr. et al. | Mar 2009 | B2 |
7502259 | Gorobets | Mar 2009 | B2 |
7509441 | Merry et al. | Mar 2009 | B1 |
7596643 | Merry, Jr. et al. | Sep 2009 | B2 |
7653778 | Merry, Jr. et al. | Jan 2010 | B2 |
7685337 | Merry, Jr. et al. | Mar 2010 | B2 |
7685338 | Merry, Jr. et al. | Mar 2010 | B2 |
7685374 | Diggs et al. | Mar 2010 | B2 |
7733712 | Walston et al. | Jun 2010 | B1 |
7765373 | Merry et al. | Jul 2010 | B1 |
7898855 | Merry, Jr. et al. | Mar 2011 | B2 |
7912991 | Merry et al. | Mar 2011 | B1 |
7913061 | Gorobets et al. | Mar 2011 | B2 |
7936603 | Merry, Jr. et al. | May 2011 | B2 |
7962792 | Diggs et al. | Jun 2011 | B2 |
8078918 | Diggs et al. | Dec 2011 | B2 |
8090899 | Syu | Jan 2012 | B1 |
8095851 | Diggs et al. | Jan 2012 | B2 |
8108692 | Merry et al. | Jan 2012 | B1 |
8122185 | Merry, Jr. et al. | Feb 2012 | B2 |
8127048 | Merry et al. | Feb 2012 | B1 |
8135903 | Kan | Mar 2012 | B1 |
8151020 | Merry, Jr. et al. | Apr 2012 | B2 |
8161227 | Diggs et al. | Apr 2012 | B1 |
8166245 | Diggs et al. | Apr 2012 | B2 |
8194340 | Boyle et al. | Jun 2012 | B1 |
8194341 | Boyle et al. | Jun 2012 | B1 |
8243525 | Kan | Aug 2012 | B1 |
8254172 | Kan | Aug 2012 | B1 |
8261012 | Kan | Sep 2012 | B2 |
8296625 | Diggs et al. | Oct 2012 | B2 |
8312207 | Merry, Jr. et al. | Nov 2012 | B2 |
8316176 | Phan et al. | Nov 2012 | B1 |
8341339 | Boyle et al. | Dec 2012 | B1 |
8375151 | Kan | Feb 2013 | B1 |
8392635 | Booth et al. | Mar 2013 | B2 |
8397107 | Syu et al. | Mar 2013 | B1 |
8407449 | Colon et al. | Mar 2013 | B1 |
8423722 | Deforest et al. | Apr 2013 | B1 |
8433858 | Diggs et al. | Apr 2013 | B1 |
8443167 | Fallone et al. | May 2013 | B1 |
8447920 | Syu | May 2013 | B1 |
8458435 | Rainey, III et al. | Jun 2013 | B1 |
8478930 | Syu | Jul 2013 | B1 |
8489854 | Colon et al. | Jul 2013 | B1 |
8503237 | Horn | Aug 2013 | B1 |
8521972 | Boyle et al. | Aug 2013 | B1 |
8549236 | Diggs et al. | Oct 2013 | B2 |
8583835 | Kan | Nov 2013 | B1 |
8601311 | Horn | Dec 2013 | B2 |
8601313 | Horn | Dec 2013 | B1 |
8612669 | Syu et al. | Dec 2013 | B1 |
8612804 | Kang et al. | Dec 2013 | B1 |
8615681 | Horn | Dec 2013 | B2 |
8638602 | Horn | Jan 2014 | B1 |
8639872 | Boyle et al. | Jan 2014 | B1 |
8683113 | Abasto et al. | Mar 2014 | B2 |
8700834 | Horn et al. | Apr 2014 | B2 |
8700950 | Syu | Apr 2014 | B1 |
8700951 | Call et al. | Apr 2014 | B1 |
8706985 | Boyle et al. | Apr 2014 | B1 |
8707104 | Jean | Apr 2014 | B1 |
8745277 | Kan | Jun 2014 | B2 |
8793429 | Call et al. | Jul 2014 | B1 |
20020184436 | Kim et al. | Dec 2002 | A1 |
20030065899 | Gorobets | Apr 2003 | A1 |
20040083405 | Chang et al. | Apr 2004 | A1 |
20040210706 | In et al. | Oct 2004 | A1 |
20050166028 | Chung et al. | Jul 2005 | A1 |
20080098195 | Cheon et al. | Apr 2008 | A1 |
20090150599 | Bennett | Jun 2009 | A1 |
20100082919 | Chen et al. | Apr 2010 | A1 |
20100106897 | Yoshimura | Apr 2010 | A1 |
20100174849 | Walston et al. | Jul 2010 | A1 |
20100180068 | Matsumoto et al. | Jul 2010 | A1 |
20100250793 | Syu | Sep 2010 | A1 |
20110099323 | Syu | Apr 2011 | A1 |
20110161621 | Sinclair et al. | Jun 2011 | A1 |
20110283049 | Kang et al. | Nov 2011 | A1 |
20110307651 | Wong | Dec 2011 | A1 |
20120110247 | Eleftheriou et al. | May 2012 | A1 |
20120260020 | Suryabudi et al. | Oct 2012 | A1 |
20120278531 | Horn | Nov 2012 | A1 |
20120284460 | Guda | Nov 2012 | A1 |
20120324191 | Strange et al. | Dec 2012 | A1 |
20130132638 | Horn et al. | May 2013 | A1 |
20130145106 | Kan | Jun 2013 | A1 |
20130166819 | Yerushalmi et al. | Jun 2013 | A1 |
20130185508 | Talagala et al. | Jul 2013 | A1 |
20130290793 | Booth et al. | Oct 2013 | A1 |
20140059405 | Syu et al. | Feb 2014 | A1 |
20140115427 | Lu | Apr 2014 | A1 |
20140133220 | Danilak et al. | May 2014 | A1 |
20140136753 | Tomlin et al. | May 2014 | A1 |