1. Field of the Invention
The present invention relates to solid-state storage devices. More specifically, the present invention relates to systems and methods for measuring and monitoring the useful life of solid-state storage devices in operating environments.
2. Description of the Related Art
Rotating hard disk drives (HDD) used, for example, in desktop, laptop, notebook, sub-notebook, tablet and embedded computers support an industry-standard advanced technology attachment (ATA) command called Self Monitoring and Reporting Technology (SMART). The SMART function was designed to act as an “early warning system” for pending problems with mechanical media such as HDDs. The integrated controller on the HDD works in conjunction with various sensors to monitor a variety of different parameters within the HDD, such as mechanical wear of the HDD's spindle motor, to determine if any of the parameters are drifting from a norm that would indicate a possible problem with the HDD.
By contrast with HDDs, solid-state storage subsystems generally do not have moving parts. Thus, many of the parameters monitored by the SMART function used in HDDs are not applicable to solid-state storage subsystems. Solid-state storage subsystems generally include non-volatile storage components that can lose the ability to retain data stored thereon after approximately hundreds of thousands to approximately millions of write/erase cycles.
Generally, non-volatile storage components used in solid-state storage subsystems have a finite number of program/erase cycles (usually specified by component vendors as “endurance”) that are recommended or guaranteed for proper data storage and retrieval. The number of such cycles varies by orders of magnitude based on the type of storage component used. Unfortunately, however, there is currently no method that can reliably determine or predict when the recommended or guaranteed endurance in a particular non-volatile storage component will be exceeded. Thus, solid-state storage subsystems are often allowed to operate beyond the specified endurance until a failure occurs, causing unscheduled system down time and potentially significant data loss.
Thus, it would be advantageous to develop a technique and system for reporting information from a solid-state storage subsystem to a host system that uses the information to measure or determine the useful life remaining in the non-volatile storage components of the solid-state storage subsystem.
The present invention comprises a non-volatile solid-state storage subsystem designed to internally maintain usage statistics information reflective of the wear state, and thus the remaining useful life, of the subsystem's memory array. The storage subsystem may, for example, be in the form of a detachable or removable device that plugs into, and receives power and commands via, a standard slot or port of a host computer system. In one embodiment, the storage subsystem supports one or more commands for enabling the host system to read the usage statistics information, or data derived therefrom, to evaluate the subsystem's remaining life expectancy. The host system may use this information for various purposes, such as to display or report information to a user regarding the remaining life of the subsystem, and/or to vary a subsystem usage policy (e.g., to avoid storing mission critical data on a device that is near the end of its useful life).
Neither this summary nor the following detailed description purports to define the invention. The invention is defined by the claims.
A preferred embodiment of the invention will now be described with reference to the drawings summarized below, which are intended to illustrate, and not limit the present invention.
A solid-state storage subsystem, and associated processes that may be implemented by a host computer, will now be described with reference to the drawings. This description is intended to illustrate a preferred embodiment of the invention, and not limit the invention. The invention is defined by the claims.
I. Overview
The solid-state storage subsystem 112 comprises a controller 114 and a non-volatile memory (NVM) array 116. The NVM array may, but need not, be implemented using NAND memory components. As is conventional, the controller 114 is configured (typically via firmware) to write data to, and read data from, the NVM array in response to commands from the host 110. The controller also preferably implements a wear-leveling algorithm, as is known in the art, to distribute write operates across memory blocks of the NVM array. The storage subsystem 112 may be in the form of a detachable device and may communicate with any standard or unique communications interface, including but not limited to parallel, serial ATA, IEEE, RS232/423, PCMCIA, USB, Firewire (IEEE-1394), FibreChannel, or PCI Express bus. The storage subsystem 112 may also receive its power from the host over this bus.
As discussed in detail below, as the controller 114 performs write operations to the memory array 114, it updates a non-user-data area of the array (i.e., an area not exposed to the host's operating system) with usage statistics information reflective of the number of program/erase cycles that have been executed. This information preferably includes a set of counters, with different counters corresponding to different blocks or areas of the memory array; however, the usage statistics may be maintained in any of a variety of formats. These counters are initially set to zero (or some other selected starting value) when the device is manufactured or first initialized, and are incremented over time as program/erase cycles are performed. In some embodiments, the usage statistics data stored in the memory subsystem 112 also includes timestamps, or other temporal data, received from the host; this temporal data may be used to calculate the useful life of the subsystem 112 in terms of time (e.g., days and hours), as may be desirable for some applications.
In addition to industry standard commands, the controller 114 supports, and the driver 113 issues, one or more vendor-specific commands that provide host access to some or all of the usage statistics information. The controller 114 may provide the usage statistics data to the host in its raw format, and/or in a summarized or aggregated form. The host system 114 may use the retrieved usage statistics in a variety of ways so as to reduce the likelihood of data loss. For example, the host system, via the driver 113 or another software component, may display information, such as a gauge (see
Users may use the reported information in a variety of ways. For example, based on historical usage and the predicted amount of useful life remaining, a user may decide to replace the storage subsystem 112 during a regularly scheduled maintenance of the host system. As another example, the resale value of the host system 110 and/or the solid-state storage 112 system may be based at least in part on the useful life remaining in the solid-state storage subsystem.
As another example of how the retrieved usage data may be used, the host system 110 may be programmed to use the usage data to adjust its use of the subsystem 112. For instance, as discussed in further detail below, in a host system that periodically logs data to the solid-state storage subsystem 112, the host 110 may reduce the frequency with which it logs such data as the subsystem approaches the end of its useful life. The host system may also vary its usage policy so that mission critical data is only stored on a subsystem that has not yet reached a particular wear threshold, such as 75%.
Thus, the user and/or the host system 110 can reduce or eliminate the cause of solid-state storage subsystem endurance-related failures. In one embodiment, for example, the host system or user can set a wear threshold that, when met, indicates that the solid-state storage subsystem is in need of preventative maintenance and/or replacement. In addition, or in other embodiments, the host 110 can use data from two time periods and their respective timestamps to calculate the remaining lifespan of the storage subsystem 112. For example, as discussed below, the driver 113 may be configured to periodically write a timestamp to the subsystem 112 (or another storage device) together with information about the subsystem's current wear level, data usage information or endurance data collection, and to retrieve and analyze this information to predict the amount of time before the subsystem fails.
The storage subsystem 112 may, for example, be a solid-state memory card that plugs into a slot of the host system 110 and complies with at least one of the following card specifications: CompactFlash, PCMCIA, SmartMedia, MultiMediaCard, SecureDigital, Memory Stick, ATA/ATAPI. The storage subsystem 112 may, for example, have a housing and signal interface that complies with one of the following specifications: sub 1 inch hard disk drive, 1.8 inch hard disk drive, 2.5 inch hard disk drive and 3.5 inch hard disk drive. A custom form factor and/or signal interface may alternatively be used.
In one embodiment, the controller 114 executes a firmware program to perform processes as described herein and comprises an ATA flash disk controller available from Silicon Storage Technology, Inc. of Sunnyvale Calif. as part number SST55LD019A. The controller 114 may alternatively be implemented using another type of device, such as an application-specific integrated circuit (ASIC), or may comprise multiple distinct devices. Further, although the controller 114 preferably executes firmware, a controller that does not execute a firmware program may be used.
The NVM array 116 comprises a plurality of solid-state storage devices 118 coupled to the controller 114. The solid-state storage devices 118 may comprise, for example, flash integrated circuits, Chalcogenide RAM (C-RAM), Phase Change Memory (PC-RAM or PRAM), Programmable Metallization Cell RAM (PMC-RAM or PMCm), Ovonic Unified Memory (OUM), Resistance RAM (RRAM), NAND memory, NOR memory, EEPROM, Ferroelectric Memory (FeRAM), or other discrete NVM chips. The solid-state storage devices 118 may be physically divided into blocks, pages and sectors, as is known in the art.
The host system 110 exchanges control signals 122 with the controller 114 to coordinate the reading and writing of data to and from the solid-state storage devices 118. The controller 114 handles the read and write operations by sending memory control signals 120 to the NVM array 116. The control signals 122 may include, for example, read commands and write commands. The control signals 122 may be used to send commands selected from, for example, industry standard command sets such as those provided by ATA, CF card or PC card standards to read from or write data to standard storage devices. The host system 110 also exchanges data signals 124 with the controller 114. The data signals may include, for example, data to be written to the NVM array 116, data read from the NVM array, and monitored data, as discussed below.
To retrieve some or all of the stored usage statistics data, the host system 110, via the driver 113, sends a defined sequence of vendor-specific commands to the controller 114. In some cases, the host system 110 may transmit this data, or information derived therefrom, over a computer network to another node.
II. Example User Interface
As shown in
In the example shown in
Other types of displays may also be used, such as the status bar shown in
In some embodiments, the storage subsystem 112 may itself be configured to display information about its current wear state. For example, the storage subsystem may include a small LCD or other display that generates a gauge image similar to that shown in
III. Calculation of Remaining Useful Life
In step 303, the counter values of the valid blocks are summed. The number of valid blocks is also multiplied by 2,000,000 in step 304 to determine the total number of possible writes for the solid-state storage subsystem 112. The 2,000,000 value reflects the number of erase cycles specified as the endurance of most solid-state storage device 118, any may be varied significantly to accommodate different types of memory devices. In other embodiments, the value may be set below this common value so as to reduce the risk of losing data due to a failure. For example, in some embodiments, the threshold is set in a range between approximately 70% and approximately 90% of the specified endurance of the solid-state storage. Finally, in step 305, the sum of the block counters from step 303 is divided by the total number of possible writes from step 304 to estimate the amount of useful life remaining in the storage subsystem 112 as a percentage.
The ATA interface allows vendors to create vendor-specific commands in order to properly engage with the hardware the vendor manufactures for an ATA interface. In certain embodiments discussed herein, in addition to industry standard commands, the controller 114 supports, and the driver 113 issues, one or more vendor-specific commands that provide host access to some or all of the usage statistics information, such as reading counter information for data blocks. Furthermore, one or more vendor-specific commands may provide the host access to information regarding the number of invalid blocks.
IV. Organization of Memory Array
In the example shown in
One data storage method is to map logical addresses to fixed physical locations on the solid-state storage devices 118. However, when a host application updates the same data repeatedly, direct mapping can quickly cause one or more data blocks 402 to wear out due to the large number of program/erase cycles. Repeatedly updating the same group of sectors is common. For example, file systems generally maintain data that describes the allocation of sectors 414 to files. Such data is generally located in a predetermined area on a solid-state storage device 118.
To prevent failures due to repeated program/erase cycles in high-volume locations, the controller 114 remaps logical data to spare data blocks 404 when a particular data block 402 reaches its limit of specified endurance. When using spare data blocks 404 for this purpose, the remaining useful life of the NVM array 116 is directly related to the number of spare data blocks 404 remaining. For example,
However, the common use of wear leveling makes it more difficult to predict the amount of life remaining in a solid-state storage device 118. Wear leveling is generally used to map the same logical data to different physical locations. Thus, program/erase cycles can be evenly spaced across the NVM array 116. Wear leveling may include, for example, monitoring the number of program/erase cycles for the data blocks 402 and changing the logical-to-physical address mapping of one or more data blocks 402 so as to direct future high-volume re-writes to data blocks 402 that have historically been used less often. Thus, the spare data blocks 404 are not generally used until the program/erase cycles have been spread across the data blocks 402 such that a large number of the data blocks 402 have reached their specified endurance limit. When the number of spare data blocks 404 falls below a selected threshold, the controller 114 may, in some embodiments, be configured to interrupt and send a notification message to the host system 110.
Referring again to
In the illustrated embodiment, a predetermined page in a data block 402 is designated as a block counter 408 used by the controller 114 to store monitored data. In certain embodiments discussed herein, the block counter 408 may store the number of times substantially the data block was erase. The block counter 408 may also store the number of times substantially all the data blocks 402 in the NVM array 116 are erased, the number of times substantially all the data blocks 402 in a corresponding solid-state storage device 118 are erased, the number of data blocks 402 that are at or near a threshold value, the number of spare data blocks 404 used, combinations of the foregoing, or the like. In certain embodiments, the block counter 408 may be kept in any storage area generally not accessible by an end user using logical block addressing access.
In order to implement wear-leveling, data is written to pages 406 in a first block 300 until all available pages in the first block 300 store data. As each page 406 is written, its write counter 416 is incremented. When the first block 300 is full, the data is moved to a second block, the first block 300 is erased, and the first block's threshold counter 412 is incremented. After a threshold value is met, the erase counter 410 is incremented and the threshold counter 412 is reset. The combination of the erase counter 410 and the threshold counter 412 make up the total number value for the block counter 408.
Although the usage data is maintained in the NVM array 116 in the preferred embodiment, the controller 114 could alternatively store this data is a separate non-volatile memory. For example, the controller 114 could include its own internal non-volatile memory that is used for this purpose, or could access a separate NVM array that is used to store such status information.
In one embodiment, the controller 114 outputs the raw counter data to the host system 110, and the driver 113 analyzes this data to determine, for example, the amount of life left in the NVM array 116 and/or individual solid-state storage devices 118. The driver 113 may also predict, for example, when the NVM array 116 will likely reach its overall specified endurance. As mentioned above, the controller 114 may alternatively perform some or all of the analysis of the counter data, and output the result to the host 110.
In certain embodiments, usage is also tracked at the page 406 and/or sector 414 levels. Returning to
The one or more counters 416 include, for example, a revision counter representing the number of times the controller 114 has written to the particular sector 414 and an erase counter representing the number of times the controller 114 has erased the corresponding data block 402. In addition, or in other embodiments, the counters 416 include a pointer to the location of the counter 408 in the data block 402. The controller 114 accesses monitored data from one or more of the counters 408, 416 in response to a command from the host system 110. The host system 110 may request the monitored data at any time, as part of a background process, and/or on a polling schedule (e.g., hourly, daily, and/or monthly). In certain embodiments, the controller 114 accesses monitored data directly from the counters 416 in the sectors 414. In other embodiments, the controller 114 accesses monitored data from both the counters 416 in the sectors 414 and the counters 408 in the data blocks 402.
The magnification of the block counter 408 illustrates a data structure configured to track erase cycles for a data block 402 according to an embodiment. This data structure may be generated and maintained by the controller 114 via execution of a firmware program. The block counter 408 includes an erase counter 410 and a threshold counter 412. The erase counter 410 is incremented each time the data block 402 is erased. The threshold counter 412 is incremented when the erase counter 410 reaches a predetermined threshold. When the threshold counter is incremented, the controller 114 performs wear leveling on the corresponding data block 402 by remapping corresponding logical addresses to different physical locations in other data blocks 402.
V. Example Applications Involving Multiple Subsystems
For example, the storage manager 615 may use the usage data to perform wear leveling at the device or subsystem level. For instance, where two solid-state storage subsystems 112 are connected to the host system 110, and the first storage subsystem 112 has more wear than the second, the storage manager 615 may choose to direct data storage to the second subsystem so as to reduce wear on the first storage subsystem 112. The storage manager may also attempt, where possible, to store infrequently-changing data to the first storage subsystem, while directing data that changes relatively frequently to the second storage subsystem.
The storage manager 615 may additionally or alternatively differentiate between critical and non-critical data. For example, the storage manager 615 may choose to store less critical data on a device 112 with more wear and to store more critical data on a device 112 with less wear. Application programs that generate or supply such data may notify the storage manager of the data's type (e.g., critical versus non-critical) at the time a write operation is requested. The storage manager 615 may differentiate between critical and non-critical data during every write to the storage subsystems 112, or at other configurable times.
While certain embodiments of the inventions have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the invention. Indeed, the novel methods and systems described herein may be embodied in a variety of other forms. Furthermore, various omissions, substitutions and changes in the form of the methods and systems described herein may be made without departing from the spirit of the inventions. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the inventions.
This application is a continuation of U.S. application Ser. No. 12/688,815, filed on Jan. 15, 2010, which is a continuation of U.S. application Ser. No. 11/429,936, filed on May 8, 2006, which issued as U.S. Pat. No. 7,653,778. The disclosures of the aforementioned prior applications are hereby incorporated by reference.
Number | Name | Date | Kind |
---|---|---|---|
4296464 | Woods | Oct 1981 | A |
5442768 | Sudoh et al. | Aug 1995 | A |
5737563 | Shigeeda | Apr 1998 | A |
5768612 | Nelson | Jun 1998 | A |
5784698 | Brady et al. | Jul 1998 | A |
5890219 | Scaringella et al. | Mar 1999 | A |
6000006 | Bruce et al. | Dec 1999 | A |
6014727 | Creemer | Jan 2000 | A |
6081878 | Estakhri et al. | Jun 2000 | A |
6088765 | Ohtsuka | Jul 2000 | A |
6092095 | Maytal | Jul 2000 | A |
6249838 | Kon | Jun 2001 | B1 |
6269434 | Tanaka | Jul 2001 | B1 |
6272589 | Aoki | Aug 2001 | B1 |
6401214 | Li | Jun 2002 | B1 |
6434648 | Assour et al. | Aug 2002 | B1 |
6452950 | Ohlsson et al. | Sep 2002 | B1 |
6530034 | Okada et al. | Mar 2003 | B1 |
6546456 | Smith et al. | Apr 2003 | B1 |
6564173 | Arntz et al. | May 2003 | B1 |
6675281 | Oh et al. | Jan 2004 | B1 |
6704012 | Lefave | Mar 2004 | B1 |
6732221 | Ban | May 2004 | B2 |
6754765 | Chang et al. | Jun 2004 | B1 |
6761580 | Chang | Jul 2004 | B2 |
6871272 | Butterworth | Mar 2005 | B2 |
6892248 | Thayer | May 2005 | B2 |
6944063 | Chen et al. | Sep 2005 | B2 |
6944717 | Yoneyama et al. | Sep 2005 | B2 |
6976190 | Goldstone | Dec 2005 | B1 |
6996623 | Kawano et al. | Feb 2006 | B1 |
7054986 | Zhao et al. | May 2006 | B2 |
7079395 | Garnett et al. | Jul 2006 | B2 |
7262961 | Motoe et al. | Aug 2007 | B2 |
7277978 | Khatami et al. | Oct 2007 | B2 |
7447944 | Hu | Nov 2008 | B2 |
7464306 | Furuhjelm et al. | Dec 2008 | B1 |
7596643 | Merry, Jr. et al. | Sep 2009 | B2 |
7653778 | Merry, Jr. et al. | Jan 2010 | B2 |
7694188 | Raghuraman et al. | Apr 2010 | B2 |
7962792 | Diggs et al. | Jun 2011 | B2 |
8078918 | Diggs et al. | Dec 2011 | B2 |
8122185 | Merry, Jr. et al. | Feb 2012 | B2 |
8151020 | Merry, Jr. et al. | Apr 2012 | B2 |
20020138602 | Vinberg | Sep 2002 | A1 |
20030069671 | Yashiki et al. | Apr 2003 | A1 |
20030131093 | Aschen et al. | Jul 2003 | A1 |
20030188007 | Unger | Oct 2003 | A1 |
20030227451 | Chang | Dec 2003 | A1 |
20040128414 | Rudelic | Jul 2004 | A1 |
20040145952 | Chen et al. | Jul 2004 | A1 |
20040190877 | Matsuno et al. | Sep 2004 | A1 |
20040228197 | Mokhlesi | Nov 2004 | A1 |
20040246841 | Miyamoto | Dec 2004 | A1 |
20040260967 | Guha et al. | Dec 2004 | A1 |
20050036387 | Seal et al. | Feb 2005 | A1 |
20050044454 | Moshayedi | Feb 2005 | A1 |
20050047396 | Helm et al. | Mar 2005 | A1 |
20050197017 | Chou et al. | Sep 2005 | A1 |
20050268007 | Nakabayashi | Dec 2005 | A1 |
20050281112 | Ito et al. | Dec 2005 | A1 |
20060056813 | Sutardja | Mar 2006 | A1 |
20060085670 | Carver et al. | Apr 2006 | A1 |
20060085836 | Lyons, Jr. et al. | Apr 2006 | A1 |
20060095647 | Battaglia et al. | May 2006 | A1 |
20060159098 | Munson et al. | Jul 2006 | A1 |
20060248387 | Nicholson et al. | Nov 2006 | A1 |
20060282709 | Shu et al. | Dec 2006 | A1 |
20070008186 | Michaels et al. | Jan 2007 | A1 |
20070053513 | Hoffberg | Mar 2007 | A1 |
20070073944 | Gormley | Mar 2007 | A1 |
20070124130 | Brunet et al. | May 2007 | A1 |
20070124542 | Ven | May 2007 | A1 |
20070159710 | Lucas | Jul 2007 | A1 |
20070180328 | Cornwell et al. | Aug 2007 | A1 |
20070201814 | Yamauchi | Aug 2007 | A1 |
20070260811 | Merry, Jr. et al. | Nov 2007 | A1 |
20070266200 | Gorobets et al. | Nov 2007 | A1 |
20070268791 | Grow et al. | Nov 2007 | A1 |
20080046766 | Chieu et al. | Feb 2008 | A1 |
20080109591 | Kim et al. | May 2008 | A1 |
20090037643 | Ohtsuka et al. | Feb 2009 | A1 |
20090063895 | Smith | Mar 2009 | A1 |
20100011260 | Nagadomi et al. | Jan 2010 | A1 |
Number | Date | Country |
---|---|---|
0589597 | Mar 1994 | EP |
2006118646 | Sep 2006 | WO |
Number | Date | Country | |
---|---|---|---|
20120151130 A1 | Jun 2012 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 12688815 | Jan 2010 | US |
Child | 13399907 | US | |
Parent | 11429936 | May 2006 | US |
Child | 12688815 | US |