Patent Grant
9239781
The present invention relates generally to a storage control system and more particularly to a system for data management and control.
Data storage, often called storage or memory, refers to computer components and recording media that retain digital data. Data storage is a core function and fundamental component of consumer and industrial electronics, especially devices such as computers, televisions, cellular phones, mobile devices, and digital video cameras.
Recently, forms of long-term storage other than electromechanical hard disks have become feasible for use in computers. NOT-AND (NAND) flash is one form of non-volatile memory used in solid-state storage devices. The memory cells are arranged in typical row and column fashion with circuitry for accessing individual cells and placing the memory transistors of those cells store an analog value that can be interpreted to hold two logical states in the case of Single Level Cell (SLC) or more than two logical states in the case of Multi Level Cell (MLC).
A flash memory cell is light in weight, occupies very little space, and consumes less power than electromechanical disk drives. Construction of a storage system with this type of memory allows for much higher bandwidths and input/output operations per second (IOPS) than typical electromechanical disk drives. More importantly, it is especially rugged and can operate at a much high temperature range. It will withstand without adverse effects repeated drops, each of which would destroy a typical electromechanical hard disk drive. A problem exhibited by flash memory is that it tends to have a limited life in use.
Thus, a need still remains for better data management devices. In view of the increasing demand for data management devices, it is increasingly critical that answers be found to these problems. In view of the ever-increasing commercial competitive pressures, along with growing consumer expectations and the diminishing opportunities for meaningful product differentiation in the marketplace, it is critical that answers be found for these problems. Additionally, the need to reduce costs, improve efficiencies and performance, and meet competitive pressures adds an even greater urgency to the critical necessity for finding answers to these problems.
Solutions to these problems have been long sought but prior developments have not taught or suggested any solutions and, thus, solutions to these problems have long eluded those skilled in the art.
The present invention provides a method of operation of a storage control system, including: partitioning memory channels with memory devices; selecting a super device with one of the memory devices from one of the memory channels, the super device having a super chip select connected to chip selects of the memory devices; and selecting a super block associated with the super device.
The present invention provides a storage control system, including: a channel control module for partitioning memory channels with memory devices; a device control module for selecting a super device with one of the memory devices from one of the memory channels, the super device having a super chip select connected to chip selects of the memory devices; and a block control module for selecting a super block associated with the super device.
Certain embodiments of the invention have other steps or elements in addition to or in place of those mentioned above. The steps or elements will become apparent to those skilled in the art from a reading of the following detailed description when taken with reference to the accompanying drawings.
The following embodiments are described in sufficient detail to enable those skilled in the art to make and use the invention. It is to be understood that other embodiments would be evident based on the present disclosure, and that system, process, or mechanical changes may be made without departing from the scope of the present invention.
In the following description, numerous specific details are given to provide a thorough understanding of the invention. However, it will be apparent that the invention may be practiced without these specific details. In order to avoid obscuring the present invention, some well-known circuits, system configurations, and process steps are not disclosed in detail.
The drawings showing embodiments of the system are semi-diagrammatic and not to scale and, particularly, some of the dimensions are for the clarity of presentation and are shown exaggerated in the drawing FIGS.
Where multiple embodiments are disclosed and described having some features in common, for clarity and ease of illustration, description, and comprehension thereof, similar and like features one to another will ordinarily be described with similar reference numerals. The embodiments have been numbered first embodiment, second embodiment, etc. as a matter of descriptive convenience and are not intended to have any other significance or provide limitations for the present invention.
The term “module” referred to herein can include software, hardware, or a combination thereof in the present invention in accordance with the context in which the term is used. For example, the software can be machine code, firmware, embedded code, and application software. Also for example, the hardware can be circuitry, processor, computer, integrated circuit, integrated circuit cores, a microelectromechanical system (MEMS), passive devices, environmental sensors including temperature sensors, or a combination thereof.
Referring now to
The memory controller 104 provides data control and management of the memory array 106. The memory controller 104 interfaces with the host system 108 and controls the memory array 106 to transfer data between the host system 108 and the memory array 106.
The memory array 106 includes an array of memory devices 110 including flash memory devices or non-volatile memory devices. The memory array 106 can include pages of data or information. The host system 108 can request the memory controller 104 for reading, writing, and deleting data from or to a logical address space of a storage device or the memory sub-system 102 that includes the memory array 106.
The memory devices 110 can include chip selects 112, which are defined as control inputs, for enabling the memory devices 110. Each of the chip selects 112 can be used to control the operation of one of the memory devices 110. Each of the chip selects 112 can be used to control sub addresses or logical units (LUNs) within each of the memory devices 110. When the chip selects 112 are enabled, the memory devices 110 are in active state for operation including reading, writing, or erasing.
Referring now to
The control unit 202 can be implemented in a number of different manners. For example, the control unit 202 can be a processor, an embedded processor, a microprocessor, a hardware control logic, a hardware finite state machine (FSM), a digital signal processor (DSP), or a combination thereof.
The control interface 210 can be used for communication between the control unit 202 and other functional units in the memory controller 104. The control interface 210 can also be used for communication that is external to the memory controller 104.
The control interface 210 can receive information from the other functional units or from external sources, or can transmit information to the other functional units or to external destinations. The external sources and the external destinations refer to sources and destinations external to the memory controller 104.
The control interface 210 can be implemented in different ways and can include different implementations depending on which functional units or external units are being interfaced with the control interface 210. For example, the control interface 210 can be implemented with a dedicated hardware including an application-specific integrated circuit (ASIC), a configurable hardware including a field-programmable gate array (FPGA), a discrete electronic hardware, or a combination thereof.
The storage unit 204 can include both hardware and the software 212. For example, the software 212 can include control firmware. The storage unit 204 can include a volatile memory, a nonvolatile memory, an internal memory, an external memory, or a combination thereof. For example, the storage unit 204 can be a nonvolatile storage such as non-volatile random access memory (NVRAM), Flash memory, disk storage, or a volatile storage such as static random access memory (SRAM).
The storage unit 204 can include a storage interface 214. The storage interface 214 can also be used for communication that is external to the memory controller 104. The storage interface 214 can receive information from the other functional units or from external sources, or can transmit information to the other functional units or to external destinations. The external sources and the external destinations refer to sources and destinations external to the memory controller 104.
The storage interface 214 can include different implementations depending on which functional units or external units are being interfaced with the storage unit 204. The storage interface 214 can be implemented with technologies and techniques similar to the implementation of the control interface 210.
The memory interface unit 206 can enable external communication to and from the memory controller 104. For example, the memory interface unit 206 can permit the memory controller 104 to communicate with the memory array 106 of
The memory interface unit 206 can include a memory interface 216. The memory interface 216 can be used for communication between the memory interface unit 206 and other functional units in the memory controller 104. The memory interface 216 can receive information from the other functional units or can transmit information to the other functional units.
The memory interface 216 can include different implementations depending on which functional units are being interfaced with the memory interface unit 206. The memory interface 216 can be implemented with technologies and techniques similar to the implementation of the control interface 210.
The host interface unit 208 allows the host system 108 of
The control unit 202 can operate the host interface unit 208 to send control or status information generated by the memory controller 104 to the host system 108. The control unit 202 can also execute the software 212 for the other functions of the memory controller 104. The control unit 202 can further execute the software 212 for interaction with the memory array 106 via the memory interface unit 206.
The functional units in the memory controller 104 can work individually and independently of the other functional units. For illustrative purposes, the memory controller 104 is described by operation of the memory controller 104 with the host system 108 and the memory array 106. It is understood that the memory controller 104, the host system 108, and the memory array 106 can operate any of the modules and functions of the memory controller 104.
Referring now to
The channel control module 302 is for performing operations associated with memory channels 404 of
Referring now to
The memory groups 402 can be identified by the channel control module 302 of
Each of the squares in
The channel control module 302 can partition each of the memory channels 404 that are associated with a number of the memory devices 110. Each of the memory channels 404 is defined as a physical bank or a physical group of the memory devices 110 that share a common data bus and a common set of associated control signals.
The memory channels 404 can represent an independent bus or communication path. Each of the memory channels 404 can represent an independent execution unit. Each of the memory channels 404 can include an ability to perform independent and parallel operations with respect to other instances of the memory channels 404.
Actions, commands, and status information transmitted within one of the memory channels 404 can be limited to the memory devices 110 that are connected to the one of the memory channels 404 and can be independent from other instances of the memory channels 404 within a storage device. Data bus and control signals of one of the memory channels 404 cannot be shared with the other instances of the memory channels 404.
Each of the memory channels 404 can include a number of bits of data based on a type of the memory devices 110. For example, each of the memory channels 404 can include 8 bits, 16 bits, or any number of bits of data bus depending on the type of the memory devices 110 used and the construction of the memory controller 104. Also for example, the memory channels 404 can represent physical channels.
For example, one of the memory channels 404 can control or have ownership of a set of the memory devices 110 that share a common data bus and control signals. Also for example, the memory devices 110 can represent flash devices or flash components that independently operate from each other.
The device control module 304 of
The device control module 304 can group the memory devices 110 from the memory channels 404 to form the super devices 406. For example, if there are 16 of the memory channels 404 and 8 of the memory devices 110 per each of the memory channels 404, there can be 8 of the memory devices 110, one from each of the memory channels 404, grouped together for each of the super devices 406 and 16 of the super devices 406 as shown in
Each of the super devices 406 can be associated with one of the memory devices 110 from each of the memory channels 404. Although each of the super devices 406 is shown having 8 of the memory devices 110 for illustration purposes, it is understood that each of the super devices 406 can include any number of the memory devices 110. For example, each of the super devices 406 can include up to a number of the memory channels 404. Also for example, all of the super devices 406 can have the same structure and size. Each of the super devices 406 can include any number of the memory devices 110 per each of the memory channels 404. In this case, there can be multiple passes made or accessed through the memory channels 404. For example, each of the super devices 406 can include a number of the memory channels 404 with 2 of the memory devices 110 per each of the memory channels 404.
Each of the super devices 406 can include one of the memory devices 110 from each of the memory channels 404. Each of the super devices 406 can include a number of the super blocks 502 of
The erase blocks 408 are units of a non-volatile memory device or other types of block oriented non-volatile memory devices. For example, the erase blocks 408 can be one of atomic units of a NAND flash memory device or other types of block oriented non-volatile memory devices, which no finer control over erasing memory elements is capable. Also for example, the erase blocks 408 can form a part of a logically addressable memory space within a NAND flash or other type of block oriented non-volatile memory devices.
The erase blocks 408 can represent the smallest unit of memory that can be erased. The memory devices 110 can represent NAND packages that can include multiple dies or logically addressable units (LUNs). Each of the dies or LUNs can include multiple planes, each of which can include a number of the erase blocks 408. Each of the erase blocks 408 can include a number of pages, which can include a number of bytes.
For example, each of the planes can include approximately 2048 of the erase blocks 408. Also for example, each of the erase blocks 408 can include 256 pages, each of which can include 8K bytes, where each 1K bytes includes 1024 bytes. A die can include approximately 4096 of the erase blocks 408 divided into two of the planes with 2048 of the erase blocks 408 per plane.
As an example, each of the super devices 406 can access one of the super blocks 502 at any given instance. Each of the super blocks 502 can include a number of the erase blocks 408 that are accessible by one of the super devices 406 at any given time. There can be one of the erase blocks 408 active for each of the memory channels 404 within one of the super devices 406. The erase blocks 408 can be assigned across the memory channels 404 so that they can be accessed in a parallel approach. One of the erase blocks 408 can be assigned per each of the memory channels 404 to form one of the super blocks 502 in the super devices 406.
In an example of one of the super devices 406 having 8 of the memory devices 110, there can be 8 active instances of the erase blocks 408 associated with the one of the super devices 406. The 8 active instances of the erase blocks 408 can belong to a specific instance of the super blocks 502. Also as an example, there can be multiple active instances of the erase blocks 408 per each of the memory devices 110.
The present invention can include architecture and use of the memory devices 110 including NAND flash or other block oriented non-volatile memory organized in efficient groups to reduce the overhead of and runtime management in a solid state storage device. The present invention can include the super blocks 502 as an organization of memory blocks for parallel use by multiple memory controllers and the logical and physical linkage between the memory blocks. The present invention can provide additional organization for data protection via RAID operations with reduced overhead for both runtime recycling and drive initialization speed. The present invention can provide an additional structure for managing bad block memory elements without detriment or loss of system performance or efficiency.
The super blocks 502 provide the following effects. One of the effects is amount of metadata that is stored in each of the erase blocks 408 is minimized or reduced. Such metadata can now be stored at a super-block level or in one of the super blocks 502. Another of the effects is fewer blocks are managed as well as reduction in time to initialize these data structures.
Another of the effects is host data is written to the super blocks 502 in a deterministic order. Another of the effects is recycling can traverse or access the super blocks 502 in the same order by which the host system 108 writes data. As a result, recycling operations can preserve the same order used in write operations providing data striping for optimal read-back performance.
It has been discovered that the super devices 406 provide improved performance by grouping of the memory devices 110 that are controlled in parallel across a number of the memory channels 404.
Referring now to
Each of the super blocks 502 can be identified and selected by the block control module 306 of
There can be a number of the super devices 406 included in a solid-state drive or a storage system dictated by a number of addressable elements within the memory channels 404 that can be controlled in a parallel method of operation. There can be a number of the super blocks 502 per each of the super devices 406. Any of the super blocks 502 can be associated with one and only one of the super devices 406.
All the chip selects 112 of
One of the super chip selects 504 is selected to be associated with one of the super blocks 502 and a group of the chip selects 112 of the memory devices 110 that are associated with the one of the super blocks 502. The one of the super chip selects 504 can be used to control the group of the chip selects 112. The super chip selects 504 are associated with the super blocks 502. The super chip selects 504 are provided by the block control module 306 to control operation of the memory devices 110.
The super chip selects 504 can be used to assert or de-assert the chip selects 112. The chip selects 112 can be asserted or de-asserted to enable or disable, respectively, the memory devices 110. For example, the chip selects 112 can be asserted or de-asserted by the block control module 306 assigning the super chip selects 504 to an active or inactive state such that the chip selects 112 are driven to logic ‘1’ or ‘0’, respectively, to enable or disable the memory devices 110.
Each of the super blocks 502 can span or include only one of the super chip selects 504. By having multiple of the super chip selects 504 for multiple of the super blocks 502, one of the super chip selects 504 can be available for writing host data from the host system 108 of
If there are several pending host reads, there is a good chance that one of them can be located on one of the super chip selects 504 that are idle. If data requested for a host read operation was written by a large-block host write operation, there is a good chance that all the data can be read with one of the super chip selects 504 that was used for the large-block host write operation.
The last instance of the erase blocks 408 at the end of one of the super blocks 502 can be written by the block control module 306. The last instance of the erase blocks 408 can be written with parity information 506 for RAID-5 architectures. As such, the size of each of the super blocks 502 can determine the overhead in the RAID-5 architectures. The last instance of the erase blocks 408 is the last block to be accessed in the order by which the erase blocks 408 in the super blocks 502 are written by the host system 108. RAID-5 refers to an implementation for redundant array of independent disks (RAID) systems that uses block-level striping with parity data distributed across all member disks.
An allocation of the erase blocks 408 within one of the super blocks 502 can provide additional area for storage of the parity information 506 including RAID parity data. The parity information 506 can only be complete after a full stripe across all the erase blocks 408 within the super blocks 502 are written. The parity information 506 can be stored within one of the super blocks 502.
The parity information 506 is defined as redundancy data that is used to rebuild or reconstruct a memory array to recover lost or corrupt data. The parity information 506 can be calculated by the block control module 306 using exclusive-OR (XOR) binary operation performed on the data to be stored.
Other effects include, for RAID-5 operations, the last of the erase blocks 408 in the super blocks 502 can be reserved for parity bits by the block control module 306. As a result, parity pages are recycled along with data pages. Having the parity pages in a dedicated instance of the erase blocks 408, possibility of read disturb on these pages is significantly reduced.
The parity pages can be read only if there is an error correction code (ECC) error in one of the data pages in the super blocks 502. The parity pages can only be used in a case of an uncorrectable error and thus can have an extremely low read count, making them less susceptible to read disturb issues. Data retention in parity blocks can be better than that of data blocks in the super blocks 502 that are read a large number of times.
A read disturb in the memory devices 110 is a change of values of memory cells by an act of reading a memory cell or other memory cells within the memory devices 110. For example, in a NAND flash, read disturbs can be limited to cells within one of the erase blocks 408. An example of this is reading from one page of memory within one of the erase blocks 408 in the memory devices 110 can corrupt and change memory cells in other pages within the one of the erase blocks 408. The read disturb affect refers to the process of reading memory will create errors in the memory. Read disturbs are not always permanent and can be corrected when the memory is rewritten.
Each time a page is read in one of the erase blocks 408, a small amount of charge change is experienced by other pages within the one of the erase blocks 408. Therefore, by reading pages in one of the erase blocks 408, the bit error rate (BER) of the other pages can increase. By locating the RAID-5 parity data in the erase blocks 408 other than the erase blocks 408 being used for user data, the parity data does not suffer the same read disturbs as the normal data.
Read-write and read-erase conflicts can be minimized by the block control module 306. The conflicts can be minimized by partitioning the super blocks 502 using the super chip selects 504. By rebalancing the super blocks 502 with the memory access module 308 of
There are positive factors of having the super blocks 502. The positive factors include fewer blocks to manage, fewer metadata pages 508, infrastructure support for a RAID-5 implementation, and minimization of writes to a logical-to-physical (L2P) log during recycling. The metadata pages 508 can be written by the block control module 306 to include logical page indexes. The metadata pages 508 can be grouped and written once per each of the super blocks 502 as opposed to keeping metadata in each of the erase blocks 408. This grouping of the metadata can also make traversing it during drive initialization more efficient.
The term “metadata” refers to non-host related information that is in addition to and stored along with host data and that aids or supports retrieval, organization, health of, and current condition or state of the memory devices 110. For example, metadata can include non-host related information including logical-to-physical mapping information of the host data within the memory devices 110. Also for example, metadata can include non-host related information including logical block address (LBA) information, super block identification (ID) values, program/erase (PE) cycles, aging data, and other NAND statistics that is collected as a consequence of normal operations.
As an example, there can be transient metadata held or stored in the storage unit 204 with a Random access memory (RAM), a Static random-access memory (SRAM), and a double data rate-synchronous dynamic random access memory (DDRAM). As another example, there can be persistent metadata held or stored in the memory devices 110.
The memory devices 110 that form the super devices 406 can be fixed through the life of a storage drive. It is understood that this assignment of the memory devices 110 to the super devices 406 can be determined in a number of ways because there can be unpopulated instances of the memory devices 110 in the storage drive.
Although there are three of the super blocks 502 shown in
A number of concurrent writes can be per logical unit (LUN) or dies. Each of the pages can be specifically on separate instances of the memory channels 404, or LUNs, and not just within one of the erase blocks 408.
Multiple of the erase blocks 408 grouped into each of the super blocks 502 by the block control module 306 provide several advantages. One of the advantages is that there fewer of the super blocks 502 than a number of the erase blocks 408 to manage and initialize by the block control module 306. Instead of managing lists of the erase blocks 408, lists of the super blocks 502 can be managed by the block control module 306. As a result, data management resources including management data structures for managing the super blocks 502 are smaller or less.
By creating the super blocks 502 that span multiple of the chip selects 112 of the memory devices 110, additional advantages are achieved. One advantage is that the super blocks 502 provide an infrastructure that supports an internal redundant array of independent disks (RAID) architecture including RAID-5. The super blocks 502 can be created in a way that facilitates efficient host performance, recycling, RAID, and load balancing performed by the memory access module 308.
The term “internal” used above for RAID refers to RAID within and specific to one of the super blocks 502. RAID parity information can be part of and limited to contents of the one of the super blocks 502. As such, operations to the one of the super blocks 502 self-contained with respect to providing RAID error protection to host data and any stored metadata information.
A number of the erase blocks 408 in each of the super blocks 502 can be determined by the super devices 406 of
Within one of the super blocks 502, each of the erase blocks 408 does not have to be associated with the same erase block number (or address) in each of the elements. As an example, a portion of the 23rd instance of the super blocks 502 can include a number of the erase blocks 408 as follows.
Channel 0, CEO, LUN0, erase block 23
Channel 1, CEO, LUN0, erase block 23
Channel 2, CEO, LUN0, erase block 25
Channel 3, CEO, LUN0, erase block 23
Channel 4, CEO, LUN0, erase block 27
In this example, “Channel” is a corresponding channel number of the memory channels 404, “CE0” is chip enable 0, “LUN0” is logical unit 0, and “erase block” is an nth instance of the erase blocks 408.
Holes in contiguous addressing of erase blocks will exist from both factory marked bad erase blocks and erase blocks that have been determined to be bad during normal operation of the storage device. Remapped specifically refers to the substitution of one physical erase block for another.
Implementation of RAID-5 presents significant problems for recycling because parity pages need to be recycled along with data pages. By implementing RAID within the super blocks 502, data and parity pages are recycling together by the memory access module 308.
The super blocks 502 also provide a way for a recycling operation performed by the memory access module 308 to traverse data in the order by which that data is written. This operation minimizes the number of writes to the logical-to-physical log. This operation minimizes the number of writes to the logical-to-physical log, especially when the host system 108 writes larger data blocks.
Other advantages include fewer statistics, which can be included in the metadata pages 508, need to be managed by the block control module 306. These statistics can include a program-erase count 510 and a valid-page count 512. There can be fewer of the metadata pages 508.
The program-erase count 510 is defined as a numerical value indicating a number of times or cycles the memory devices 110 associated with the corresponding instances of the super blocks 502 are programmed or erased by the memory access module 308. The valid-page count 512 is defined as a numerical value indicating a number of pages that are valid in each of the super blocks 502.
Since host data can be written in the order it arrives from the host and because all data pages within the memory devices 110 can be written sequentially and contiguously, one or more new version of the same host logical sector can exist within the super blocks 502. In this case, only one of the host sectors can be valid, making other copies in the super blocks 502 invalid. The valid state of the pages within one of the super blocks 502 can be held in both the metadata pages 508 stored within the super blocks 502 and in the logical-to-physical (L2P) tabling system within the runtime volatile memory in the storage device.
Each of the super blocks 502 can include one of the metadata pages 508, denoted as Page-63. Each of the metadata pages 508 can span multiple physical pages and is not limited to a single physical page depending on a number of elements in each of the super blocks 502 and the amount of metadata required for each element. Each of the super blocks 502 can include only one of the metadata pages 508. The metadata pages 508 can include logical page indexes of the super blocks 502. If the size of the super blocks 502 is properly determined, each of the metadata pages 508 can have a size that matches a size of a physical page in the memory devices 110.
The metadata pages 508 can include any additional information stored along with host data that can aid in retrieval, organization, health of, and current condition or state of the memory devices 110. The metadata pages 508 can include but is not limited to a logical to physical mapping information of the host data within the memory devices 110. The metadata pages 508 that are held within one of the super blocks 502 can represent metadata specific to the one of the super blocks 502.
During portions of drive operations such as but not limited to drive initialization, the metadata pages 508 can be used to restore a host logical to physical mapping of data and a valid state of data stored within the one of the super blocks 502. The metadata pages 508 can also be used for storing the health and use status of the super blocks 502. The metadata pages 508 can be used to record the current recycling or runtime state of the super blocks 502.
For illustration purposes, the metadata pages 508 are labeled by “page-63” to indicate that the metadata pages 508 are stored in page 63 of one of the erase blocks 408 having 64 pages, although it is understood that the metadata pages 508 can be stored in the last page or any of the pages in the erase blocks 408. This allowed the firmware to know a priori where the metadata was held and could be accessed. In this example, this can allow firmware to be constructed to always search in a predetermined location for metadata.
Recycling operations performed by the memory access module 308 can traverse or access data by following the same order by which host-write operations follow. As such, the original striping can be preserved. A number of operations for writing to the L2P log can be minimized.
The super-block organization is conducive to implementing RAID 5 within the storage drive. The last instance of the erase blocks 408 in each of the super blocks 502 can include parity pages. For example, parity can be generated using exclusive-OR operations (XOR). The memory controller 104 can include support for initializing, building, and writing the parity pages.
Factory and run-time defects can be handled by the memory access module 308. If one of the erase blocks 408 within one of the super blocks 502 is bad, the entirety of the one of the super blocks 502 can optionally be thrown away. In addition, bad instances of the erase blocks 408 in the one of the super blocks 502 can be skipped or ignored by the memory access module 308. For example, the memory access module 308 can be implemented with the software 212 of
Statistics for the erase blocks 408 can be used by the block control module 306 to estimate remaining lifetime of the storage drive or to extend the life of the storage drive if the super blocks 502 are provided with erase-block granularity. Depending on how the memory devices 110 are populated in the storage drive with reduced capacity, the chip selects 112 of the memory devices 110 that are not used can be processed by the block control module 306.
Each of the super blocks 502 can include the program-erase count 510. The program-erase count 510 and the valid-page count 512 can be managed at a super-block level by the block control module 306. The program-erase count 510 can be calculated by the block control module 306 each time the memory devices 110 are accessed for a program operation or an erase operation by the memory access module 308. The program-erase count 510 can be used by the block control module 306 to determine if wear begins to deteriorate the integrity of the memory devices 110.
The super blocks 502 can include any number of unusable blocks 514, which are defined as blocks that are no longer available for access. If one of the erase blocks 408 is one of the unusable blocks 514, writing and recycling operations performed by the memory access module 308 on the corresponding instance of the super blocks 502 can skip over the one of the erase blocks 408.
The super blocks 502 can include remapped blocks for the unusable blocks 514. The erase blocks 408 can represent spare blocks 516, which are blocks that are available for use. If the memory devices 110 include the spare blocks 516, one of the erase blocks 408 that is one of the unusable blocks 514 can be remapped to one of the spare blocks 516 by the memory access module 308. Remapping is preferably not spanned over different instances of the memory devices 110.
There can be multiple levels of performance improvements having to do with both the super blocks 502 and the use of remapping of the erase blocks 408. The performance improvements from the use of the super blocks 502 can be from but not limited to the centralized storage of the metadata pages 508 in a single location instead of in each of the erase blocks 408 in the super blocks 502. Each access to one of the erase blocks 408 can incur a latency overhead of reading and writing. By only having to access one of the erase blocks 408 in the super blocks 502, there can be an effective elimination of multiple erase block accesses to retrieve the same amount of information. For host data reading and writing, the remapping function can allow each of the super blocks 502 to have a full compliment of the erase blocks 408. All of the erase blocks 408 within one of the super devices 406 can be accessed in a parallel approach with the one of the super devices 406 active.
It has been discovered that the super blocks 502 spanning multiple of the memory channels 404 provide improved performance since multiple pages within each of the super blocks 502 are capable of being written concurrently by the memory access module 308.
It has also been discovered that multiple of the erase blocks 408 grouped into each of the super blocks 502 by the block control module 306 provide improved performance with reduced data management resources since there are fewer of the super blocks 502 than a number of the erase blocks 408 to manage and initialize by the block control module 306.
It has further been discovered that the super blocks 502 spanning multiple of the chip selects 112 of the memory devices 110 provide improved reliability with an infrastructure that supports an internal redundant array of independent disks (RAID) architecture including RAID-5.
It has further been discovered that the super blocks 502 including a number of the erase blocks 408 that are bad or remapped provides improved memory resource utilization since the super blocks 502 with the unusable blocks 514 do not have to be thrown away but instead the unusable blocks 514 are able to be remapped with the spare blocks 516 by the memory access module 308 thereby eliminating the shortcoming of traditional super blocks where a bad block renders the entire super block useless.
It has further been discovered that the super blocks 502 provide improved performance since there are fewer statistics, including the program-erase count 510 and the valid-page count 512, included in the metadata pages 508 that need to be managed by the block control module 306.
It has further been discovered that remapping of the unusable blocks 514 performed by the memory access module 308 provide improved performance since the super blocks 502 that are incomplete cause write performance degradation.
Referring now to
The super blocks 502 of
For example, the super blocks 502 can represent a 1024-page erase block. In this example, there is one of the metadata pages 508 of
For example, “8K” mentioned above refers to a number of user bytes in a NAND flash page. In the memory devices 110, pages can include a power of 2 bytes of user data and additional bytes for use by ECC and the metadata pages 508. “8K” refers to 8192 bytes. A page can include 8192+640 bytes of storage, in which case 8K bytes (8192) can be used for host data and 640 extra bytes can be used for but not limited to storing logical block address (LBA) information, ECC parity bytes, and NAND health information. The grouping of data pages in a given size can allow software tabling to be regularly organized. The term “regularly” refers to in this case consistent in size.
The super blocks 502 can be used to implement RAID-5 architectures within the storage drive. The parity pages can be stored by the memory access module 308 in the last instance of the erase blocks 408 within each of the super blocks 502. All the parity pages would be included in the same instance of the erase blocks 408.
For example, data can be stored by the memory access module 308 in the first six instances of the memory channels 404 in one of the super blocks 502. In this example, either data or optionally one of the metadata pages 508 can be stored by the memory access module 308 in the seventh instance of the memory channels 404 and a parity page can be stored by the memory access module 308 in the eighth or last instance of the memory channels 404.
Since the instance of the erase blocks 408 can be used or read only if there are uncorrectable error correction code (ECC) errors, the instance of the erase blocks 408 would not experience read disturbs. By having parity stored along with the super blocks 502, the erase blocks 408 with the parity can be erased along with the rest of the super blocks 502 during a recycling process performed by the memory access module 308.
When RAID architecture is used by the memory access module 308 for protection against a die or device, page, or erase block failure, a RAID stripe operation can be performed such that a parity page is not stored in the same instance of the memory devices 110 that are used to store data pages. After performing iteration through the super blocks 502, the parity page can be written by the memory access module 308. As a result, a RAID buffer can be used to maintain the current parity page by the memory access module 308 for each instance of the super blocks 502 that is open.
The super blocks 502 that are open are blocks that are still under construction. In other words, the super blocks 502 are still being added to or being filled with user data or the metadata pages 508 have not been written. Any time there are unwritten pages in any of the erase blocks 408 that make up one of the super blocks 502, the one of the super blocks 502 is considered open. In other words, open also refers to not completely logically written. The super blocks 502 can be closed without writing all of the physical pages, but when the state of the super blocks 502 is set to “closed”, no more writing is allowed to take place, even if there are gaps or un-written pages within the super blocks 502.
For example, the memory access module 308 can include 12 RAID buffers for building or generating parity pages. As a result, 12 different RAID stripes can be operated at any given time. When RAID is not implemented on a super block basis, larger sizes of the super blocks 502 and less space for the parity pages can be used by the memory access module 308.
Referring now to
The logical pages 702 are different from the physical pages 704. The logical pages 702 do not represent the physical arrangement of pages in the memory devices 110. On the other hand, the physical pages 704 represent the physical arrangement of pages in the memory devices 110. For example, the physical pages 704 can represent flash pages.
The example diagram depicts a first physical instance, denoted as “Device 0”, of the memory devices 110. The first physical instance can include 128 of the physical pages 704, each of which spans across 8 of the memory channels 404 of
As an example, 8K of the physical pages 704 in Device 0 can be mapped by the block control module 306 to two 4K of the logical pages 702, where 1K is equal to 1024. As another example, one of the metadata pages 508 can be mapped by the block control module 306 to one of the physical pages 704, denoted as “Ch 7, Device 0, Page 127”.
Referring now to
The memory access module 308 can include a flash translation layer (FTL), not shown, to recycle the super blocks 502 of
When recycling one of the super blocks 502, it is understood that there is no performance advantage to keep the logical pages 702 in the same instance of the memory channels 404 because the logical pages 702 are to be copied into an external memory outside the memory channels 404 by the memory access module 308. As such, the functional block diagram depicts some of the logical pages 702, denoted with strikethroughs, which are not kept and therefore not written to the external memory by the memory access module 308.
The external memory can be implemented in the memory access module 308. The external memory can be implemented using the storage unit 204. For example, the storage unit 204 can include Double Data Rate Random Access Memory (DDR RAM or DDRAM) or Scratch Pad Memory (SPM) to implement the external memory.
By having the super chip selects 504 of
There can be only one valid copy of the logical sector or page of host data. When a storage device or drive is formatted or freshly initialized, there would be no valid pages. Only after the storage device has been written to, are there valid pages. A special case is read accesses to a logical page not written can internally not have a valid page associated with them. In this special case, the storage device or drive can return a fixed uninitialized value to the host system 108. For example, a value of 0xFF in hexadecimal can be returned for an unwritten logical address space or a returned value can be set by special commands from the host system 108.
The functional block diagram depicts an example of first 16 of the logical pages 702 of one of the super blocks 502. The functional block diagram depicts how the recycle operation compact data by the memory access module 308. For clarity purposes, the metadata pages 508 of
The functional block diagram depicts recycle pages 802 and write pages 804. The recycle pages 802 are a number of the logical pages 702 that are not saved to the external memory by the memory access module 308. The memory channels 404 that include the recycle pages 802, denoted with strikethroughs, are to be made available for writing data, for example, either during the recycle operation or in subsequent write operations performed by the memory access module 308.
The write pages 804 are a number of the logical pages 702 that are kept and thus to be saved to the external memory by the memory access module 308. Subsequently, the write pages 804 can be written or saved by the memory access module 308 to first 8 of the memory channels 404, some of which are made available by not keeping the recycle pages 802.
The recycle operation performed by the memory access module 308 can rebalance the data between the memory channels 404. For example, let one of the super blocks 502 spans across eight instances of the memory channels 404, denoted as Channel 0 to Channel 7. Also for example, let Channel 0, Channel 1, Channel 2, Channel 3, Channel 4, Channel 5, Channel 6, and Channel 7 include a number of the logical pages 702 that are determined valid by the memory access module 308 as 0, 1, 2, 2, 0, 0, 2, and 1, respectively. The recycle operation can rebalance the data between the memory channels 404 with Channel 0, Channel 1, Channel 2, Channel 3, Channel 4, Channel 5, Channel 6, and Channel 7 having a number of the logical pages 702 that are valid as 1, 1, 1, 1, 1, 1, 1, and 1, respectively.
Using the super blocks 502, the data that are written can be kept together, for example, in large single blocks during the write operation and after the recycle operation are completed by the memory access module 308. By using the super blocks 502, data pages can be moved by the memory access module 308 in the same order that the host system 108 writes the data pages. As such, the number of writes to the L2P log can be minimized when recycling large-block writes is performed by the memory access module 308.
As for the erase blocks 408 of
The super blocks 502 can optionally be erased in smaller groups or even singularly to accommodate read operations requested by the host system 108 or conserve operation power. It is understood that the super devices 406 that are used in the erasure operation can be different from the super devices 406 that are used in the write operation requested by the host system 108. Therefore, the erasure or recycle operation does not affect host writes.
As an example, the memory devices 110 can represent 32-nanometer (nm) multi-level cell (MLC) flash memory devices. As another example, the memory devices 110 can include a multi-plane architecture.
For example, the memory devices 110 can be used to construct a 512-gigabyte (GB) drive. The memory devices 110 can include 4096 of the erase blocks 408 and additional blocks for the spare blocks 516 of
As an example, a number of the erase blocks 408 per each of the super blocks 502, a number of the super devices 406, a number of the super blocks 502, and a size of one of the metadata pages 508 can be 4, 32, 128K, and 4 KB, respectively. As another example, a number of the erase blocks 408 per each of the super blocks 502, a number of the super devices 406, a number of the super blocks 502, and a size of one of the metadata pages 508 can be 8, 16, 64K, and 8 KB, respectively. As yet another example, a number of the erase blocks 408 per each of the super blocks 502, a number of the super devices 406, a number of the super blocks 502, and a size of one of the metadata pages 508 can be 16, 8, 32K, and 16 KB, respectively.
In the examples above, each of the super blocks 502 having eight of the erase blocks 408 can be selected by the block control module 306 of
Referring now to
The memory access module 308 can include an idle module 902, a host-write module 904, a recycle-read module 906, and a recycle-write module 908. The idle module 902, the host-write module 904, the recycle-read module 906, or the recycle-write module 908 can be performed based on the operational states of the super devices 406. Regardless of the dimension of the super blocks 502 of
The state diagram depicts an operation of the memory access module 308 either with or without recycling operations. When recycling is not enabled, the super devices 406 can be operated and controlled by the idle module 902 or the host-write module 904. When recycling is enabled, the super devices 406 can be operated and controlled by the idle module 902, the host-write module 904, the recycle-read module 906, or the recycle-write module 908.
When the super devices 406 are controlled by the idle module 902, the super devices 406 are in a healthy state. If recycling is enabled, the super devices 406 are the healthiest from the recycle operation. The super devices 406 can be subsequently used for host writes. The healthy state of the super devices 406 will be subsequently described in more details.
When the super devices 406 are controlled by the host-write module 904, the super devices 406 are servicing host writes requested from the host system 108 of
When one of the super devices 406 is controlled by the recycle-read module 906, the one of the super devices 406 includes the super blocks 502 that are being recycled. Valid pages can be read out or the super blocks 502 can be erased by the recycle-read module 906. Each time a recycle operation performed for the super blocks 502 is completed, the memory access module 308 can determine if the super devices 406 are to be transitioned to a different state by transferring control to the appropriate module in the memory access module 308.
When one of the super devices 406 is controlled by the recycle-write module 908, the one of the super devices 406 is writing relocated valid pages from another of the super devices 406 that is controlled by the recycle-read module 906. Instances of the super devices 406 that are being written can be different from instances of the super devices 406 that are being read so that the next set of recycle pages can be read while the current set of pages are being written. As such, the super devices 406 can be rebalanced by the memory access module 308.
A recycle operation is a process of reclaiming the super blocks 502 that are older or written a longer time in the past. The super blocks 502 that are reclaimed can have only a portion of pages valid or the age of the super blocks 502 can require the data to be moved for retention reasons. In this process, one or more of the super blocks 502 can be identified to read valid pages from and rewrite them to a new open super block. During the recycle operation, valid pages can be read from a donor super block and written to a new open super block, and stale or invalid pages (pages where there is a new temporal copy) are not read from (or moved out of) the donor super block.
Partitioning the super devices 406 into the states described above by the memory access module 308 can avoid several conflicts. For example, a host-write operation performed by the host-write module 904 conflict with an erase operation performed either by the recycle-read module 906 or the recycle-write module 908 can be avoided because each of these operations is performed for different instances of the super devices 406. The memory access module 308 can optionally include a scheduler implementation in a case of conflict with host reads because the host reads can be associated with any of the erase blocks 408 of
The idle module 902, the recycle-read module 906, and the recycle-write module 908 form a loop. The healthy state of each of the super devices 406 can be determined based on a number of the super blocks 502 that are ready-for-use. In other words, the higher the number of the super blocks 502 that are ready-for-use, the “healthier” the super devices 406 are.
The super devices 406 in this loop work together to create healthier super devices. At any given time, the super devices 406 controlled by the idle module 902 can represent the healthiest (or close to the healthiest) super devices in this recycle loop. As a result, the memory access module 308 can swap the super devices 406 controlled by the host-write module 904 with the super devices 406 controlled by the idle module 902.
If state transitions negatively affect performance, hysteresis can be used so that the super devices 406 can stay in each state a little longer. The state transitions can negatively affect the performance because an increase of a number of state transitions cause an increase in execution time of an operation and thus decreasing an overall system throughput. For example, hysteresis can be provided by increasing a threshold numerical value for comparing an output of a function of state input parameters resulting in a number of the state transitions would decrease thereby improving the performance.
As for remapping bad blocks, a number of the erase blocks 408 that are available in a die or LUN can be allocated so that they can be used in place of factory or run-time defects. For example, 56 of the erase blocks 408 can be saved and be used to replace defective blocks.
For example, since there can be over 56 bad blocks over the lifetime of the memory devices 110 of
Remapping bad blocks can optionally be performed by the memory access module 308 to provide an advantage of increased performance since an incomplete super block decreases write performance. When bad blocks are remapped, the memory access module 308 can interface with the block control module 306 of
The block control module 306 can be used to determine which of the erase blocks 408 can be used to replace the bad blocks. The erase blocks 408 used for replacing the bad blocks can include those that make up or are included in the super blocks 502. Such instances of the erase blocks 408 can include the erase blocks 408 that are previously remapped.
The block control module 306 can also determine instances of the super blocks 502 to which the erase blocks 408 belong. This operation is performed when super-block statistics, including the valid-page count 512 of
A mapping operation of the bad blocks can be captured in a persistent manner, for example, using a serial data link protocol including a Serial Peripheral Interface (SPI) or any other synchronous serial data link protocol by the memory access module 308. Persistent manner refers to storage of information across power cycles. Configuration information and operational firmware can be stored in an SPI device, which can represent a NOT-OR (NOR) type non-volatile memory or a non-volatile low power flash memory device. The SPI device can be separate from the memory devices 110 that are used for storing host data. The SPI device can be seldom written and can have extremely long persistence, which is good for but not limited to storing the configuration information and the operational firmware.
The mapping data structures can be rebuilt during initialization by the memory access module 308. In order for the remapping operation to be beneficial, the cost of using the mapping data structures is less than the theoretical increased performance.
Functions or operations of the memory controller 104 as described above can be implemented with the control unit 202 of
As an example, the block control module 306 can be implemented with the control unit 202 to select the super blocks 502 associated with only one instance of the super devices 406 or the super chip selects 504 of
As an example, the block control module 306 can be implemented with the control unit 202 to select the super blocks 502 having the program-erase count 510 of
For example, the idle module 902 can be implemented with the control unit 202 to operate and control the super devices 406. Also for example, the host-write module 904 can be implemented with the control unit 202 to operate and control the super devices 406. Further for example, the recycle-read module 906 can be implemented with the control unit 202 to control the super devices 406, erase the super blocks 502. Yet further for example, the recycle-write module 908 can be implemented with the control unit 202 to control the super devices 406 and write the relocated valid pages from the another of the super devices 406 that is controlled by the recycle-read module 906.
Referring now to
Thus, it has been discovered that the storage control system 100 of the present invention furnishes important and heretofore unknown and unavailable solutions, capabilities, and functional aspects for a storage control system with data management mechanism. The resulting method, process, apparatus, device, product, and/or system is straightforward, cost-effective, uncomplicated, highly versatile, accurate, sensitive, and effective, and can be implemented by adapting known components for ready, efficient, and economical manufacturing, application, and utilization.
Another important aspect of the present invention is that it valuably supports and services the historical trend of reducing costs, simplifying systems, and increasing performance.
These and other valuable aspects of the present invention consequently further the state of the technology to at least the next level.
While the invention has been described in conjunction with a specific best mode, it is to be understood that many alternatives, modifications, and variations will be apparent to those skilled in the art in light of the aforegoing description. Accordingly, it is intended to embrace all such alternatives, modifications, and variations that fall within the scope of the included claims. All matters hithertofore set forth herein or shown in the accompanying drawings are to be interpreted in an illustrative and non-limiting sense.
This application claims the benefit of U.S. Provisional Patent Application Ser. No. 61/596,185 filed Feb. 7, 2012, and the subject matter thereof is incorporated herein by reference thereto.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4048481 | Bailey, Jr. et al. | Sep 1977 | A |
| 4839587 | Flatley et al. | Jun 1989 | A |
| 5034744 | Obinata | Jul 1991 | A |
| 5210854 | Beaverton et al. | May 1993 | A |
| 5311395 | McGaha et al. | May 1994 | A |
| 5450354 | Sawada et al. | Sep 1995 | A |
| 5479638 | Assar et al. | Dec 1995 | A |
| 5784174 | Fujino et al. | Jul 1998 | A |
| 5790828 | Jost | Aug 1998 | A |
| 5930504 | Gabel | Jul 1999 | A |
| 5949785 | Beasley | Sep 1999 | A |
| 5963983 | Sakakura et al. | Oct 1999 | A |
| 6034897 | Estakhri et al. | Mar 2000 | A |
| 6069827 | Sinclair | May 2000 | A |
| 6091652 | Haehn et al. | Jul 2000 | A |
| 6275436 | Tobita et al. | Aug 2001 | B1 |
| 6345367 | Sinclair | Feb 2002 | B1 |
| 6356447 | Scafidi | Mar 2002 | B2 |
| 6381176 | Kim et al. | Apr 2002 | B1 |
| 6381670 | Lee et al. | Apr 2002 | B1 |
| 6412080 | Fleming et al. | Jun 2002 | B1 |
| 6529997 | Debiez | Mar 2003 | B1 |
| 6552581 | Gabara | Apr 2003 | B1 |
| 6587915 | Kim | Jul 2003 | B1 |
| 6618249 | Fairchild | Sep 2003 | B2 |
| 6661503 | Yamaguchi et al. | Dec 2003 | B1 |
| 6728913 | Parker | Apr 2004 | B1 |
| 6763424 | Conley | Jul 2004 | B2 |
| 6775792 | Ulrich et al. | Aug 2004 | B2 |
| 6778387 | Fairchild | Aug 2004 | B2 |
| 6850443 | Lofgren et al. | Feb 2005 | B2 |
| 6854070 | Johnson et al. | Feb 2005 | B2 |
| 6871304 | Hadjihassan et al. | Mar 2005 | B2 |
| 6903972 | Lasser et al. | Jun 2005 | B2 |
| 6906961 | Eggleston et al. | Jun 2005 | B2 |
| 6975028 | Wayburn et al. | Dec 2005 | B1 |
| 7082495 | DeWhitt et al. | Jul 2006 | B2 |
| 7107389 | Inagaki et al. | Sep 2006 | B2 |
| 7139864 | Bennett et al. | Nov 2006 | B2 |
| 7233497 | Simon et al. | Jun 2007 | B2 |
| 7243186 | Liang et al. | Jul 2007 | B2 |
| 7298888 | Hamar | Nov 2007 | B2 |
| 7330927 | Reeve et al. | Feb 2008 | B1 |
| 7333364 | Yu et al. | Feb 2008 | B2 |
| 7350101 | Nguyen et al. | Mar 2008 | B1 |
| 7355896 | Li et al. | Apr 2008 | B2 |
| 7434122 | Jo | Oct 2008 | B2 |
| 7441067 | Gorobets et al. | Oct 2008 | B2 |
| 7516267 | Coulson et al. | Apr 2009 | B2 |
| 7613871 | Tanaka et al. | Nov 2009 | B2 |
| 7620710 | Kottomtharayil et al. | Nov 2009 | B2 |
| 7620769 | Lee et al. | Nov 2009 | B2 |
| 7639532 | Roohparvar et al. | Dec 2009 | B2 |
| 7661054 | Huffman et al. | Feb 2010 | B2 |
| 7679948 | Park et al. | Mar 2010 | B2 |
| 7693422 | Alicherry et al. | Apr 2010 | B2 |
| 7738502 | Chang et al. | Jun 2010 | B2 |
| 7743216 | Lubbers et al. | Jun 2010 | B2 |
| 7818525 | Frost et al. | Oct 2010 | B1 |
| 7827348 | Lee et al. | Nov 2010 | B2 |
| 7830164 | Earle et al. | Nov 2010 | B2 |
| 7853749 | Kolokowsky | Dec 2010 | B2 |
| 7979614 | Yang | Jul 2011 | B1 |
| 8000161 | Stan et al. | Aug 2011 | B2 |
| 8001135 | Perlmutter et al. | Aug 2011 | B2 |
| 8010738 | Chilton et al. | Aug 2011 | B1 |
| 8028123 | Kilzer et al. | Sep 2011 | B2 |
| 8046645 | Hsu et al. | Oct 2011 | B2 |
| 8051241 | Feldman et al. | Nov 2011 | B2 |
| 8072805 | Chou et al. | Dec 2011 | B2 |
| 8095724 | Ji et al. | Jan 2012 | B2 |
| 8095765 | Asnaashari et al. | Jan 2012 | B2 |
| 8117396 | Fair et al. | Feb 2012 | B1 |
| 8127202 | Cornwell et al. | Feb 2012 | B2 |
| 8145984 | Sommer et al. | Mar 2012 | B2 |
| 8154921 | Mokhlesi et al. | Apr 2012 | B2 |
| 8169825 | Shalvi et al. | May 2012 | B1 |
| 8205028 | Sakarda | Jun 2012 | B1 |
| 8209677 | Shintani et al. | Jun 2012 | B2 |
| 8219724 | Caruso et al. | Jul 2012 | B1 |
| 8219776 | Forhan et al. | Jul 2012 | B2 |
| 8228701 | Sokolov et al. | Jul 2012 | B2 |
| 8245101 | Olbrich et al. | Aug 2012 | B2 |
| 8250621 | Cha | Aug 2012 | B2 |
| 8254172 | Kan | Aug 2012 | B1 |
| 8259506 | Sommer et al. | Sep 2012 | B1 |
| 8289801 | Smith et al. | Oct 2012 | B2 |
| 8296534 | Gupta et al. | Oct 2012 | B1 |
| 8332578 | Frickey, III et al. | Dec 2012 | B2 |
| 8363413 | Paquette et al. | Jan 2013 | B2 |
| 8369141 | Sommer et al. | Feb 2013 | B2 |
| 8386700 | Olbrich et al. | Feb 2013 | B2 |
| 8386860 | Tseng et al. | Feb 2013 | B2 |
| 8397101 | Goss et al. | Mar 2013 | B2 |
| 8407409 | Kawaguchi | Mar 2013 | B2 |
| 8464106 | Filor et al. | Jun 2013 | B2 |
| 8503238 | Wu et al. | Aug 2013 | B1 |
| 8521981 | Strauss et al. | Aug 2013 | B2 |
| 8560770 | Haines et al. | Oct 2013 | B2 |
| 8601203 | Holbrook et al. | Dec 2013 | B2 |
| 8612669 | Syu et al. | Dec 2013 | B1 |
| 8612804 | Kang | Dec 2013 | B1 |
| 8661184 | Wood et al. | Feb 2014 | B2 |
| 8694811 | Raju et al. | Apr 2014 | B2 |
| 8750052 | Aoki et al. | Jun 2014 | B2 |
| 8793556 | Northcott et al. | Jul 2014 | B1 |
| 8799747 | Goss et al. | Aug 2014 | B2 |
| 8832506 | Griffin et al. | Sep 2014 | B2 |
| 8862818 | Ozdemir | Oct 2014 | B1 |
| 8880838 | Kaiser et al. | Nov 2014 | B2 |
| 8984216 | Fillingim | Mar 2015 | B2 |
| 9043668 | Goss et al. | May 2015 | B2 |
| 9063844 | Higgins et al. | Jun 2015 | B2 |
| 9069468 | Mehra et al. | Jun 2015 | B2 |
| 9116401 | Kim et al. | Aug 2015 | B2 |
| 20020056025 | Qiu et al. | May 2002 | A1 |
| 20020156891 | Ulrich et al. | Oct 2002 | A1 |
| 20020159285 | Morley et al. | Oct 2002 | A1 |
| 20030033308 | Patel et al. | Feb 2003 | A1 |
| 20030046603 | Harari et al. | Mar 2003 | A1 |
| 20030074592 | Hasegawa | Apr 2003 | A1 |
| 20030163633 | Aasheim et al. | Aug 2003 | A1 |
| 20040080985 | Chang et al. | Apr 2004 | A1 |
| 20040088511 | Bacon et al. | May 2004 | A1 |
| 20040252670 | Rong et al. | Dec 2004 | A1 |
| 20050021904 | Iaculo et al. | Jan 2005 | A1 |
| 20050038792 | Johnson | Feb 2005 | A1 |
| 20050073884 | Gonzalez et al. | Apr 2005 | A1 |
| 20050076102 | Chen et al. | Apr 2005 | A1 |
| 20050144516 | Gonzalez et al. | Jun 2005 | A1 |
| 20060015683 | Ashmore et al. | Jan 2006 | A1 |
| 20060020745 | Conley et al. | Jan 2006 | A1 |
| 20060022054 | Elhamias et al. | Feb 2006 | A1 |
| 20060080505 | Arai et al. | Apr 2006 | A1 |
| 20060136682 | Haridas et al. | Jun 2006 | A1 |
| 20060143365 | Kikuchi | Jun 2006 | A1 |
| 20060143475 | Herbert et al. | Jun 2006 | A1 |
| 20060253641 | Gatzemeier et al. | Nov 2006 | A1 |
| 20060256624 | Eggleston et al. | Nov 2006 | A1 |
| 20060282644 | Wong | Dec 2006 | A1 |
| 20060294574 | Cha | Dec 2006 | A1 |
| 20070050536 | Kolokowsky | Mar 2007 | A1 |
| 20070061511 | Faber | Mar 2007 | A1 |
| 20070067598 | Fujimoto | Mar 2007 | A1 |
| 20070079152 | Winick et al. | Apr 2007 | A1 |
| 20070083779 | Misaka et al. | Apr 2007 | A1 |
| 20070226592 | Radke | Sep 2007 | A1 |
| 20070234004 | Oshima et al. | Oct 2007 | A1 |
| 20070260811 | Merry, Jr. et al. | Nov 2007 | A1 |
| 20070263444 | Gorobets et al. | Nov 2007 | A1 |
| 20070276973 | Tan et al. | Nov 2007 | A1 |
| 20080028246 | Witham | Jan 2008 | A1 |
| 20080046630 | Lasser | Feb 2008 | A1 |
| 20080052446 | Lasser et al. | Feb 2008 | A1 |
| 20080082736 | Chow et al. | Apr 2008 | A1 |
| 20080126720 | Danilak | May 2008 | A1 |
| 20080183918 | Dhokia et al. | Jul 2008 | A1 |
| 20080189588 | Tanaka et al. | Aug 2008 | A1 |
| 20080263289 | Hosoya et al. | Oct 2008 | A1 |
| 20080313505 | Lee et al. | Dec 2008 | A1 |
| 20090006900 | Lastras-Montano et al. | Jan 2009 | A1 |
| 20090019321 | Radke | Jan 2009 | A1 |
| 20090070651 | Diggs et al. | Mar 2009 | A1 |
| 20090083587 | Ng et al. | Mar 2009 | A1 |
| 20090089485 | Yeh | Apr 2009 | A1 |
| 20090091990 | Park et al. | Apr 2009 | A1 |
| 20090109786 | Ye et al. | Apr 2009 | A1 |
| 20090125670 | Keays | May 2009 | A1 |
| 20090138654 | Sutardja | May 2009 | A1 |
| 20090146721 | Kurooka et al. | Jun 2009 | A1 |
| 20090157948 | Trichina et al. | Jun 2009 | A1 |
| 20090164702 | Kern | Jun 2009 | A1 |
| 20090164710 | Choi et al. | Jun 2009 | A1 |
| 20090172248 | You | Jul 2009 | A1 |
| 20090172262 | Olbrich et al. | Jul 2009 | A1 |
| 20090179707 | Higashino | Jul 2009 | A1 |
| 20090228634 | Nakamura et al. | Sep 2009 | A1 |
| 20090228761 | Perlmutter et al. | Sep 2009 | A1 |
| 20090259819 | Chen et al. | Oct 2009 | A1 |
| 20090259896 | Hsu et al. | Oct 2009 | A1 |
| 20090271562 | Sinclair | Oct 2009 | A1 |
| 20090287975 | Kim et al. | Nov 2009 | A1 |
| 20090300238 | Panabaker et al. | Dec 2009 | A1 |
| 20090323419 | Lee et al. | Dec 2009 | A1 |
| 20090327581 | Coulson | Dec 2009 | A1 |
| 20090327591 | Moshayedi | Dec 2009 | A1 |
| 20100017650 | Chin et al. | Jan 2010 | A1 |
| 20100023674 | Aviles | Jan 2010 | A1 |
| 20100050053 | Wilson et al. | Feb 2010 | A1 |
| 20100082890 | Heo et al. | Apr 2010 | A1 |
| 20100122019 | Flynn et al. | May 2010 | A1 |
| 20100128537 | Suhail et al. | May 2010 | A1 |
| 20100138592 | Cheon | Jun 2010 | A1 |
| 20100165689 | Rotbard et al. | Jul 2010 | A1 |
| 20100169541 | Freikorn | Jul 2010 | A1 |
| 20100172179 | Gorobets et al. | Jul 2010 | A1 |
| 20100174845 | Gorobets et al. | Jul 2010 | A1 |
| 20100217898 | Priborsky et al. | Aug 2010 | A1 |
| 20100217915 | O'Connor et al. | Aug 2010 | A1 |
| 20100223531 | Fukutomi et al. | Sep 2010 | A1 |
| 20100228928 | Asnaashari et al. | Sep 2010 | A1 |
| 20100262792 | Hetzler et al. | Oct 2010 | A1 |
| 20100262795 | Hetzler et al. | Oct 2010 | A1 |
| 20100262875 | Hetzler et al. | Oct 2010 | A1 |
| 20100287328 | Feldman et al. | Nov 2010 | A1 |
| 20100293367 | Berke et al. | Nov 2010 | A1 |
| 20100312954 | Jeon et al. | Dec 2010 | A1 |
| 20100318719 | Keays et al. | Dec 2010 | A1 |
| 20100332726 | Wang | Dec 2010 | A1 |
| 20110002224 | Tamura | Jan 2011 | A1 |
| 20110016239 | Stenfort | Jan 2011 | A1 |
| 20110055455 | Post et al. | Mar 2011 | A1 |
| 20110055468 | Gonzalez et al. | Mar 2011 | A1 |
| 20110066788 | Eleftheriou et al. | Mar 2011 | A1 |
| 20110072423 | Fukata | Mar 2011 | A1 |
| 20110078393 | Lin | Mar 2011 | A1 |
| 20110099342 | Ozdemir | Apr 2011 | A1 |
| 20110107144 | Ohara | May 2011 | A1 |
| 20110131365 | Zhang et al. | Jun 2011 | A1 |
| 20110131447 | Prakash et al. | Jun 2011 | A1 |
| 20110132000 | Deane et al. | Jun 2011 | A1 |
| 20110138100 | Sinclair | Jun 2011 | A1 |
| 20110145473 | Maheshwari | Jun 2011 | A1 |
| 20110161775 | Weingarten | Jun 2011 | A1 |
| 20110190963 | Glassl et al. | Aug 2011 | A1 |
| 20110191522 | Condict et al. | Aug 2011 | A1 |
| 20110191649 | Lim et al. | Aug 2011 | A1 |
| 20110209032 | Choi et al. | Aug 2011 | A1 |
| 20110238892 | Tsai et al. | Sep 2011 | A1 |
| 20110239088 | Post | Sep 2011 | A1 |
| 20110258496 | Tseng et al. | Oct 2011 | A1 |
| 20110314219 | Ulrich et al. | Dec 2011 | A1 |
| 20110320687 | Belluomini et al. | Dec 2011 | A1 |
| 20120008401 | Katz et al. | Jan 2012 | A1 |
| 20120011336 | Saika | Jan 2012 | A1 |
| 20120023144 | Rub | Jan 2012 | A1 |
| 20120047318 | Yoon et al. | Feb 2012 | A1 |
| 20120047320 | Yoo et al. | Feb 2012 | A1 |
| 20120047409 | Post et al. | Feb 2012 | A1 |
| 20120066450 | Yochai et al. | Mar 2012 | A1 |
| 20120079348 | Naeimi | Mar 2012 | A1 |
| 20120079355 | Patapoutian et al. | Mar 2012 | A1 |
| 20120096217 | Son et al. | Apr 2012 | A1 |
| 20120124046 | Provenzano | May 2012 | A1 |
| 20120124273 | Goss et al. | May 2012 | A1 |
| 20120151260 | Zimmermann et al. | Jun 2012 | A1 |
| 20120170365 | Kang et al. | Jul 2012 | A1 |
| 20120185706 | Sistla et al. | Jul 2012 | A1 |
| 20120213004 | Yun et al. | Aug 2012 | A1 |
| 20120216085 | Weingarten et al. | Aug 2012 | A1 |
| 20120236656 | Cometti | Sep 2012 | A1 |
| 20120239858 | Melik-Martirosian | Sep 2012 | A1 |
| 20120254686 | Esumi et al. | Oct 2012 | A1 |
| 20120266011 | Storer et al. | Oct 2012 | A1 |
| 20120266048 | Chung et al. | Oct 2012 | A1 |
| 20120278530 | Ebsen | Nov 2012 | A1 |
| 20120278531 | Horn | Nov 2012 | A1 |
| 20120284587 | Yu et al. | Nov 2012 | A1 |
| 20120297113 | Belluomini et al. | Nov 2012 | A1 |
| 20120311402 | Tseng et al. | Dec 2012 | A1 |
| 20120317334 | Suzuki et al. | Dec 2012 | A1 |
| 20120324191 | Strange et al. | Dec 2012 | A1 |
| 20120331207 | Lassa et al. | Dec 2012 | A1 |
| 20130007380 | Seekins et al. | Jan 2013 | A1 |
| 20130007543 | Goss et al. | Jan 2013 | A1 |
| 20130054881 | Ellis et al. | Feb 2013 | A1 |
| 20130060994 | Higgins et al. | Mar 2013 | A1 |
| 20130061019 | Fitzpatrick et al. | Mar 2013 | A1 |
| 20130073788 | Post et al. | Mar 2013 | A1 |
| 20130080691 | Weingarten et al. | Mar 2013 | A1 |
| 20130094289 | Sridharan et al. | Apr 2013 | A1 |
| 20130100600 | Yin et al. | Apr 2013 | A1 |
| 20130104005 | Weingarten et al. | Apr 2013 | A1 |
| 20130124792 | Melik-Martirosian et al. | May 2013 | A1 |
| 20130151753 | Jeon et al. | Jun 2013 | A1 |
| 20130198436 | Bandic et al. | Aug 2013 | A1 |
| 20130205102 | Jones et al. | Aug 2013 | A1 |
| 20130232290 | Ish et al. | Sep 2013 | A1 |
| 20130238833 | Vogan et al. | Sep 2013 | A1 |
| 20130265825 | Lassa | Oct 2013 | A1 |
| 20130332791 | Chu | Dec 2013 | A1 |
| 20140036589 | Parthasarathy et al. | Feb 2014 | A1 |
| 20140059359 | Bahirat | Feb 2014 | A1 |
| 20140108891 | Strasser et al. | Apr 2014 | A1 |
| 20140129874 | Zaltsman et al. | May 2014 | A1 |
| 20140158525 | Greene | Jun 2014 | A1 |
| 20140181370 | Cohen et al. | Jun 2014 | A1 |
| 20140208174 | Ellis et al. | Jul 2014 | A1 |
| 20140372777 | Reller et al. | Dec 2014 | A1 |
| Number | Date | Country |
|---|---|---|
| 1 956 489 | Aug 2008 | EP |
| 1 990 921 | Nov 2008 | EP |
| 2 498 259 | Sep 2012 | EP |
| 2012129859 | Jul 2012 | JP |
| WO 2009042298 | Apr 2009 | WO |
| WO 2011156466 | Dec 2011 | WO |
| Entry |
|---|
| International Search Report and Written Opinion dated Jun. 12, 2014, received in PCT/US2014/018972, which corresponds to U.S. Appl. No. 13/779,352, 12 pages (Schmier). |
| International Search Report and Written Opinion dated May 14, 2014, received in International Patent Application No. PCT/US2014/017168, which corresponds to U.S. Appl. No. 14/076,115, 6 pages (Fitzpatrick). |
| International Search Report and Written Opinion dated May 14, 2014, received in International Patent Application No. PCT/US2014/017169, which corresponds to U.S. Appl. No. 14/076,148, 6 pages (Fitzpatrick). |
| Ulinktech, “ATA Command Table (in Alphabetic Order),” Feb. 6, 2011, https://web.archive.org/web/20110206060820/http://www.ulinktech.com/downloads/AT, 6 pages. |
| International Search Report and Written Opinion dated Aug. 22, 2014, received in International Patent Application No. PCT/US2014/032978, which corresponds to U.S. Appl. No. 14/081,992, 10 pages (Ellis). |
| International Search Report dated Mar. 25, 2014, received in International Patent Application No. PCT/US2013/072400, which corresponds to U.S. Appl. No. 13/690,337, 3 pages (Ellis). |
| Invitation to Pay Additional Fees dated Jul. 25, 2014, received in International Patent Application No. PCT/US2014/021290, which corresponds to U.S. Appl. No. 13/791,797, 8 pages (Dean). |
| International Search Report and Written Opinion dated Jul. 31, 2014, received in International Patent Application No. PCT/US2014/031465, which corresponds to U.S. Appl. No. 13/851,928, 13 pages (Ellis). |
| International Search Report and Written Opinion dated Jul. 31, 2014, received in International Patent Application No. PCT/US2014/033876, which corresponds to U.S. Appl. No. 13/861,326, 9 pages (Fitzpatrick). |
| Cooke, “Introduction to Flash Memory (T1A),” Flash Memory Summit, Aug. 22, 2008, Micron Technology, Inc., 102 pages. |
| Gal et al., “Algorithms and Data Structures for Flash Memories,” ACM Computing Surveys, Jun. 2005, vol. 37, No. 2, 30 pages. |
| IBM Corporation, “Systems Management, Work Management,” Version 5, Release 4, 9th Edition, Feb. 2006, pp. 1-21. |
| O'Brien, “SMART Storage Systems Optimus SAS Enterprise SSD Review,” SMART Storage Systems, Oct. 9, 2012, 44 pages. |
| Spanjer, “Flash Management—Why and How?” Smart Modular Technologies, Nov. 2009, http://www.scantec.de/fileadmin/pdf/Smart—Modular/Flash-Management.pdf, 14 pages. |
| Texas Instruments, “Power Management IC for Digital Set Top Boxes,” SLVSA10A, Sep. 2009, pp. 1-22. |
| International Search Report and written Opinion dated Dec. 20, 2013, received in PCT/US2013/045282, which corresponds to U.S. Appl. No. 13/493,949, 7 pages (Ellis). |
| International Search Report and Written Opinion dated Nov. 7, 2014, received in International Patent Application No. PCT/US2014/049732, which corresponds to U.S. Appl. No. 14/334,350, 13 pages (Fitzpatrick). |
| International Search Report and Written Opinion dated Oct. 17, 2014, received in International Patent Appilcation No. PCT/US2014/049734, which corresponds to U.S. Appl. No. 14/332,259, 8 pages (Higgins). |
| International Search Report and Written Opinion dated Oct. 23, 2014, received in International Patent Application No. PCT/US2014/049736, which corresponds to U.S. Appl. No. 14/446,249 8 pages (Fitzpatrick). |
| International Search Report and Written Opinion dated Nov. 5, 2014, received in International Patent Application No. PCT/US2014/049282, which corresponds to U.S. Appl. No. 13/957,407, 12 pages (Fitzpatrick). |
| Narayanan et al., “Migrating Server Storage to SSDs: Analysis of Tradeoffs,” Computer Systems, Apr. 2009, 12 pages. |
| Shiraz et al., “Block Aging Prevention Technique (BAP) for Flash Based Solid State Disks,” 7th International Conference on Emerging Technologies (ICET), Sep. 5, 2011, 6 pages. |
| Tai et al, “Prolongation of Lifetime and the Evaluation Method of Dependable SSD,” 25 International Symposium on Defect and Fault Tolerance in VLSI Systems, 2010, NJ, USA, 8 pages. |
| Tseng et al., “Understanding the Impact of Power Loss on Flash Memory,” DAC'11, Jun. 5-10, 2011, San Diego, California, 6 pages. |
| Yimo et al., “WeLe-RAID: A SSD-Based RAID for System Endurance and Performance,” Jan. 2011, Network and Parallel Computing, Springer, 14 pages. |
| International Search Report and Written Opinion dated Jan. 9, 2015, received in International Patent Application No. PCT/US2014/049731, which corresponds to U.S. Appl. No. 14/334,324, 9 pages (Fitzpatrick). |
| International Search Report and Written Opinion dated Feb. 18, 2015, received in International Patent Application No. PCT/US2014/065401, which corresponds to U.S. Appl. No. 14/082,031, 9 pages (Higgins). |
| International Search Report dated Apr. 15, 2014, received in International Patent Application No. PCT/US2013/078340, which corresponds to U.S. Appl. No. 13/746,542, 11 pages (Ellis). |
| Number | Date | Country | |
|---|---|---|---|
| 20130205102 A1 | Aug 2013 | US |
| Number | Date | Country | |
|---|---|---|---|
| 61596185 | Feb 2012 | US |
