Systems for managing access to solid-state memory often perform complicated tasks such as maintaining drive coherency and mapping logical to physical addresses. In addition to receiving and processing memory access requests from host systems, these management systems often also process memory access requests related to internal maintenance operations such as garbage collection and wear leveling. The management systems may also be configured to respond to changing conditions in the physical memory media, including handling return status of memory commands executed in the media.
Systems and methods which embody the various features of the invention will now be described with reference to the following drawings, in which:
While certain embodiments of the inventions will be described, these embodiments are presented by way of example only, and are not intended to limit the scope of the inventions. Indeed, the novel methods and systems described herein may be embodied in a variety of other forms. Furthermore, various omissions, substitutions and changes in the form of the methods and systems described herein may be made without departing from the spirit of the inventions. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the inventions.
Embodiments of the invention described herein are directed to pipelined or multi-staged systems and methods for controlling and managing access to physical memory media in solid-state storage subsystems. In an embodiment, the system separates media access request management tasks such as address translation, preparation for execution at the physical memory media, and completion status reporting into discrete stages and processing components to improve performance. In an embodiment, the components in the various stages of processing media access requests cooperate with one another by using a common data structure termed a media access request object to share information on a pending request that is being processed and report any result data and/or completion status information to an entity that originated the media access request. In an embodiment, the components in the various stages of processing media access requests are executed on different processors and/or processor threads. In an embodiment, the system manages the execution timing of requests on memory units (e.g., dies) of a solid-state storage subsystem to improve concurrent use of the memory units.
An overview of a solid state storage subsystem 120 implementing a multi-stage media access control system embodiment is presented in
As shown in this embodiment, the plurality of media requesters 101 send media access requests to a media access request manager 102 in the storage subsystem 120. In an embodiment, the media access request manager 102 performs any necessary processing including address translation and ensures the media requests can be executed on the memory arrays 105. The address translation performed by the media access request manager 102 can include translation from a logical to a physical address. In an embodiment, once translation is complete, the media access request manager 102 then sends the media access requests to a media access request executor 103 in the storage subsystem 120. The media access request executor 103 is in communication with a media controller 104, and in an embodiment the media access request executor 103 determines when to send media requests to the media controller 104 for execution in the memory arrays 105. In an embodiment, since the media access requests are embedded in media access request objects, the media access request executor, as part of the sending process, may also perform any required translation of data within the media access request objects to data in a format that is understandable by the media controller 104. The media controller 104 performs media requests on the memory arrays 105 and returns a status and/or an error message to the media access request executor 103. In an embodiment, the media access request executor 103 in turn returns the status/error message of media access requests to the plurality of media access requesters 101.
Storage Subsystem Overview with Media Request Objects and Pointers
In an embodiment, the shared pointer memory 202 includes a memory structure 202b with memory allocated to memory access request objects. The shared pointer memory 202 can also include a plurality of pointers to the memory access request objects. In an embodiment, each media access requester is associated with a dedicated pool of memory and may maintain its own free pointer pool.
As described below, in an embodiment, each component may access and update information in a memory access request object located in shared pointer memory 202. Each component's access is denoted by dotted lines in
Returning to the top of
Once the media access request object is populated with request data, the media access requesters 201 can then pass the pointer for the media access request object to the media access request manager 203. The passing of the pointer, rather than the object itself, reduces the memory footprint necessary to implement this shared message architecture and improves system performance. In an embodiment, the media access request manager 203 maintains an invalid page table 205 and a mapping table 204. The mapping table 204 maintains a mapping between logical and physical addresses, while the invalid page table 205 maintains a list of physical addresses that contain invalid data. Typically, addresses (e.g., pages) referenced by the invalid page table can be freed through a garbage collection operation and made ready for further use. In an embodiment, the media access request manager 203 processes the incoming requests and resolves any contention between requests submitted by the plurality of media access requesters 201. For example, if the execution of a request requires an alteration to the mapping table 204 and/or invalid page table 205, the media access request manager 203 may update the tables to reflect the alteration. In certain circumstances, a media request may specify a media operation on data that has been altered since the request was first issued by the originating media access requester. In these circumstances, the media access request manager 203 may alter the request to conform to the alterations, or the media access request manager 203 may reject the request due to the conflict and return the associated media access request pointer to the originating media access requester.
In an embodiment, the media access request manager 203 may also populate the physical address field of the media access request object with a physical address that corresponds to a logical address indicated in the media access request object. For a read operation, determining a physical address that corresponds to a logical address may involve a simple lookup in the mapping table 204. For write operations, the media access request manager 203 may maintain a list of upcoming physical write locations associated with the media access requester that submitted the request. In an embodiment, the upcoming physical write locations may be associated with an open group of blocks, and the media access request manager 203 can choose a write location from the open group of blocks that is currently assigned to the requester. In an embodiment, the media access request manager 203 is configured to assign one or more open groups of blocks to individual requesters 201. In further embodiments, other ways of assigning a write location to a logical address may be implemented in any way understood to one skilled in the art.
In an embodiment, after processing the request as described above, the media access request manager 203 passes the media access request pointer to a media access request executor 206. The media access request executor 206 may be in communication with a media controller 207 to execute commands on memory arrays 208. The media access executor 206 schedules execution of requests on the memory arrays 208. In an embodiment, each memory array 208 has an associated execution queue, which stores commands issued by the media controller 207 in response to requests dispatched by the media access executor 206. In an embodiment, the media access request executor 206 maintains a record of the outstanding commands in each execution queue, and passes requests to the media controller according to the outstanding commands in each execution queue. In some embodiments, the media access request executor 206 passes the media access request pointer to the media controller 207, and the media controller 207 executes a command specified by the referenced media access request object on the memory arrays 208. In other embodiments, the media access request executor 206 translates data within the media access request object into a format executable by the media controller 207 on the memory arrays 208.
After execution of the media request on the memory arrays 208, the memory arrays 208 and/or media controller 207 in an embodiment return result data such as a status (e.g., indicating success or failure) and any other information associated with the execution to the media access request executor 206. For example, associated information for a read operation may include the data read from the memory arrays. In certain embodiments, the media access request executor 206 populates a status field in the media access request object with the returned result data and other returned information. In other embodiments, the media controller 207 populates the media request object data fields with result data.
Once result data is inserted into the media request object, in an embodiment the associated media access request pointer can then be passed from the media access request executor 206 to a request status pool/status reporter 209. In an embodiment, the media access requesters 201 will poll the request status reporter/pool 209 to retrieve any updated status for requests referenced by media access request pointers associated with that requester. In another embodiment, the request status pool/reporter can push status information to the associated requester(s).
As the media access request object is passed among the components of the media request control system within the storage subsystem, various portions of the media access request object are populated. For example, the media access requester may populate fields such as the requester ID data field 406 (indicating the originating requester), the operation data field 408, and the logical chunk number data field 402. For a write request, the media access requester may populate data to be written into buffers 411. In an embodiment, when the media access request object is passed to the media access request manager, the physical chunk number field 404, and the plane field 405 are populated. In an embodiment, if the request cannot be executed, the media access request manager can update the result data field 407. After the execution of the media request in the physical memory media, the media access executor and/or the media controller can update the result data field 408, and for some operations, e.g., a read operation, the buffers 411 with data read from the target physical address.
A detailed view of an embodiment of the media access request manager is shown in
In an embodiment, the media access request manager 505 receives media access request pointers from the request queues 504a-c. After receiving the media access request pointers, the media access request manager 505 in an embodiment updates the media access request object with reference to a mapping table 506 and an invalid page table 507. In an embodiment, the media access request manager 505 maintains a designated write location for each requester or request queue. In an embodiment, the media access request manager 505 determines a physical address for the write location corresponding to an open block in a group of blocks for each requester or request queue. In an embodiment, after receiving a write command request at the media access request manager 505, the mapping table 506 is updated with the new physical address corresponding to the logical address of the request. The invalid page table 507 is then updated to indicate that the physical address formerly mapped to the logical address is invalid. In an embodiment, the media access request manager 505 also updates an invalid page counter for the block or group of blocks corresponding to the old physical address. In an embodiment, the media access request manager 505 is the only entity with access rights to write to the mapping table 506 and the invalid page table 507. Restricting write accesses to the media access request manager 505 ensures a single point of control for these tables, and prevents possible contentions and data coherency errors that may result if the requesters were granted direct access to the tables.
In an embodiment, when the media access request manager 505 receives media access request pointers from the request queues 504a-c, the media access request manager 505 may determine a ratio at which to process media access request pointers from each queue. While requests from each requester must generally be executed in-order relative to other requests from that requester, requests from each requester may generally be executed in a different order than received relative to requests from another requester. For example, if a read command from the user data manager 502 is directed to the same logical address as a garbage collection command from the garbage collector 503, the media access request manager 505 can ensure that the physical address of each command is updated to appropriately compensate for the order of execution. If the garbage collection command is processed first, the read command can be directed to the new physical address corresponding to the same logical address. If the read command is processed first, the read command is executed prior to any data movement associated with the garbage collection command and revisions to the physical addresses of the data. As such, the media access request manager ensures coherency with the mapping and invalid page tables, thereby preventing downstream request execution conflicts.
In an embodiment, the option of processing requests from the queues in different ratios allows the media access request manager 505 to throttle, or provide dynamic ratios, of requests received from each request queue 504 and thereby prioritize a specific requester over other requesters. For example, the media access request manager 505 can receive and process two or three or more requests from the user data manager 502 for every request from garbage collector 503 (or vice versa). In an embodiment, the media access request manager 505 temporarily halts receiving media access request pointers from a request queue when certain conditions are met. For example, if the invalid page table shows a high need for garbage collection, the requests for user data manager 502 may be throttled down or halted to allow for more garbage collection requests to be processed. Alternatively, if the user data manager 502 has an unusually high number of requests, or if the request queue 504a for the user data manager 502 is nearing a threshold, the requests associated with the garbage collector 503 may be throttled down or halted. After preparing the media access request for execution, the media access request manager 505 passes the media access request pointer to the media access request executor.
In an embodiment, when the media access request executor 602 has determined that a media access request should be executed, the media access request executor 602 will pass the media access request to media controller 605. As described above, in certain embodiments the media access request executor 602 can pass a media access request pointer to media controller 605, while in other embodiments the media access request executor 602 performs a translation to a format acceptable to the media controller 605. Media controller 605 directs each media access request to an execution queue 606 associated with each memory arrays 607. Each memory array 607 may be accessed via an associated data channel. The execution queue 606 for each memory array 607 may be maintained on the memory arrays 607, or may be maintained as a part of media controller 605. As media access requests are completed on memory arrays 607, the media controller 605 will return a status and any associated information to the media access request executor 602. As necessary, the media access request executor 602 incorporates the status and associated information into the media access request object and passes the media access request pointer to status pool 608. In an embodiment, the media controller 605 populates the status and associated information fields of the media access request object instead of the media access request executor 602. As described above, the media access requesters receive media access request pointers from the status pool 608.
One example method of determining when to execute media access requests is shown in
In various embodiments, each memory unit is capable of having a certain threshold number of outstanding requests at once. As such, the media access request executor cannot send additional requests past this outstanding request threshold. The outstanding request threshold is typically dictated by the depth of the execution queue 606 depicted in
In an embodiment, as the request list is traversed, the current position in the request list will reach a reset point and return to the first skipped request to determine if that request can now be executed. The reset point can be determined in a variety of ways. The current position can return to the first skipped request upon the triggering of a condition. As examples, the condition can be based at least partially on any of: the number of requests that have been skipped (in total or for a particular memory unit), receiving an indication notifying of a request completion from a memory unit that has been skipped, and reaching the end of the request list. In various embodiments, as the request list is continually changing to reflect processed requests and new incoming requests from the media access request manager, periodic snapshots of the request list are taken and the aforementioned condition for reset may also be based on reaching the end of a snapshot of the request list.
In alternate embodiments, other methods for maintaining a list of requests to be executed may be used. For example, at the media access request executor level, the received requests may be placed in individual queues associated with each memory unit. The media access request executor may then send media access requests from the queues to the media controller by checking whether the associated media unit has reached a threshold in the media execution counter. In an embodiment, the media access request executor may check only those queues which have not reached the threshold in the execution counter. In an embodiment, the media access request executor checks the queues associated with each memory unit by a round robin algorithm. A flag can be used to designate which memory units have an execution counter that has reached the threshold. According to another embodiment, the media access request executor reorders the media access requests in the request list at least partly based on the status of the associated memory units. The status of the associated memory units may be determined with reference to the execution counters. Other data management means can be used to control the flow of media requests to the media controller in accordance with this disclosure.
By temporarily skipping certain requests, the media access request executor increases the concurrent use of the memory units (by sending requests to those units before other units have finished requests). Further, by using a reset point, the media access request executor can return to process skipped requests to preserve data coherency.
The features and attributes of the specific embodiments disclosed above may be combined in different ways to form additional embodiments, all of which fall within the scope of the present disclosure. Although the present disclosure provides certain preferred embodiments and applications, other embodiments that are apparent to those of ordinary skill in the art, including embodiments which do not provide all of the features and advantages set forth herein, are also within the scope of this disclosure. Accordingly, the scope of the present disclosure is intended to be defined only by reference to the appended claims.
This application is a divisional application of U.S. patent application Ser. No. 13/166,985, filed Jun. 23, 2011, entitled “System and Method for Multistage Processing in Memory Storage Subsystem, the specification of which is incorporated herein by reference. U.S. patent application Ser. No. 13/166,985 claims priority from co-pending provisional U.S. Patent Application Ser. No. 61/385,132 (Atty. Docket No. T4213.P), filed on Sep. 21, 2010, the specification of which is incorporated herein by reference.
Number | Date | Country | |
---|---|---|---|
61385132 | Sep 2010 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 13166985 | Jun 2011 | US |
Child | 14885018 | US |