Computing systems generally utilize non-volatile storage media, such as hard disk drives, to store information, and to ensure the availability of stored information across power cycles. In some instances, computing systems or applications executed by computing systems require that data be committed to a disk drive in a specific order, such that subsequent read requests to the disk drive result in retrieval of the expected data.
However, the systems and protocols used to interact with storage devices such as disk drives may be unable to ensure such a specific order of writes. For example, multithreaded systems may allow multiple applications or processes to access a disk drive simultaneously or near-simultaneously, and in doing so, may reorder various operations performed by the applications or processes. Further, current disk drives or other non-volatile storage media can utilize volatile cache memory in an attempt to increase speed and efficiency of the drive. Use of cache memory, and protocols associated therewith, may result in reordering of operations prior to commitment to a non-volatile memory. Reordering of operations to the non-volatile memory may result in unexpected values being written to or read from the non-volatile memory by the computing system, and therefore result in errors.
To compensate for the out-of-order environment in which writes can be committed to the non-volatile memory, an application has the option to ensure that all of its previous writes have completed before completion of the next write by performing an application file operation flush, which causes all data of all previously completed writes to be forced out of the system and committed to the storage device's permanent media. However, implementation of a flush command on the storage device is generally time consuming, and can degrade performance. Consequently, many storage subsystems (or software/firmware/hardware associated therewith) decline to perform flush commands. Ignoring a program's (e.g., application's or operating system's) flush commands may increase performance relative to performing the commands, but results in potential data inconsistency.
An alternative technique for ensuring data consistency is to serialize all ordered writes throughout the host computing device and mark each one as a Forced Unit Access (FUA). Use of FUA writes ensures that each write is committed to non-volatile media before the command is completed. However, due to the delay in completion, FUA commands tend to be significantly slower than a typical write command. As a result, many storage subsystems (or software/firmware/hardware thereof) fail to respect a program's request to perform FUA writes.
Systems and methods that embody the various features of the invention will now be described with reference to the following drawings, in which:
While certain embodiments are described, these embodiments are presented by way of example only, and are not intended to limit the scope of protection. Indeed, the novel methods and systems described herein may be embodied in a variety of other forms. Furthermore, various omissions, substitutions, and changes in the form of the methods and systems described herein may be made without departing from the scope of protection.
Overview
Generally, some embodiments of systems and methods described herein address the performance issues related to the use of flush commands and FUA writes by providing a storage device enabled to notify an application (or other element of a computing system) when a specified set of criteria have been satisfied. For example, a storage device such as a disk drive or solid state drive may be configured to receive and process notification requests from a device driver or other component of a computing system. Each notification request may specify one or more previous commands completed by the storage device, and request that the disk drive notify the device driver when the data associated with the command has been committed to non-volatile memory. In order to facilitate such notifications, the device driver may label or tag each command (or data item associated with a command) issued to the storage device with an identifier. In general, a data item is any data referenced by a command, which may include, for example, data to be written at one or more logical block addresses (LBAs) of a storage device, data objects referenced by software programs, data records forming portions of databases, or any other collection or organization of data to be stored within a storage device. This identifier may be unique, or may be shared among multiple commands or data items. Each notification command may specify a collection of identifiers for previous write commands (or data items associated therewith) that, in some embodiments, must be completed prior to transmission of a notification to the storage device. By managing notifications related to data items, a device driver or other element of a computing system can ensure data items are committed to non-volatile memory of a storage device according to the desired order of an application.
In one embodiment, an application's, operating system or other program's desired write order may be determined based on the use of flush request or FUA write requests. For example, a device driver of a storage device may interpret each flush request as a requirement that all previously completed write commands be committed prior to completion of the flush request. In order to distinguish the timing of requested writes, the device driver may utilize tags or other identifiers previously associated with each write (or set of writes) request(s). On receiving a flush request, the device driver may issue a notification command, that a notification be sent by the storage device when all previously tagged completed write commands have been committed. Subsequent to receiving the notification by the storage device, the device driver can report completion of the flush request to the application, operating system or other program. As will be described in more detail below, utilization of notifications can enable successful processing of an application's flush request without requiring a storage device to commit all previously cached data and without monopolizing communications with the storage device. Therefore, use of notifications can ensure data consistency of all commands completed prior to the flush request without incurring the performance penalties associated with performing flush commands or FUA writes.
In another embodiment, a device driver may interpret each flush request as a requirement that all previously transmitted data related to a specific file or program be committed to non-volatile memory prior to completion of the flush request. Illustratively, an application issuing a flush request may solely be interested in committing writes to one or more files associated with that application. Accordingly, a device driver may utilize command identifiers to distinguish between commands requested by various applications or modifying various files. On receiving a flush request, the device driver may issue a notification command, that a notification be sent by the storage device when all data associated with the specific application or specific file have been committed to non-volatile memory (e.g., destaged from a cache of the storage device). Subsequent to receiving the notification by the storage device, the device driver can report completion of the flush request to the application. Advantageously, notifications related to specific applications or files may be received more quickly than non-application or non-file specific notifications (e.g., due to the lower number of completed commands required for the notification), thereby increasing the speed and efficacy of the application's flush request.
In yet another embodiment, an operating system or other component of a computer system can utilize ordering commands to enforce the ordering of writes on a storage device (e.g., to implement efficient write barriers). Write barriers generally ensure that commands executed prior to the write barrier are committed to non-volatile memory prior to commands executed after implementation of the write barrier. Accordingly, an operating system may receive flush requests from an application, and interpret each flush request as creation of a write barrier. The operating system may immediately notify the application of completion of the flush request, and thereafter, as will be described below, ensure that any previous requested writes are committed prior to subsequent requested writes. In one instance, implementation of write barriers may utilize storage device notification commands, such that an operating system or device driver may queue requests received after establishing a write barrier until after receiving a notification that all previous completed write commands have been committed on a storage device. In another instance, a storage device may be modified to support implementation of ordering commands, enabling specification by a device driver of an order that specified data items should be committed to non-volatile memory.
While aspects of the present application may be described with reference to write commands, embodiments of this disclosure may be utilized with respect to any request to store data to non-volatile memory of a storage device. For example, embodiments of the present application may be utilized in connection with “put” commands, where a storage device is in communication with a host system over a communication network. Accordingly, notification commands may be used to determine whether data items corresponding to a “put” command have been committed to non-volatile memory of, for example, a network attached storage (NAS) system.
The above will be more fully appreciated and understood with reference to the following description.
System Overview
The applications 112 and operating systems 114 may provide various functionalities to users of the host system 110 which require or are assisted by storage of data within the storage system 120. Accordingly, the host system may include one or more device drivers 116 configured to enable transmission of data storage commands (e.g., read or write commands) to the storage system 120. Such commands may be transmitted via a communication link between the host system 110 and storage system 120. In one embodiment, the communication link may include a storage bus, such as a serial ATA (SATA) storage bus or other storage bus utilizing the advanced host controller interface (AHCI) standard. In another embodiment, the communication link may include a system bus, such as a Peripheral Component Interconnect (PCI) or PCI Express bus. In still other embodiments, the communication link may include network communication links (e.g., Ethernet links enabling communication via transmission control protocol (TCP)/internet protocol (IP) standards).
The data storage system 120, which may correspond, for example, to a hard disk drive (HDD), solid state hybrid drive (SSHD), or solid state drive (SSD), includes a controller 122, a non-volatile memory 124, and a volatile cache memory 126. In one embodiment, the non-volatile memory 124 includes magnetic storage media (e.g., a magnetic disk). In another embodiment, the non-volatile memory 124 includes a solid state memory implemented, for example, by flash integrated circuits, Chalcogenide RAM (C-RAM), Phase Change Random Access Memory (PC-RAM or PRAM), Programmable Metallization Cell RAM (PMC-RAM or PMCm), Ovonic Unified Memory (OUM), Resistance RAM (RRAM), NAND memory (e.g., single-level cell (SLC) memory, multi-level cell (MLC) memory, or any combination thereof), NOR memory, electrically erasable programmable read only memory (EEPROM), Ferroelectric Random Access Memory (FeRAM), Magnetoresistive RAM (MRAM), other discrete NVM (non-volatile memory) chips, or any combination thereof. The non-volatile memory of
The volatile cache memory 126 of
The controller 122 is configured to receive data and/or storage access commands from device drivers 116 of a host system 110. Storage access commands communicated by a storage interface coupled with the controller 122 can include write data and read data commands issued by the host system 110. The controller 122 can execute the received commands with respect to the non-volatile memory 124. In some instances, the controller 122 can utilize the volatile cache memory 126 to temporarily store data written by the host system 110, prior to committing such data to the non-volatile memory 124. Storage of data in the volatile cache memory 126 may generally be referred to as “caching” such data. Thereafter, the controller can selectively transfer data from the volatile cache memory 126 to the non-volatile memory 124. Transfer of data from the volatile cache memory 126 to the non-volatile memory 124 can generally be referred to as “destaging” the data. In some instances, the controller 122 may implement “native command queue (NCQ)” functionality. Generally described, NCQ functionality enables the controller 122 to prioritize the order at which data is destaged from the volatile cache memory 126 in order to optimize the performance of the non-volatile memory 124. Because NCQ functionality is well known within the art, it will not be further described herein.
In accordance with some embodiments of the present disclosure, the controller 122 may be configured to receive and process notification commands received from the host system 110. Each notification command may identify one or more items of data which the host system 110 previously requested be written to the non-volatile memory 124. As will be described in more detail below, each item of data transmitted from the host system 110 to the storage system 120 may include an identifier associated with the data item by, e.g., the device driver 116. Accordingly, a notification command may identify data items by use of associated identifiers.
The controller 122 may be configured to utilize the identifiers within the notification command as criteria for generating a corresponding notification to the host system that each identified data item has been committed to the non-volatile memory 124. For example, the controller 122 may receive a notification command identifying three previous data items requested to be written to the non-volatile memory 124, e.g., data items ‘A’, ‘B’, and ‘C’. Each data item may initially reside within the volatile cache memory 126, awaiting commitment to the non-volatile memory 124. Thereafter, the controller may monitor for the commitment of the identified data items (e.g., data items ‘A’, ‘B’, and ‘C’) to the non-volatile memory 124. On destaging of the identified data items, and commitment of such data items to the non-volatile memory 124, the controller 122 can generate a notification to the host system 110, indicating that the identified data items (e.g., data items ‘A’, ‘B’, and ‘C’) have been committed to the non-volatile memory 124.
The operating system 114, device driver 116, or other modules within the host system 110 can utilize the notification functionality of the storage system 120 in order to process flush requests of software programs (e.g., applications or operating systems), without requiring a corresponding flush command to the device. In some examples below, for the sake of brevity, some of the flush requests will be described as being issued by applications. It is to be understood that such flush requests apply generally to any such write requests issued by any software program. As will be appreciated by one skilled in the art, implementing a flush command on a storage system (e.g., the storage system 120) is generally associated with significant performance degradation. Specifically, implementation of such a flush command may require the storage system 120 to completely quiesce the volatile cache memory 126, without regard to the specific data items included within the volatile cache memory 126. Moreover, implementation of such a flush command may monopolize communications along the communication link, such that additional commands cannot be transmitted to the storage system 120 prior to completion of the flush command.
Utilization of notification commands, in accordance with aspects of this disclosure, may enable many disadvantages of traditional flush commands on storage systems to be overcome. Moreover, in some instances, notification commands may be implemented without requiring modification of applications 112 or operating systems 114. Specifically, an application may generally utilize a flush request to ensure that all previous data written by the application has been committed to the non-volatile memory 124. Accordingly, a device driver 116 may be configured to associate data written by the application with an identifier. On receiving a flush request by the application, the device driver 116 can utilize the identifier to identify each previous data item written. The device driver 116 can further generate a notification command including the identifier, and transmit the notification command to the storage system 120. Once all identified data has been committed to the non-volatile memory 124, the controller 122 can generate a corresponding notification to the device driver 116. The device driver 116 may then notify the application that the application's flush request has completely successfully. Thus, an application's flush request may be successfully processed without requiring use of a corresponding flush command on the storage system 120. Various additional embodiments will be described in more detail below.
Identification of Data Items
At (2), the operating system processes the received write request, and transmits the request to the device driver 116A. Though not shown within
At (3), after receiving the write request, the device driver 116A generates the write command with an identifier, which may subsequently be used in generating notifications regarding the write command or the corresponding data item. In one embodiment, the device driver 116A modifies the write command itself. For example, the device driver 116A may modify one or more bits or fields of the corresponding SATA command. In another embodiment, the device driver 116A may augment the data item to be written to the non-volatile memory 124 (e.g., by prepending or appending an identifier to the data item). In one embodiment, a set of potential identifiers may be selected according to the potential performance impact of the identifier on communication with the controller 122. For example, where an identifier is embedded within a SATA command, a set of potential identifiers may be selected such that all possible identifiers may be represented without expanding the total data size of the SATA command. Illustratively, a five bit identifier may be used, resulting in 32 possible identifiers. The use of larger identifiers may allow for more write commands to be uniquely identified, but may result in a more substantial impact on performance. In another embodiment, pre-existing identifiers for data items, such as record locators or object keys, may be utilized by the device driver in generating notification commands in accordance with some embodiments of the present disclosure. In some such embodiments, the pre-existing nature of a data item identifier may allow interaction (3) to be omitted. Selection of identifiers will be described in more detail below with respect to
Thereafter, at (4), the write command and corresponding data item, as modified to include an identifier, are transmitted to the controller 122 via the communication link. In one embodiment, the write command is transmitted via SATA or other storage protocol over a storage bus. In another embodiment, the write command is transmitted via a system bus, such as a PCI bus (e.g., by adherence to the non-volatile memory express (NVMe) specification). In still more embodiments, the write command may be transmitted via a network communication link (e.g., via TCP/IP).
At (5), on receiving the write command and corresponding data item, the controller 122 caches the data item into the volatile cache memory 126 (not shown in
At (6), the data item previously cached to the volatile cache memory 126 is destaged by the controller 122 including committing the data item to the non-volatile memory 124 at (6a). While interaction (6) (including interaction (6a)) is depicted in
Processing of Flush Requests
With reference to
At (3), the device driver 116A processes the flush request by generating a specific notification command corresponding to the flush request. In one embodiment, the device driver 116A can determine that the flush request may be satisfied when data items of all previous write commands transmitted to the controller 122 have been committed to the non-volatile memory 124. Accordingly, the device driver 116A may determine a set of identifiers corresponding to outstanding write commands (e.g., write commands not yet known by the device driver 116A to have been committed to the non-volatile memory 124), and generate a notification command for such identifiers. In one instance, the set of identifiers corresponding to outstanding write commands can include all identifiers assigned to write commands since a previous flush request was received. In another instance, the set of identifiers corresponding to outstanding write commands can include any identifier for which a notification of commitment to the non-volatile memory 124 has not yet been received at the device driver 116A.
As an illustrative example, the device driver 116A may have previously received three write commands, and subsequently transmitted the three write commands to the controller 122 and received the notification of the write commands completion. Prior to transmission, the device driver 116A may have associated the write commands (and corresponding data items) with three identifiers (e.g., ‘A’, ‘B’, and ‘C’). On receiving a flush request, the device driver 116 may be unaware of whether the three data items of the three commands have been committed to the non-volatile memory 124. Accordingly, the device driver may generate a notification command corresponding to the three identifiers, ‘A’, ‘B’, and ‘C’. As will be described below, the controller 122 may then process the notification command, and once it has determined that the three identifiers have been committed to memory, it may notify the device driver 116A.
In some embodiments, notifications may be targeted, such that only a subset of data items corresponding to outstanding write commands must be committed to non-volatile memory 124 in order to satisfy the notification. In one instance, targeting may be based on a specific file in use by the application 112A. As described above, a flush request can generally correspond to a request by the application 112A that all data previously transmitted to the storage system 120 be committed to non-volatile memory 124. However, in many instances, the application 112A may be unaware of many of the data items previously transmitted to the storage system 120 (e.g., because such data items were transmitted by alternative applications). Accordingly, the application 112A's flush request may be satisfied by ensuring that all previous data items associated with the application 112A have been committed to the non-volatile memory 124 (as opposed to all previous data items, regardless of source). Accordingly, the notification generated by the device driver 116A may be targeted specifically to previous data items associated with the application 112A, or with previous data items associated with a file used by application 112A.
As an illustrative example, assume again that three previous data items (e.g., data items ‘A’, ‘B’, and ‘C’) are not yet known to the device driver 116A as having been committed to non-volatile memory 124. In such an instance, only a single data item, such as data item ‘A’, may be associated with the application 112A or a file written by the application 112A. Therefore, the device driver may assume that the applications 112A's flush request will be successfully processed after commitment of the data item ‘A’ to the non-volatile memory 124, without regard to the commitment of data items ‘B’ and ‘C’. The device driver 116A can therefore create a notification specifically targeted to data item ‘A’. Due to the limited criteria associated with the targeted notification, the criteria of targeted notifications may be satisfied more quickly than general notifications. As described in more detail below, this may result in quicker completion of the application 112A's flush request.
With continued reference to
While not shown in
After determining that the criteria of the previously received notification command are satisfied, the controller 122, at (6), transmits a notification of criteria satisfaction to the device driver 116A. The notification may be transmitted according to any communication channel between the controller 122 and the device driver 116A. For example, where the controller 122 and the device driver 116A communicate via a storage bus, the notification may be transmitted via a storage protocol, such as SATA. Where the controller 122 and the device driver 116A communicate via a communication network, network packets may be used to transmit a notification. Still further, where the controller 122 and the device driver 116A communicate via a system bus, alternative notifications may be utilized. For example, in one embodiment, the controller 122 may utilize an interrupt signal to transmit a notification to the device driver 116A. In instances where interrupt signals are utilized, specific information regarding the notification may be placed in a memory of the host system 110 or storage system 120 for retrieval by the device driver 116A. For example, the controller 122 may maintain a portion of memory in which a set number of bits are allocated for each potential identifier assigned to data items. Thereafter, the controller 122 may update the portion of memory to indicate whether each identifier is associated with cached data items. Illustratively, where each potential identifier is assigned a single bit within memory, a value of one may indicate that the identifier is associated with a cached data item on the storage system 120, while an indicator of zero may indicate that the identifier is not associated with a cached data item on the storage system 120. In such an instance, the controller 122 may utilize processor interrupts to indicate that the device driver 116A should inspect the assigned memory for information regarding the status of data items.
Based on the transmitted notification, the device driver is made aware that the previously identified data items have been committed to the non-volatile memory 124 of the storage system 120. At (7), the device driver 116A notifies the operating system 114A that the flush request has been successfully processed. Similarly, at (8), the operating system 114A notifies the application 112A that the flush request has been successfully processed. Accordingly, the interactions of
With reference to
Specifically, at (1), the application 112A transmits a flush request to the operating system 114A. As noted above, a flush request generally corresponds to a request by the application that all prior data items be committed to non-volatile memory 124 of the storage system 120 prior to processing subsequent requests to the storage system 120. Because the controller 122 is configured to implement ordering commands, the operating system 114A can ensure that this request is complied with by the controller 122 (as discussed in more detail below). The operating system 114A may therefore immediately (or substantially immediately) notify the application 112A of completion of the flush request.
Thereafter, at (3), the operating system 114A transmits a request to the device driver 116A to implement a write barrier. Generally described, the request may correspond to a request that all data items previously transmitted to the device driver 116A for writing be committed to non-volatile memory 124 prior to commitment of subsequent data items. In one embodiment, the operating system 114A may include metadata with the request. As described above, metadata may include, e.g., data related to the application 112A or to the file written by the application 112A.
After reception of the write barrier request, at (4), the device driver 116A generates an ordering command implementing the write barrier on the controller 122. In one embodiment, the ordering command identifies a set of previous write commands, or data items associated with previous write commands, as required to be committed to the non-volatile memory 124 prior to committing data items of any subsequent write commands. For example, the ordering command may include, as ordering criteria, identifiers of a set of commands not known to the device driver 116A to be committed to the non-volatile memory 124 (e.g., identifiers utilized since the last ordering command was transmitted). In another embodiment, an ordering command may identify only a select set of commands not known to the device driver 116A to be committed to the non-volatile memory 124. Illustratively, the device driver 116A may utilize metadata provided by the operating system 114 to generate an ordering command targeting previous write requests associated with the application 112. Accordingly, if three identifiers are associated with outstanding data items, but only a single identifier is associated with the application 112A, the device driver 116A may generate an ordering command requiring that data items associated with that single identifier be committed to non-volatile memory 124 prior to processing subsequent commands.
Thereafter, at (5), the device driver 116A transmits the generated ordering command, including ordering criteria (e.g., identifiers associated with previous data items), to the controller 122. As discussed above, the controller 122 is configured thereafter, at (6), to destage cached data items in accordance with the ordering command. Illustratively, the controller 122 may delay processing of subsequent commands received from the device driver 116A until each data item identified within the ordering command has been destaged and committed to non-volatile memory 124. In this manner, any subsequent write requests from the application 112A can be ensured to be committed after previous write requests, therefore satisfying the flush request. Advantageously, use of ordering commands to implement write barriers can enable nearly instantaneous completion of flush requests, without requiring a notification to be generated and transmitted from the controller 122. In some instances, however, ordering commands may require modification to additional elements of the host system 110, such as the operating system 114A.
Effects of Flush Commands on Work Queues
With reference to
With reference to
With reference to
With reference to
Routines for Implementing Write Notifications
With reference to
In the instance that the device driver determines, at block 504, that the current write command is not associated with a previous identifier or previously received (e.g., outstanding) write request, the routine continues at block 508, where the device driver determines whether any identifiers are available to be associated with the write command. As discussed above, a set of identifiers for use by the device driver may be selected according to any number of criteria. In one embodiment, the set of identifiers may be selected such that an identifier may be included within a write command (or a data item thereof) without substantially increasing the transmission time of the write command or the data item. For example, identifiers may be selected such that they utilize previously reserved bits within a write command protocol. In another embodiment, the device driver may utilize characteristics of the data item when selecting an identifier. For example, where a component of the host system, such as the operating system, has previously associated the data item with a unique identifier, the device driver may continue to utilize such an identifier for the purposes of generating notification commands. Illustratively, pre-existing identifiers may include universally unique identifiers (UUIDs) utilized by a host system or storage device, such as record locators or object keys.
In some instances, the device driver may utilize a new identifier for each flush request received from a program. For example, the device driver may, on reception of a flush request, increment a currently used identifier, and utilize the incremented identifier for subsequent write commands. Because each flush request may be associated with a corresponding notification, the device driver may, subsequent to use, release an identifier as available after receiving a notification corresponding to the identifier. In this manner, the device driver may implement a rotating identifier pool for each write command. Advantageously, use of a rotating identifier pool based on received flush requests may require little or no metadata to be received at the device driver.
In another embodiment, the device driver may utilize a new identifier for each set of associated write commands (e.g., each file written to, each application performing a write, etc.). For example, where two applications are simultaneously writing to a storage system, the device driver may utilize different identifiers for each application. The device driver may increment the identifier of an individual application when that application transmits a flush request. In this manner, flush requests of an individual application are not required to affect the identifiers associated with the write commands of other applications. Similarly to as described above, the device driver may designate any identifier associated with an outstanding notification command as unavailable, and subsequent to receiving the notification command, designate the identifier as available.
In the instance that the device driver determines, at block 508, that no identifier is currently available (e.g., when all current identifiers are associated with outstanding notification commands, other files, or other programs), the routine continues at block 510, where the device driver awaits a notification corresponding to a currently unavailable identifier. As noted above, on receiving a notification that data items associated with a previously-unavailable identifier have been committed to non-volatile memory of the storage system, the device driver can designate the identifier as available. The routine 500 may then return to block 508 for recognition of the newly-available identifier.
In the instance that the device driver determines, at block 508, that an identifier is available for association with a write command (or data items thereof), the routine 500 continues at block 512, where the write command is modified with the available identifier. In one instance, the write command itself may be associated with the identifier (e.g., by inserting a bit pattern of the identifier within the write command). In another instance, the data items associated with the write command may be modified to include the identifier (e.g., by prepending, appending, or otherwise encoding the data item with the identifier).
Thereafter, the routine continues to block 514, where the device driver transmits the modified write command to the storage system (e.g., to a controller) for subsequent processing. Illustratively, the storage system may cache the data item associated with the write command for subsequent commitment to non-volatile memory. The storage system may also maintain a record of the identifier associated with the write command (and data items thereof), in order to process corresponding notification commands.
Optionally, at block 516, the device driver may attempt to reclaim previously used identifiers. Reclamation may be beneficial, for example, if the pool of identifiers available to the device driver has fallen below a predetermined threshold. In one embodiment, the device driver may attempt to reclaim identifiers by generating notification commands corresponding to the identifiers, and transmitting such notification commands to the storage system. Subsequent to receiving a notification that all data items associated with an identifier have been committed to non-volatile memory of the storage system, the device driver can then designate the identifier as available. Routines to generate and process notification commands will be described in more detail below. In another embodiment, reclamation of previously used identifiers may be accomplished at least in part based on aggregation of previously used identifiers. Illustratively, the device driver may be programmed to aggregate “stale” identifiers to a single identifier, to increase the overall number of available identifiers. Staleness of an identifier may be determined, for example, based on the number of flush requests received or amount of time passed since the identifier was assigned to a write command. On determining that multiple identifiers should be aggregated, the device driver can send an aggregation command to the storage system (e.g., to a controller) specifying each source identifier (e.g., identifiers to be aggregated) and a resultant identifier (the identifier to which each source identifier should be aggregated). In response, the controller 122 can identify any outstanding data items with a source identifier, and alter each data item to be associated with the resultant identifier. In some instances, aggregation may trigger a notification to the device driver that a source identifier is no longer associated with any cached data items awaiting commitment to non-volatile memory. The routine may thereafter end at block 518.
With reference to
At block 554, the device driver generates a notification command based on the received flush request. As described above, the notification command includes criteria related to previously used identifiers. In one embodiment, the criteria of the notification command may be targeted based on metadata of the flush request. For example, the notification command can be associated with previously used identifiers of a specific program or specific data file. Illustratively, if the flush request is received from a specific application, the notification command can be associated with identifiers utilized for write commands of the specific application (e.g., since the previous flush request of the specific application). In this manner, notification commands can be specifically targeted to data items associated with a specific application or file. While notification commands are depicted in
At block 556, the generated notification command is transmitted to the storage system (e.g., to a controller within the storage system). As will be described in more detail below with respect to
With reference to
At block 606, the storage system determines whether the criteria of the notification command have been satisfied. In one embodiment, the storage system may inspect cache memory to determine whether any data items associated with identifiers of the notification command are awaiting destaging. In the instance that no data item associated with an identifier of the notification command exists within the cache memory, the storage system can determine that the notification criteria have been satisfied, and the routine 600 proceeds to block 610, described below.
Alternatively, if the notification criteria are not yet satisfied at block 606, the routine 600 proceeds to block 608, where the storage system continues to process data items by destaging data items from the cache memory and committing data items to non-volatile memory. During implementation of block 608, the storage system may continue to communicate with the host system to receive write commands, read commands, notification commands, or otherwise provide available functionality. The routine 600 then returns to block 606, to determine whether the criteria of the notification command have been satisfied, as described above. This routine 600 continues to loop in this manner until the criteria of the notification command have been satisfied (e.g., until data items in the cache have been destaged such that no data item is associated with an identifier of the notification command).
Thereafter, the routine 600 continues to block 610, where the storage system generates a notification indicating that the criteria of the notification command have been satisfied. At block 612, the notification is transmitted to the host system via any available communication channel (e.g., via a storage bus, system bus, or communication network). As noted above, such a notification may be utilized by the host system to indicate that a flush request has been successfully processed.
While described above according to illustrative embodiments, the routines of
The disclosed systems and methods for utilization of notification or ordering commands can enable more efficient processing of flush requests from software programs. Further, the disclosed systems and methods may increase data consistency in storage devices, especially in instances where flush requests would otherwise be ignored by storage devices (due to their typical inefficiency). Specifically, the utilization of notification or ordering commands can enable applications to ensure that previous data items have been committed to non-volatile memory prior to completion. However, in contrast to existing flush mechanisms, the notification and ordering commands disclosed herein can enable data consistency without requiring quiescing of a work queue or monopolization of a communication bus. Moreover, the targeted notifications disclosed herein can limit the scope of a flush request, such that the speed and efficiency of the request are increased relative to a non-specific full-cache flush. Still further, the ordering commands disclosed herein can enable near immediate success of an application's flush requests, without reducing the consistency of the data. Such increases in efficiency of flush requests may greatly increase the speed and efficiency of storage devices, and consequently the performance of programs utilizing such storage devices.
Other Variations
Those skilled in the art will appreciate that in some embodiments, additional or alternative modules can be used to perform compression and formatting. Mapping units, compressed mapping units, and/or virtual storage units can include additional or alternative fields. The actual steps taken in the disclosed processes, such as the routines illustrated in
While certain embodiments have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the protection. Indeed, the novel methods and systems described herein may be embodied in a variety of other forms. Furthermore, various omissions, substitutions and changes in the form of the methods and systems described herein may be made without departing from the spirit of the protection. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the protection. For example, the systems and methods disclosed herein can be applied to hard disk drives, hybrid hard drives, and the like. In addition, other forms of storage (e.g., DRAM (“Dynamic Random Access Memory”) or SRAM (“Static Random Access Memory”), battery backed-up volatile DRAM or SRAM devices, EPROM (“Erasable Programmable Read-Only Memory”), EEPROM (“Electrical EPROM”), etc.) may additionally or alternatively be used. As another example, the various components illustrated in the figures may be implemented as software and/or firmware on a processor, ASIC/FPGA, or dedicated hardware. Hardware components, such as processors, ASICs, FPGAs, and the like, can have logic circuitry. Also, the features and attributes of the specific embodiments disclosed above may be combined in different ways to form additional embodiments, all of which fall within the scope of the present disclosure. Although the present disclosure provides certain preferred embodiments and applications, other embodiments that will be apparent to those of ordinary skill in the art, including embodiments which do not provide all of the features and advantages set forth herein, are also within the scope of this disclosure. Accordingly, the scope of the present disclosure is intended to be defined only by reference to the appended claims.
This application claims benefit under 35 U.S.C. §119(e) to U.S. Provisional Patent Application No. 61/976,023 titled “MANAGING CONSISTENCY IN STORAGE MEDIA” and filed on Apr. 7, 2014, the disclosure of which is hereby incorporated by reference in its entirety. This application is related to co-pending application Ser. No. 14/322,520 filed on Jul. 2, 2014, entitled “PROCESSING FLUSH REQUESTS BY UTILIZING STORAGE SYSTEM WRITE NOTIFICATIONS”, the disclosure of which is also hereby incorporated by reference in its entirety.
Number | Name | Date | Kind |
---|---|---|---|
4703420 | Irwin | Oct 1987 | A |
5077733 | Whipple | Dec 1991 | A |
5276662 | Shaver, Jr. et al. | Jan 1994 | A |
5465338 | Clay | Nov 1995 | A |
5590299 | Bennett | Dec 1996 | A |
5809522 | Novak | Sep 1998 | A |
5860089 | Ishii | Jan 1999 | A |
5893149 | Hagersten | Apr 1999 | A |
6018789 | Sokolov et al. | Jan 2000 | A |
6065095 | Sokolov et al. | May 2000 | A |
6078452 | Kittilson et al. | Jun 2000 | A |
6081447 | Lofgren et al. | Jun 2000 | A |
6092149 | Hicken et al. | Jul 2000 | A |
6092150 | Sokolov et al. | Jul 2000 | A |
6094707 | Sokolov et al. | Jul 2000 | A |
6105104 | Guttmann et al. | Aug 2000 | A |
6111717 | Cloke et al. | Aug 2000 | A |
6145052 | Howe et al. | Nov 2000 | A |
6175893 | D'Souza et al. | Jan 2001 | B1 |
6178056 | Cloke et al. | Jan 2001 | B1 |
6191909 | Cloke et al. | Feb 2001 | B1 |
6195218 | Guttmann et al. | Feb 2001 | B1 |
6205494 | Williams | Mar 2001 | B1 |
6208477 | Cloke et al. | Mar 2001 | B1 |
6223303 | Billings et al. | Apr 2001 | B1 |
6230233 | Lofgren et al. | May 2001 | B1 |
6246346 | Cloke et al. | Jun 2001 | B1 |
6249393 | Billings et al. | Jun 2001 | B1 |
6256695 | Williams | Jul 2001 | B1 |
6262857 | Hull et al. | Jul 2001 | B1 |
6263459 | Schibilla | Jul 2001 | B1 |
6272694 | Weaver et al. | Aug 2001 | B1 |
6278568 | Cloke et al. | Aug 2001 | B1 |
6279089 | Schibilla et al. | Aug 2001 | B1 |
6289484 | Rothberg et al. | Sep 2001 | B1 |
6292912 | Cloke et al. | Sep 2001 | B1 |
6310740 | Dunbar et al. | Oct 2001 | B1 |
6317850 | Rothberg | Nov 2001 | B1 |
6327106 | Rothberg | Dec 2001 | B1 |
6337778 | Gagne | Jan 2002 | B1 |
6369969 | Christiansen et al. | Apr 2002 | B1 |
6384999 | Schibilla | May 2002 | B1 |
6388833 | Golowka et al. | May 2002 | B1 |
6405342 | Lee | Jun 2002 | B1 |
6408357 | Hanmann et al. | Jun 2002 | B1 |
6408406 | Parris | Jun 2002 | B1 |
6411452 | Cloke | Jun 2002 | B1 |
6411458 | Billings et al. | Jun 2002 | B1 |
6412083 | Rothberg et al. | Jun 2002 | B1 |
6415349 | Hull et al. | Jul 2002 | B1 |
6425128 | Krapf et al. | Jul 2002 | B1 |
6441981 | Cloke et al. | Aug 2002 | B1 |
6442328 | Elliott et al. | Aug 2002 | B1 |
6445524 | Nazarian et al. | Sep 2002 | B1 |
6449767 | Krapf et al. | Sep 2002 | B1 |
6453115 | Boyle | Sep 2002 | B1 |
6470420 | Hospodor | Oct 2002 | B1 |
6480020 | Jung et al. | Nov 2002 | B1 |
6480349 | Kim et al. | Nov 2002 | B1 |
6480932 | Vallis et al. | Nov 2002 | B1 |
6483986 | Krapf | Nov 2002 | B1 |
6487032 | Cloke et al. | Nov 2002 | B1 |
6490635 | Holmes | Dec 2002 | B1 |
6493173 | Kim et al. | Dec 2002 | B1 |
6499083 | Hamlin | Dec 2002 | B1 |
6519104 | Cloke et al. | Feb 2003 | B1 |
6525892 | Dunbar et al. | Feb 2003 | B1 |
6545830 | Briggs et al. | Apr 2003 | B1 |
6546489 | Frank, Jr. et al. | Apr 2003 | B1 |
6550021 | Dalphy et al. | Apr 2003 | B1 |
6552880 | Dunbar et al. | Apr 2003 | B1 |
6553457 | Wilkins et al. | Apr 2003 | B1 |
6578106 | Price | Jun 2003 | B1 |
6580573 | Hull et al. | Jun 2003 | B1 |
6594183 | Lofgren et al. | Jul 2003 | B1 |
6600620 | Krounbi et al. | Jul 2003 | B1 |
6601137 | Castro et al. | Jul 2003 | B1 |
6603622 | Christiansen et al. | Aug 2003 | B1 |
6603625 | Hospodor et al. | Aug 2003 | B1 |
6604220 | Lee | Aug 2003 | B1 |
6606682 | Dang et al. | Aug 2003 | B1 |
6606714 | Thelin | Aug 2003 | B1 |
6606717 | Yu et al. | Aug 2003 | B1 |
6611393 | Nguyen et al. | Aug 2003 | B1 |
6615312 | Hamlin et al. | Sep 2003 | B1 |
6618788 | Jacobs | Sep 2003 | B1 |
6639748 | Christiansen et al. | Oct 2003 | B1 |
6647481 | Luu et al. | Nov 2003 | B1 |
6654193 | Thelin | Nov 2003 | B1 |
6657810 | Kupferman | Dec 2003 | B1 |
6661591 | Rothberg | Dec 2003 | B1 |
6665772 | Hamlin | Dec 2003 | B1 |
6687073 | Kupferman | Feb 2004 | B1 |
6687078 | Kim | Feb 2004 | B1 |
6687850 | Rothberg | Feb 2004 | B1 |
6690523 | Nguyen et al. | Feb 2004 | B1 |
6690882 | Hanmann et al. | Feb 2004 | B1 |
6691198 | Hamlin | Feb 2004 | B1 |
6691213 | Luu et al. | Feb 2004 | B1 |
6691255 | Rothberg et al. | Feb 2004 | B1 |
6693760 | Krounbi et al. | Feb 2004 | B1 |
6694477 | Lee | Feb 2004 | B1 |
6697885 | Goodfellow | Feb 2004 | B1 |
6697914 | Hospodor et al. | Feb 2004 | B1 |
6704153 | Rothberg et al. | Mar 2004 | B1 |
6708251 | Boyle et al. | Mar 2004 | B1 |
6710951 | Cloke | Mar 2004 | B1 |
6711628 | Thelin | Mar 2004 | B1 |
6711635 | Wang | Mar 2004 | B1 |
6711660 | Milne et al. | Mar 2004 | B1 |
6715044 | Lofgren et al. | Mar 2004 | B2 |
6724982 | Hamlin | Apr 2004 | B1 |
6725329 | Ng et al. | Apr 2004 | B1 |
6735650 | Rothberg | May 2004 | B1 |
6735693 | Hamlin | May 2004 | B1 |
6744772 | Eneboe et al. | Jun 2004 | B1 |
6745283 | Dang | Jun 2004 | B1 |
6751402 | Elliott et al. | Jun 2004 | B1 |
6757481 | Nazarian et al. | Jun 2004 | B1 |
6772281 | Hamlin | Aug 2004 | B2 |
6781826 | Goldstone et al. | Aug 2004 | B1 |
6782449 | Codilian et al. | Aug 2004 | B1 |
6791779 | Singh et al. | Sep 2004 | B1 |
6792486 | Hanan et al. | Sep 2004 | B1 |
6799274 | Hamlin | Sep 2004 | B1 |
6811427 | Garrett et al. | Nov 2004 | B2 |
6826003 | Subrahmanyam | Nov 2004 | B1 |
6826614 | Hanmann et al. | Nov 2004 | B1 |
6832041 | Boyle | Dec 2004 | B1 |
6832929 | Garrett et al. | Dec 2004 | B2 |
6845405 | Thelin | Jan 2005 | B1 |
6845427 | Atai-Azimi | Jan 2005 | B1 |
6850443 | Lofgren et al. | Feb 2005 | B2 |
6851055 | Boyle et al. | Feb 2005 | B1 |
6851063 | Boyle et al. | Feb 2005 | B1 |
6853731 | Boyle et al. | Feb 2005 | B1 |
6854022 | Thelin | Feb 2005 | B1 |
6862660 | Wilkins et al. | Mar 2005 | B1 |
6874065 | Pong | Mar 2005 | B1 |
6880043 | Castro et al. | Apr 2005 | B1 |
6882486 | Kupferman | Apr 2005 | B1 |
6884085 | Goldstone | Apr 2005 | B1 |
6888831 | Hospodor et al. | May 2005 | B1 |
6892217 | Hanmann et al. | May 2005 | B1 |
6892249 | Codilian et al. | May 2005 | B1 |
6892313 | Codilian et al. | May 2005 | B1 |
6895455 | Rothberg | May 2005 | B1 |
6895500 | Rothberg | May 2005 | B1 |
6898730 | Hanan | May 2005 | B1 |
6910099 | Wang et al. | Jun 2005 | B1 |
6928470 | Hamlin | Aug 2005 | B1 |
6931439 | Hanmann et al. | Aug 2005 | B1 |
6934104 | Kupferman | Aug 2005 | B1 |
6934713 | Schwartz et al. | Aug 2005 | B2 |
6940873 | Boyle et al. | Sep 2005 | B2 |
6943978 | Lee | Sep 2005 | B1 |
6948082 | Gschwind et al. | Sep 2005 | B2 |
6948113 | Shaver | Sep 2005 | B1 |
6948165 | Luu et al. | Sep 2005 | B1 |
6950267 | Liu et al. | Sep 2005 | B1 |
6954733 | Ellis et al. | Oct 2005 | B1 |
6961814 | Thelin et al. | Nov 2005 | B1 |
6965489 | Lee et al. | Nov 2005 | B1 |
6965563 | Hospodor et al. | Nov 2005 | B1 |
6965966 | Rothberg et al. | Nov 2005 | B1 |
6967799 | Lee | Nov 2005 | B1 |
6968422 | Codilian et al. | Nov 2005 | B1 |
6968450 | Rothberg et al. | Nov 2005 | B1 |
6973495 | Milne et al. | Dec 2005 | B1 |
6973570 | Hamlin | Dec 2005 | B1 |
6976190 | Goldstone | Dec 2005 | B1 |
6983316 | Milne et al. | Jan 2006 | B1 |
6986007 | Procyk et al. | Jan 2006 | B1 |
6986154 | Price et al. | Jan 2006 | B1 |
6995933 | Codilian et al. | Feb 2006 | B1 |
6996501 | Rothberg | Feb 2006 | B1 |
6996669 | Dang et al. | Feb 2006 | B1 |
7002926 | Eneboe et al. | Feb 2006 | B1 |
7003674 | Hamlin | Feb 2006 | B1 |
7006316 | Sargenti, Jr. et al. | Feb 2006 | B1 |
7009820 | Hogg | Mar 2006 | B1 |
7023639 | Kupferman | Apr 2006 | B1 |
7024491 | Hanmann et al. | Apr 2006 | B1 |
7024549 | Luu et al. | Apr 2006 | B1 |
7024614 | Thelin et al. | Apr 2006 | B1 |
7027716 | Boyle et al. | Apr 2006 | B1 |
7028174 | Atai-Azimi et al. | Apr 2006 | B1 |
7031902 | Catiller | Apr 2006 | B1 |
7046465 | Kupferman | May 2006 | B1 |
7046488 | Hogg | May 2006 | B1 |
7050252 | Vallis | May 2006 | B1 |
7054937 | Milne et al. | May 2006 | B1 |
7055000 | Severtson | May 2006 | B1 |
7055167 | Masters | May 2006 | B1 |
7057836 | Kupferman | Jun 2006 | B1 |
7062398 | Rothberg | Jun 2006 | B1 |
7075746 | Kupferman | Jul 2006 | B1 |
7076604 | Thelin | Jul 2006 | B1 |
7082494 | Thelin et al. | Jul 2006 | B1 |
7088538 | Codilian et al. | Aug 2006 | B1 |
7088545 | Singh et al. | Aug 2006 | B1 |
7092186 | Hogg | Aug 2006 | B1 |
7095577 | Codilian et al. | Aug 2006 | B1 |
7099095 | Subrahmanyam et al. | Aug 2006 | B1 |
7106537 | Bennett | Sep 2006 | B1 |
7106947 | Boyle et al. | Sep 2006 | B2 |
7110202 | Vasquez | Sep 2006 | B1 |
7111116 | Boyle et al. | Sep 2006 | B1 |
7114029 | Thelin | Sep 2006 | B1 |
7120737 | Thelin | Oct 2006 | B1 |
7120806 | Codilian et al. | Oct 2006 | B1 |
7126776 | Warren, Jr. et al. | Oct 2006 | B1 |
7129763 | Bennett et al. | Oct 2006 | B1 |
7130932 | Ghaffari | Oct 2006 | B1 |
7133600 | Boyle | Nov 2006 | B1 |
7136244 | Rothberg | Nov 2006 | B1 |
7146094 | Boyle | Dec 2006 | B1 |
7149046 | Coker et al. | Dec 2006 | B1 |
7150036 | Milne et al. | Dec 2006 | B1 |
7155616 | Hamlin | Dec 2006 | B1 |
7165124 | Fujita et al. | Jan 2007 | B2 |
7171108 | Masters et al. | Jan 2007 | B1 |
7171110 | Wilshire | Jan 2007 | B1 |
7194576 | Boyle | Mar 2007 | B1 |
7200698 | Rothberg | Apr 2007 | B1 |
7205805 | Bennett | Apr 2007 | B1 |
7206497 | Boyle et al. | Apr 2007 | B1 |
7215496 | Kupferman et al. | May 2007 | B1 |
7215771 | Hamlin | May 2007 | B1 |
7237054 | Cain et al. | Jun 2007 | B1 |
7240161 | Boyle | Jul 2007 | B1 |
7249365 | Price et al. | Jul 2007 | B1 |
7263709 | Krapf | Aug 2007 | B1 |
7274639 | Codilian et al. | Sep 2007 | B1 |
7274659 | Hospodor | Sep 2007 | B2 |
7275116 | Hanmann et al. | Sep 2007 | B1 |
7280302 | Masiewicz | Oct 2007 | B1 |
7292774 | Masters et al. | Nov 2007 | B1 |
7292775 | Boyle et al. | Nov 2007 | B1 |
7296284 | Price et al. | Nov 2007 | B1 |
7302501 | Cain et al. | Nov 2007 | B1 |
7302579 | Cain et al. | Nov 2007 | B1 |
7318088 | Mann | Jan 2008 | B1 |
7319806 | Willner et al. | Jan 2008 | B1 |
7325244 | Boyle et al. | Jan 2008 | B2 |
7330323 | Singh et al. | Feb 2008 | B1 |
7346790 | Klein | Mar 2008 | B1 |
7366641 | Masiewicz et al. | Apr 2008 | B1 |
7369340 | Dang et al. | May 2008 | B1 |
7369343 | Yeo et al. | May 2008 | B1 |
7372650 | Kupferman | May 2008 | B1 |
7380147 | Sun | May 2008 | B1 |
7392340 | Dang et al. | Jun 2008 | B1 |
7404013 | Masiewicz | Jul 2008 | B1 |
7406545 | Rothberg et al. | Jul 2008 | B1 |
7415571 | Hanan | Aug 2008 | B1 |
7436610 | Thelin | Oct 2008 | B1 |
7437502 | Coker | Oct 2008 | B1 |
7440214 | Ell et al. | Oct 2008 | B1 |
7451344 | Rothberg | Nov 2008 | B1 |
7457921 | Gabryjelski et al. | Nov 2008 | B2 |
7471483 | Ferris et al. | Dec 2008 | B1 |
7471486 | Coker et al. | Dec 2008 | B1 |
7486060 | Bennett | Feb 2009 | B1 |
7496493 | Stevens | Feb 2009 | B1 |
7518819 | Yu et al. | Apr 2009 | B1 |
7526184 | Parkinen et al. | Apr 2009 | B1 |
7539924 | Vasquez et al. | May 2009 | B1 |
7543117 | Hanan | Jun 2009 | B1 |
7551383 | Kupferman | Jun 2009 | B1 |
7562282 | Rothberg | Jul 2009 | B1 |
7574565 | De Souza | Aug 2009 | B2 |
7577973 | Kapner, III et al. | Aug 2009 | B1 |
7596797 | Kapner, III et al. | Sep 2009 | B1 |
7599139 | Bombet et al. | Oct 2009 | B1 |
7619841 | Kupferman | Nov 2009 | B1 |
7647544 | Masiewicz | Jan 2010 | B1 |
7649704 | Bombet et al. | Jan 2010 | B1 |
7653927 | Kapner, III et al. | Jan 2010 | B1 |
7656603 | Feb 2010 | B1 | |
7656763 | Jin et al. | Feb 2010 | B1 |
7657149 | Boyle | Feb 2010 | B2 |
7672072 | Boyle et al. | Mar 2010 | B1 |
7673075 | Masiewicz | Mar 2010 | B1 |
7688540 | Mei et al. | Mar 2010 | B1 |
7724461 | McFadyen et al. | May 2010 | B1 |
7725584 | Hanmann et al. | May 2010 | B1 |
7730295 | Lee | Jun 2010 | B1 |
7760458 | Trinh | Jul 2010 | B1 |
7768776 | Szeremeta et al. | Aug 2010 | B1 |
7804657 | Hogg et al. | Sep 2010 | B1 |
7813954 | Price et al. | Oct 2010 | B1 |
7827320 | Stevens | Nov 2010 | B1 |
7839588 | Dang et al. | Nov 2010 | B1 |
7843660 | Yeo | Nov 2010 | B1 |
7852596 | Boyle et al. | Dec 2010 | B2 |
7859782 | Lee | Dec 2010 | B1 |
7872822 | Rothberg | Jan 2011 | B1 |
7898756 | Wang | Mar 2011 | B1 |
7898762 | Guo et al. | Mar 2011 | B1 |
7900037 | Fallone et al. | Mar 2011 | B1 |
7907364 | Boyle et al. | Mar 2011 | B2 |
7929234 | Boyle et al. | Apr 2011 | B1 |
7933087 | Tsai et al. | Apr 2011 | B1 |
7933090 | Jung et al. | Apr 2011 | B1 |
7934030 | Sargenti, Jr. et al. | Apr 2011 | B1 |
7940491 | Szeremeta et al. | May 2011 | B2 |
7944639 | Wang | May 2011 | B1 |
7945727 | Rothberg et al. | May 2011 | B2 |
7949564 | Hughes et al. | May 2011 | B1 |
7974029 | Tsai et al. | Jul 2011 | B2 |
7974039 | Xu et al. | Jul 2011 | B1 |
7982993 | Tsai et al. | Jul 2011 | B1 |
7984200 | Bombet et al. | Jul 2011 | B1 |
7990648 | Wang | Aug 2011 | B1 |
7992179 | Kapner, III et al. | Aug 2011 | B1 |
8004785 | Tsai et al. | Aug 2011 | B1 |
8006027 | Stevens et al. | Aug 2011 | B1 |
8014094 | Jin | Sep 2011 | B1 |
8014977 | Masiewicz et al. | Sep 2011 | B1 |
8019914 | Vasquez et al. | Sep 2011 | B1 |
8040625 | Boyle et al. | Oct 2011 | B1 |
8078943 | Lee | Dec 2011 | B1 |
8079045 | Krapf et al. | Dec 2011 | B2 |
8082433 | Fallone et al. | Dec 2011 | B1 |
8085487 | Jung et al. | Dec 2011 | B1 |
8089719 | Dakroub | Jan 2012 | B1 |
8090902 | Bennett et al. | Jan 2012 | B1 |
8090906 | Blaha et al. | Jan 2012 | B1 |
8091112 | Elliott et al. | Jan 2012 | B1 |
8094396 | Zhang et al. | Jan 2012 | B1 |
8094401 | Peng et al. | Jan 2012 | B1 |
8116020 | Lee | Feb 2012 | B1 |
8116025 | Chan et al. | Feb 2012 | B1 |
8134793 | Vasquez et al. | Mar 2012 | B1 |
8134798 | Thelin et al. | Mar 2012 | B1 |
8139301 | Li et al. | Mar 2012 | B1 |
8139310 | Hogg | Mar 2012 | B1 |
8144419 | Liu | Mar 2012 | B1 |
8145452 | Masiewicz et al. | Mar 2012 | B1 |
8149528 | Suratman et al. | Apr 2012 | B1 |
8154812 | Boyle et al. | Apr 2012 | B1 |
8159768 | Miyamura | Apr 2012 | B1 |
8161328 | Wilshire | Apr 2012 | B1 |
8164849 | Szeremeta et al. | Apr 2012 | B1 |
8174780 | Tsai et al. | May 2012 | B1 |
8190575 | Ong et al. | May 2012 | B1 |
8194338 | Zhang | Jun 2012 | B1 |
8194340 | Boyle et al. | Jun 2012 | B1 |
8194341 | Boyle | Jun 2012 | B1 |
8201066 | Wang | Jun 2012 | B1 |
8271692 | Dinh et al. | Sep 2012 | B1 |
8279550 | Hogg | Oct 2012 | B1 |
8281218 | Ybarra et al. | Oct 2012 | B1 |
8285923 | Stevens | Oct 2012 | B2 |
8289656 | Huber | Oct 2012 | B1 |
8305705 | Roohr | Nov 2012 | B1 |
8307156 | Codilian et al. | Nov 2012 | B1 |
8310775 | Boguslawski et al. | Nov 2012 | B1 |
8315006 | Chahwan et al. | Nov 2012 | B1 |
8316263 | Gough et al. | Nov 2012 | B1 |
8320067 | Tsai et al. | Nov 2012 | B1 |
8324974 | Bennett | Dec 2012 | B1 |
8332695 | Dalphy et al. | Dec 2012 | B2 |
8341337 | Ong et al. | Dec 2012 | B1 |
8350628 | Bennett | Jan 2013 | B1 |
8356184 | Meyer et al. | Jan 2013 | B1 |
8370683 | Ryan et al. | Feb 2013 | B1 |
8375225 | Ybarra | Feb 2013 | B1 |
8375274 | Bonke | Feb 2013 | B1 |
8380922 | DeForest et al. | Feb 2013 | B1 |
8390948 | Hogg | Mar 2013 | B2 |
8390952 | Szeremeta | Mar 2013 | B1 |
8392689 | Lott | Mar 2013 | B1 |
8407393 | Yolar et al. | Mar 2013 | B1 |
8413010 | Vasquez et al. | Apr 2013 | B1 |
8417566 | Price et al. | Apr 2013 | B2 |
8421663 | Bennett | Apr 2013 | B1 |
8422172 | Dakroub et al. | Apr 2013 | B1 |
8427771 | Tsai | Apr 2013 | B1 |
8429343 | Tsai | Apr 2013 | B1 |
8433937 | Wheelock et al. | Apr 2013 | B1 |
8433977 | Vasquez et al. | Apr 2013 | B1 |
8458526 | Dalphy et al. | Jun 2013 | B2 |
8462466 | Huber | Jun 2013 | B2 |
8467151 | Huber | Jun 2013 | B1 |
8489841 | Strecke et al. | Jul 2013 | B1 |
8493679 | Boguslawski et al. | Jul 2013 | B1 |
8498074 | Mobley et al. | Jul 2013 | B1 |
8499198 | Messenger et al. | Jul 2013 | B1 |
8512049 | Huber et al. | Aug 2013 | B1 |
8514506 | Li et al. | Aug 2013 | B1 |
8531791 | Reid et al. | Sep 2013 | B1 |
8554741 | Malina | Oct 2013 | B1 |
8560759 | Boyle et al. | Oct 2013 | B1 |
8565053 | Chung | Oct 2013 | B1 |
8576511 | Coker et al. | Nov 2013 | B1 |
8578100 | Huynh et al. | Nov 2013 | B1 |
8578242 | Burton et al. | Nov 2013 | B1 |
8589773 | Wang et al. | Nov 2013 | B1 |
8593753 | Anderson | Nov 2013 | B1 |
8595432 | Vinson et al. | Nov 2013 | B1 |
8599510 | Fallone | Dec 2013 | B1 |
8601248 | Thorsted | Dec 2013 | B2 |
8611032 | Champion et al. | Dec 2013 | B2 |
8612650 | Carrie et al. | Dec 2013 | B1 |
8612706 | Madril et al. | Dec 2013 | B1 |
8612798 | Tsai | Dec 2013 | B1 |
8619383 | Jung et al. | Dec 2013 | B1 |
8621115 | Bombet et al. | Dec 2013 | B1 |
8621133 | Boyle | Dec 2013 | B1 |
8626463 | Stevens et al. | Jan 2014 | B2 |
8630052 | Jung et al. | Jan 2014 | B1 |
8630056 | Ong | Jan 2014 | B1 |
8631188 | Heath et al. | Jan 2014 | B1 |
8634158 | Chahwan et al. | Jan 2014 | B1 |
8635412 | Wilshire | Jan 2014 | B1 |
8640007 | Schulze | Jan 2014 | B1 |
8654619 | Cheng | Feb 2014 | B1 |
8661193 | Cobos et al. | Feb 2014 | B1 |
8667248 | Neppalli | Mar 2014 | B1 |
8670205 | Malina et al. | Mar 2014 | B1 |
8683295 | Syu et al. | Mar 2014 | B1 |
8683457 | Hughes et al. | Mar 2014 | B1 |
8687306 | Coker et al. | Apr 2014 | B1 |
8693133 | Lee et al. | Apr 2014 | B1 |
8694841 | Chung et al. | Apr 2014 | B1 |
8699159 | Malina | Apr 2014 | B1 |
8699171 | Boyle | Apr 2014 | B1 |
8699172 | Gunderson et al. | Apr 2014 | B1 |
8699175 | Olds et al. | Apr 2014 | B1 |
8699185 | Teh et al. | Apr 2014 | B1 |
8700850 | Lalouette | Apr 2014 | B1 |
8743502 | Bonke et al. | Jun 2014 | B1 |
8749910 | Dang et al. | Jun 2014 | B1 |
8751699 | Tsai et al. | Jun 2014 | B1 |
8755141 | Dang | Jun 2014 | B1 |
8755143 | Wilson et al. | Jun 2014 | B2 |
8756361 | Carlson et al. | Jun 2014 | B1 |
8756382 | Carlson et al. | Jun 2014 | B1 |
8769593 | Schwartz et al. | Jul 2014 | B1 |
8773802 | Anderson et al. | Jul 2014 | B1 |
8780478 | Huynh et al. | Jul 2014 | B1 |
8782334 | Boyle et al. | Jul 2014 | B1 |
8793532 | Tsai et al. | Jul 2014 | B1 |
8797669 | Burton | Aug 2014 | B1 |
8799977 | Kapner, III et al. | Aug 2014 | B1 |
8819375 | Pruett et al. | Aug 2014 | B1 |
8825976 | Jones | Sep 2014 | B1 |
8825977 | Syu et al. | Sep 2014 | B1 |
8856390 | Stevens | Oct 2014 | B1 |
20030084252 | Talagala | May 2003 | A1 |
20030236953 | Grieff et al. | Dec 2003 | A1 |
20040010625 | Lo et al. | Jan 2004 | A1 |
20040190554 | Galloway | Sep 2004 | A1 |
20050027894 | Ayyavu et al. | Feb 2005 | A1 |
20050027913 | Boecker et al. | Feb 2005 | A1 |
20050080842 | Takeuchi et al. | Apr 2005 | A1 |
20050102468 | Delaney et al. | May 2005 | A1 |
20050122991 | Golasky et al. | Jun 2005 | A1 |
20050186832 | Nemazie | Aug 2005 | A1 |
20060031612 | Bashford et al. | Feb 2006 | A1 |
20060075164 | Ooi | Apr 2006 | A1 |
20060248535 | Grover | Nov 2006 | A1 |
20060282709 | Shu | Dec 2006 | A1 |
20070118768 | Wolford | May 2007 | A1 |
20070189619 | Liu et al. | Aug 2007 | A1 |
20070226385 | Liu et al. | Sep 2007 | A1 |
20070226411 | Ehrlich | Sep 2007 | A1 |
20070260756 | Tseng et al. | Nov 2007 | A1 |
20080005398 | Huffman | Jan 2008 | A1 |
20090006787 | De Souza | Jan 2009 | A1 |
20090113702 | Hogg | May 2009 | A1 |
20100306551 | Meyer et al. | Dec 2010 | A1 |
20110035553 | Shepstone | Feb 2011 | A1 |
20110226729 | Hogg | Sep 2011 | A1 |
20120159042 | Lott et al. | Jun 2012 | A1 |
20120275050 | Wilson et al. | Nov 2012 | A1 |
20120281963 | Krapf et al. | Nov 2012 | A1 |
20120324980 | Nguyen et al. | Dec 2012 | A1 |
20130086311 | Huang | Apr 2013 | A1 |
20140201424 | Chen et al. | Jul 2014 | A1 |
Entry |
---|
Serial ATA: High Speed Serialized AT Attachment, Revision 1.0a, Jan. 7, 2003, pp. 1-311. |
Serial ATA International Organization: Serial ATA Revision 2.6, High Speed Serialized AT Attachment, Feb. 15, 2007, 600 pages. |
American National Standard for Information Technology, AT Attachment with Packet Interface 7, vol. 3, Serial Transport Protocols and Physical Interconnect (ATA/ATAPI-7 V3), ANSI INCITS 397-2005 (vol. 3), 305 pages. |
American National Standard for Information Technology, AT Attachment with Packet Interface 7, vol. 2, Parallel Transport Protocols and Physical Interconnect (ATA/ATAPI-7 V2), ANSI INCITS 397-2005 (vol. 2), 250 pages. |
American National Standard for Information Technology, AT Attachment with Packet Interface 7, vol. 1, Register Delivered Command Set, Logical Register Set (ATA/ATAPI-7 V1), ANSI INCITS 397-2005 (vol. 1), 393 pages. |
Office Action dated Dec. 28, 2009 from U.S. Appl. No. 12/058,572, 18 pages. |
Notice of Allowance dated Jun. 24, 2010 from U.S. Appl. No. 12/058,572, 14 pages. |
“Barriers and journaling filesystems,” LWN.net, http://lwn.net/Articles/283161, downloaded on Jul. 1, 2014, pp. 1-11. |
“Write barrier,” Wikipedia, the free encyclopedia, http://en.wikipedia.org/wiki/Write—barrier, downloaded on Jul. 1, 2014, 1 page. |
“The end of block barriers,” LWN.net, http://lwn.net/Articles/400541/, downloaded on Jul. 1, 2014, pp. 1-4. |
“Chapter 16. Write Barriers,” Fedora, http://docs.fedoraproject.org/en-US/Fedora/14/html/Storage—Administration—Guide/writebarr.html, downloaded on Jul. 1, 2014, pp. 1-2. |
Leszek Urbanski, “Electromagnetic Disk Drive,” Monolight, http://monolight.cc/2011106/barriers-caches-filesystems/, downloaded on Jul. 1, 2014, pp. 1-8. |
Office Action dated Nov. 23, 2015 from U.S. Appl. No. 14/322,520, 66 pages. |
Nathan Obr, U.S. Appl. No. 14/322,520, filed Jul. 2, 2014, 44 pages. |
Number | Date | Country | |
---|---|---|---|
61976023 | Apr 2014 | US |