The present technology pertains to the field of storage management, and particularly, directed to a method for handling input-output operations in zoned storage systems and devices.
A solid-state drive (SSD) is a data storage device that uses non-volatile solid-state memory to store persistent digitally encoded data. A solid-state drive can be configured to emulate a hard disk drive, i.e., a device that stores persistent digitally encoded data on the magnetic surfaces of rapidly rotating platters, and can be used to replace a hard disk drive in many applications. However, one difference between a solid-state drive and a hard disk drive is that solid-state drives do not have a spinning magnetic platter or actuator arm as used in hard disk drives. Therefore, solid-state drives are more rugged than hard disk drives and do not have the same operational delays.
Unfortunately, one disadvantage associated with a solid-state drive when compared to a hard disk drive is that once an erase block has been programmed with some data, the same erase block cannot be written to again until the erase block has been sequentially erased. To overcome this limitation, modern solid-state drives run complex pieces of software known as flash-translation layers to hide these complications by presenting a uniformly-mutable block interface similar to a hard-drive disk. Unfortunately, this layer requires a significant amount of space to manage its block indirection metadata and garbage collection activities.
A clustered network environment 100 that may implement one or more aspects of the technology described and illustrated herein is shown in
This technology provides a number of advantages including methods, non-transitory computer readable media, and devices that more effectively and efficiently handle input-output operations in zoned storage systems. With the disclosed technology, a pair-tuple of physical zones for each logical zone is used to manage the input-output operations. In the disclosed technology, one physical zone represents the logical older copy of the data, whereas the other physical zone represents the logical newer copy of the data. Once the zone has been fully written, the disclosed technology detaches the older physical zone from the logical zone and erasure of the data present in the older physical zone is scheduled so that the older physical zone can be used to perform other input-output operations. Additionally, the disclosed technology uses a zone random write area (or a buffer space) that can be used as a staging buffer to manage random input-output operations so that random input-output operations do not trigger the rewriting or erasing of the data in the access unit of the solid-state drive. By using the techniques illustrated below, the disclosed technology is able to support random input-output operations on the solid-state drives without significant operational delay.
In this example, node computing devices 106(1)-106(n) can be primary or local storage controllers or secondary or remote storage controllers that provide client devices 108(1)-108(n), with access to data stored within data storage devices 110(1)-110(n). The data storage apparatuses 102(1)-102(n) and/or node computing device 106(1)-106(n) of the examples described and illustrated herein are not limited to any particular geographic areas and can be clustered locally and/or remotely. Thus, in one example the data storage apparatuses 102(1)-102(n) and/or node computing device 106(1)-106(n) can be distributed over a plurality of storage systems located in a plurality of geographic locations. In another example, a clustered network can include data storage apparatuses 102(1)-102(n) and/or node computing device 106(1)-106(n) residing in a same geographic location (e.g., in a single onsite rack).
In the illustrated example, one or more of the client devices 108(1)-108(n), which may be, for example, personal computers (PCs), computing devices or storage (e.g., storage servers), and other computers or peripheral devices, are coupled to the respective data storage apparatuses 102(1)-102(n) by storage network connections 112(1)-112(n). Network connections 112(1)-112(n) may include a local area network (LAN) or wide area network (WAN), for example, that utilizes Network Attached Storage (NAS) protocols, such as a Common Internet File System (CIFS) protocol or a Network File System (NFS) protocol to exchange data packets, a Storage Area Network (SAN) protocol, such as Small Computer System Interface (SCSI) or Fiber Channel Protocol (FCP), an object protocol, such as S3, etc.
Illustratively, the client devices 108(1)-108(n) may be general-purpose computers running applications, and may interact with the data storage apparatuses 102(1)-102(n) using a client/server model for exchange of information. That is, the client devices 108(1)-108(n) may request data from the data storage apparatuses 102(1)-102(n) (e.g., data on one of the data storage devices 110(1)-110(n) managed by a network storage controller configured to process I/O commands issued by the client devices 108(1)-108(n)), and the data storage apparatuses 102(1)-102(n) may return results of the request to the client devices 108(1)-108(n) via the storage network connections 112(1)-112(n).
The node computing devices 106(1)-106(n) of the data storage apparatuses 102(1)-102(n) can include network or host nodes that are interconnected as a cluster to provide data storage and management services, such as to an enterprise having remote locations, cloud storage (e.g., a storage endpoint may be stored within a data cloud), etc., for example. Such a node computing device 106(1)-106(n) can be a device attached to the fabric 104 as a connection point, redistribution point, or communication endpoint, for example. One or more of the node computing devices 106(1)-106(n) may be capable of sending, receiving, and/or forwarding information over a network communications channel, and could comprise any type of device that meets any or all of these criteria.
In an example, the node computing device 106(1) may be located on a first storage site and the node computing device 106(n) may be located at a second storage site. The node computing devices 106(1) and 106(n) may be configured according to a disaster recovery configuration whereby a surviving node provides switchover access to the storage devices 110(1)-110(n) in the event a disaster occurs at a disaster storage site (e.g., the node computing device 106(1) provides client device 108(n) with switchover data access to storage devices 110(n) in the event a disaster occurs at the second storage site). In other examples, the node computing device 106(n) can be configured according to an archival configuration and/or the node computing devices 106(1)-106(n) can be configured based on another type of replication arrangement (e.g., to facilitate load sharing). Additionally, while two node computing devices 106 are illustrated in
As illustrated in the clustered network environment 100, node computing devices 106(1)-106(n) can include various functional components that coordinate to provide a distributed storage architecture. For example, the node computing devices 106(1)-106(n) can include network modules 114(1)-114(n) and disk modules 116(1)-116(n). Network modules 114(1)-114(n) can be configured to allow the node computing devices 106(1)-106(n) (e.g., network storage controllers) to connect with client devices 108(1)-108(n) over the storage network connections 112(1)-112(n), for example, allowing the client devices 108(1)-108(n) to send input-output operations to the node computing devices 106(1)-106(n).
Further, the network modules 114(1)-114(n) can provide connections with one or more other components through the cluster fabric 104. For example, the network module 114(1) of node computing device 106(1) can access the data storage device 110(n) by sending a request via the cluster fabric 104 through the disk module 116(n) of node computing device 106(n) when the node computing device 106(n) is available. Alternatively, when the node computing device 106(n) fails, the network module 114(1) of node computing device 106(1) can access the data storage device 110(n) directly via the cluster fabric 104. The cluster fabric 104 can include one or more local and/or wide area computing networks embodied as Infiniband, Fibre Channel (FC), or Ethernet networks, for example, although other types of networks supporting other protocols can also be used.
Disk modules 116(1)-116(n) can be configured to connect data storage devices 110(1)-110(n), such as disks or arrays of disks, SSDs, flash memory, or some other form of data storage, to the node computing devices 106(1)-106(n). Often, disk modules 116(1)-116(n) communicate with the data storage devices 110(1)-110(n) according to the SAN protocol, such as SCSI, FCP, SAS, NVMe, NVMe-oF for example, although other protocols can also be used. Thus, as seen from an operating system on either of node computing devices 106(1)-106(n), the data storage devices 110(1)-110(n) can appear as locally attached. In this manner, different node computing devices 106(1)-106(n), etc. may access data blocks through the operating system, rather than expressly requesting abstract files.
While the clustered network environment 100 illustrates an equal number of network modules 114(1)-114(n) and disk modules 116(1)-116(n), other examples may include a differing number of these modules. For example, there may be a plurality of network and disk modules interconnected in a cluster that do not have a one-to-one correspondence between the network and disk modules. That is, different node computing devices can have a different number of network and disk modules, and the same node computing device can have a different number of network modules than disk modules.
Further, one or more of the client devices 108(1)-108(n) can be networked with the node computing devices 106(1)-106(n) in the cluster, over the storage connections 112(1)-112(n). As an example, respective client devices 108(1)-108(n) that are networked to a cluster may request services (e.g., exchanging of information in the form of data packets) of node computing devices 106(1)-106(n) in the cluster, and the node computing devices 106(1)-106(n) can return results of the requested services to the client devices 108(1)-108(n). In one example, the client devices 108(1)-108(n) can exchange information with the network modules 114(1)-114(n) residing in the node computing devices 106(1)-106(n) (e.g., network hosts) in the data storage apparatuses 102(1)-102(n).
In one example, the storage apparatuses 102(1)-102(n) host aggregates corresponding to physical local and remote data storage devices, such as local flash or disk storage in the data storage devices 110(1)-110(n), for example. One or more of the data storage devices 110(1)-110(n) can include mass storage devices, such as disks of a disk array. The disks may comprise any type of mass storage devices, including but not limited to magnetic disk drives, flash memory, SSDs, storage class memories and any other similar media adapted to store information, including, for example, data (D) and/or parity (P) information. In this example, the SSDs in the data storage devices 110(1)-110(n) are arranged in a zoned namespace configuration (where the logical address space of the namespace is divided into zones) where a zone is a portion of the namespace (contiguous LBA range) with specific write access rules.
The aggregates include volumes 118(1)-118(n) in this example, although any number of volumes can be included in the aggregates. The volumes 118(1)-118(n) are virtual data stores that define an arrangement of storage and one or more file systems within the clustered network environment 100. Volumes 118(1)-118(n) can span a portion of a disk or other storage device, a collection of disks, or portions of disks, for example, and typically define an overall logical arrangement of file storage. In one example volumes 118(1)-118(n) can include stored data as one or more files or objects that reside in a hierarchical directory structure within the volumes 118(1)-118(n). Volumes 118(1)-118(n) are typically configured in formats that may be associated with particular storage systems, and respective volume formats typically comprise features that provide functionality to the volumes 118(1)-118(n), such as providing an ability for volumes 118(1)-118(n) to form clusters.
In one example, to facilitate access to data stored on the disks or other structures of the data storage device 110(1)-110(n), a file system (e.g., write anywhere file system (WAFL)) may be implemented that logically organizes the information as a hierarchical structure of directories and files. In this example, respective files may be implemented as a set of disk blocks configured to store information, whereas directories may be implemented as specially formatted files in which information about other files and directories are stored.
Data can be stored as files or objects within a physical volume and/or a virtual volume, which can be associated with respective volume identifiers, such as file system identifiers (FSIDs). The physical volumes correspond to at least a portion of physical storage devices, such as the data storage device 110(1)-110(n) (e.g., a Redundant Array of Independent (or Inexpensive) Disks (RAID system)) whose address, addressable space, location, etc. does not change. Typically the location of the physical volumes does not change in that the (range of) address(es) used to access it generally remains constant.
Virtual volumes, in contrast, are stored over an aggregate of disparate portions of different physical storage devices. Virtual volumes may be a collection of different available portions of different physical storage device locations, such as some available space from disks, for example. It will be appreciated that since the virtual volumes are not “tied” to any one particular storage device, virtual volumes can be said to include a layer of abstraction or virtualization, which allows them to be resized and/or flexible in some regards.
Further, virtual volumes can include one or more logical unit numbers (LUNs), directories, Qtrees, and/or files. Among other things, these features, but more particularly the LUNS, allow the disparate memory locations within which data is stored to be identified, for example, and grouped as a data storage unit. As such, the LUNs may be characterized as constituting a virtual disk or drive upon which data within the virtual volumes is stored within an aggregate. For example, LUNs are often referred to as virtual disks, such that they emulate a hard drive, while they actually comprise data blocks stored in various parts of a volume.
In one example, the data storage devices 110(1)-110(n) can have one or more physical ports, wherein each physical port can be assigned a target address (e.g., SCSI target address). To represent respective volumes, a target address on the data storage devices 110(1)-110(n) can be used to identify one or more of the LUNs. Thus, for example, when one of the node computing devices 106(1)-106(n) connects to a volume, a connection between the one of the node computing devices 106(1)-106(n) and one or more of the LUNs underlying the volume is created.
In one example, respective target addresses can identify multiple of the LUNs, such that a target address can represent multiple volumes. The I/O interface, which can be implemented as circuitry and/or software in a storage adapter or as executable code residing in memory and executed by a processor, for example, can connect to volumes by using one or more addresses that identify the one or more of the LUNs.
Referring to
The storage operating system 212 can also manage communications for the node computing device 106(1) among other devices that may be in a clustered network, such as attached to a cluster fabric 104. Thus, the node computing device 106(1) can respond to client device requests to manage data on one of the data storage devices 110(1)-110(n) (e.g., or additional clustered devices) in accordance with the client device requests.
The storage operating system 212 can also establish one or more file systems including software code and data structures that implement a persistent hierarchical namespace of files and directories, for example. As an example, when a new data storage device (not shown) is added to a clustered network system, the storage operating system 212 is informed where, in an existing directory tree, new files associated with the new data storage device are to be stored. This is often referred to as “mounting” a file system.
In the example node computing device 106(1), memory 202 can include storage locations that are addressable by the processor(s) 200 and adapters 204, 206, and 208 for storing related software application code and data structures. The processor(s) 200 and adapters 204, 206, and 208 may, for example, include processing elements and/or logic circuitry configured to execute the software code and manipulate the data structures.
The storage operating system 212, portions of which are typically resident in the memory 202 and executed by the processor(s) 200, invokes storage operations in support of a file service implemented by the node computing device 106(1). Other processing and memory mechanisms, including various computer readable media, may be used for storing and/or executing application instructions pertaining to the techniques described and illustrated herein. For example, the storage operating system 212 can also utilize one or more control files (not shown) to aid in the provisioning of virtual machines.
Accordingly, the examples may be embodied as one or more non-transitory computer readable media having machine or processor-executable instructions stored thereon for one or more aspects of the present technology, as described and illustrated by way of the examples herein, which when executed by the processor(s) 200, cause the processor(s) 200 to carry out the steps necessary to implement the methods of this technology, as described and illustrated with the examples herein. In some examples, the executable instructions are configured to perform one or more steps of a method, such as one or more of the exemplary methods described and illustrated later with reference to
The network adapter 204 in this example includes the mechanical, electrical and signaling circuitry needed to connect the node computing device 106(1) to one or more of the client devices 108(1)-108(n) over storage network connections 112(1)-112(n), which may comprise, among other things, a point-to-point connection or a shared medium, such as a local area network. In some examples, the network adapter 204 further communicates (e.g., using TCP/IP) via the fabric 104 and/or another network (e.g. a WAN) (not shown) with cloud storage devices to process storage operations associated with data stored thereon.
The storage adapter 208 cooperates with the storage operating system 212 executing on the node computing device 106(1) to access information requested by one of the client devices 108(1)-108(n) (e.g., to access data on a data storage device 110(1)-110(n) managed by a network storage controller). The information may be stored on any type of attached array of writeable media such as magnetic disk drives, SSDs, and/or any other similar media adapted to store information.
In the exemplary data storage devices 110(1)-110(n), information can be stored in data blocks on disks. The storage adapter 208 can include input/output (I/O) interface circuitry that couples to the disks over an I/O interconnect arrangement, such as a storage area network (SAN) protocol (e.g., Small Computer System Interface (SCSI), iSCSI, hyperSCSI, Fiber Channel Protocol (FCP)). The information is retrieved by the storage adapter 208 and, if necessary, processed by the processor(s) 200 (or the storage adapter 208 itself) prior to being forwarded over the system bus 210 to the network adapter 204 (and/or the cluster access adapter 206 if sending to another node computing device in the cluster) where the information is formatted into a data packet and returned to a requesting one of the client devices 108(1)-108(n), or alternatively sent to another node computing device attached via the cluster fabric 104. In some examples, a storage driver 214 in the memory 202 interfaces with the storage adapter to facilitate interactions with the data storage devices 110(1)-110(n), as described and illustrated in more detail later with reference to
Now, an exemplary method for handling input-output operations in zoned storage systems will be illustrated with reference to
In step 310, the node computing device 106(1) identifies a logical zone to write the data within the zoned namespace SSDs within the data storage devices 110(1)-110(n), based on the data present in the write request, although the node computing device 106(1) can identify the logical zone based on the logical block address provided in the received write request. In this example, a logical zone is a portion of the SSD namespace (logical grouping of volumes) with specific write access rules and maps to one or more erase blocks on the SSDs. Additionally in this example, the logical zone is associated with a physical zone where the physical zone is a physical location within the zoned namespace SSDs where data can be read, written, or erased. The mapping of the logical zone and the physical zone is present within the mapping table stored within the memory 202, although the mapping table may be present at other memory locations such as the zoned namespace SSDs. An example step 310 is illustrated in
In step 315, the node computing device 106(1) determines if the identified logical zone has a corresponding previous physical zone using the mapping table. As illustrated above, each logical zone generally has a corresponding physical zone assigned and the correlation between the logical zone and the previous physical zone is present within the mapping table. However, when there is no previous physical zone associated with the logical zone, the mapping table would not include the data associated with the previous physical zone. In other words, the mapping table would include a blank for the previous physical zone when there is no previous physical zone associated with the logical zone. Accordingly, if the node computing device 106(1) determines that the identified logical zone does not have a previous physical zone, then the No branch is taken to step 320.
In step 320, the node computing device 106(1) identifies a new empty physical zone from a list of unused physical zones. Additionally, the node computing device 106(1) moves the physical zone that is associated with the logical zone from the mapping table to the previous physical zone column and assigns the identified new physical zone as the physical zone the exemplary flow proceeds to step 325. An example of the step 320 is illustrated in
Referring back to
In step 325, the node computing device 106(1) performs the received write operation and the step of performing the received write operation will now be further illustrated with reference to
Referring to
In step 335, the node computing device 106(1) reads the missing blocks from the previous physical zone associated with the physical zone. An example of this step will now be further illustrated with reference to the mapping table 520 in
Referring to
In step 345, the node computing device 106(1) performs the commit operation. In this example, a commit operation relates to a command that carries instructions to transfer the data from the zone random write area to the physical zone. An example of step 345 will be further illustrated with reference to
Referring to
In
Subsequently, the node computing device 106(1) receives another write request to write data B3′. As illustrated above, the received write data B3′ is temporarily staged within the zone random write area 620 as illustrated in
Referring to
While step 345 and
An exemplary method for managing a read request in zoned storage systems will be illustrated with reference to the exemplary flowchart in
In step 910, the node computing device 106(1) identifies the logical zone and the corresponding previous physical zone along with the physical zone using the mapping table stored within the memory 206, although the node computing device 106(1) can use other techniques to identify the physical zone. In this example, the mapping table is used to map the logical zone to the corresponding previous physical zone and the physical zone.
In step 915, the node computing device 106(1) determines if the received read request is before the current position of a write pointer of the physical zone where the write pointer indicates the current position within the physical zone where the data is to be committed when destaged from the zone random write area. Accordingly, when the node computing device 106(1) determines that the received read operation is before the write pointer, then the Yes branch is taken to step 920.
In step 920, the node computing device 106(1) reads the requested data from the physical zone, provides it to the requesting client device 108(1) and the exemplary method ends at step 940.
However, back in step 915, if the node computing device 106(1) determines that the received read request is not before the write pointer, then the No branch is taken to step 925. In step 925, the node computing device 106(1) determines if the received read request is after the end of the zone random write area. As illustrated above in
In step 930, the node computing device 106(1) performs the read operation from the previous physical zone and the exemplary flow proceeds to step 940 where the exemplary method ends.
However, back in step 925, if the node computing device 106(1) determines that the read is not after the end of the zone random write area, then the No branch is taken to step 935. In step 935, the node computing device 106(1) refers to zone random write area bitmap which includes the data associated with whether the data block within the zone has been written to the zone random write area. Accordingly, if the node computing device 106(1) determines that the data block has been written, then the exemplary flow proceeds to step 920 where the read operation is performed from the block that is written in the physical zone and will implicitly retrieve the data from the zone random write area attached to the physical zone. However, when the node computing device from the zone random write area determines that the block has not been written, then the exemplary flow proceeds to step 930 where the read operation is performed by the previous physical zone and the exemplary method ends at step 940.
Accordingly, as illustrated and described by way of the examples here, the above illustrated technology uses a pair-tuple of physical zones for each logical zone is used to manage the input-output operations. In the disclosed technology, one physical zone represents the logical older copy of the data, whereas the other physical zone represents the logical newer copy of the data. Once the zone has been fully written, the disclosed technology detaches the older physical zone from the logical zone and erasure of the data present in the older physical zone is scheduled so that the older physical zone can be used to perform other input-output operations. Additionally, the disclosed technology uses a zone random write area (or a buffer space) that can be used as a staging buffer to manage random input-output operations so that random input-output operations do not trigger the rewriting or erasing of the data in the access unit of the solid-state drive. By using the techniques illustrated above, the disclosed technology can support random input-output operations on the solid-state drives without significant operational delay.
Having thus described the basic concept of the technology, it will be rather apparent to those skilled in the art that the foregoing detailed disclosure is intended to be presented by way of example only, and is not limiting. Various alterations, improvements, and modifications will occur and are intended to those skilled in the art, though not expressly stated herein. These alterations, improvements, and modifications are intended to be suggested hereby, and are within the spirit and scope of the technology. Additionally, the recited order of processing elements or sequences, or the use of numbers, letters, or other designations therefore, is not intended to limit the claimed processes to any order except as may be specified in the claims. Accordingly, the technology is limited only by the following claims and equivalents thereto.
Number | Name | Date | Kind |
---|---|---|---|
5664187 | Burkes et al. | Sep 1997 | A |
6101615 | Lyons | Aug 2000 | A |
8074021 | Miller et al. | Dec 2011 | B1 |
8463991 | Colgrove et al. | Jun 2013 | B2 |
8549222 | Kleiman et al. | Oct 2013 | B1 |
8775868 | Colgrove et al. | Jul 2014 | B2 |
8832373 | Colgrove et al. | Sep 2014 | B2 |
8850108 | Hayes et al. | Sep 2014 | B1 |
8862820 | Colgrove et al. | Oct 2014 | B2 |
9003144 | Hayes et al. | Apr 2015 | B1 |
9021297 | Hayes et al. | Apr 2015 | B1 |
9134917 | Kimmel et al. | Sep 2015 | B2 |
9201600 | Hayes et al. | Dec 2015 | B1 |
9218244 | Hayes et al. | Dec 2015 | B1 |
9229808 | Colgrove et al. | Jan 2016 | B2 |
9244769 | Colgrove et al. | Jan 2016 | B2 |
9367243 | Hayes et al. | Jun 2016 | B1 |
9483346 | Davis et al. | Nov 2016 | B2 |
9495255 | Davis et al. | Nov 2016 | B2 |
9525738 | Hayes et al. | Dec 2016 | B2 |
9563506 | Hayes et al. | Feb 2017 | B2 |
9588842 | Sanvido et al. | Mar 2017 | B1 |
9594633 | Colgrove et al. | Mar 2017 | B2 |
9672125 | Botes et al. | Jun 2017 | B2 |
9672905 | Gold et al. | Jun 2017 | B1 |
9798477 | Botes et al. | Oct 2017 | B2 |
9880899 | Davis et al. | Jan 2018 | B2 |
9934089 | Hayes et al. | Apr 2018 | B2 |
9967342 | Colgrove et al. | May 2018 | B2 |
10095701 | Faibish et al. | Oct 2018 | B1 |
10180879 | Colgrove et al. | Jan 2019 | B1 |
10248516 | Sanvido et al. | Apr 2019 | B1 |
10303547 | Hayes et al. | May 2019 | B2 |
10353777 | Bernat et al. | Jul 2019 | B2 |
10372506 | Baptist et al. | Aug 2019 | B2 |
10379763 | Colgrove et al. | Aug 2019 | B2 |
10387247 | Baptist et al. | Aug 2019 | B2 |
10387250 | Resch et al. | Aug 2019 | B2 |
10387256 | Dhuse et al. | Aug 2019 | B2 |
10402266 | Kirkpatrick et al. | Sep 2019 | B1 |
10417092 | Brennan et al. | Sep 2019 | B2 |
10432233 | Colgrove et al. | Oct 2019 | B1 |
10437673 | Baptist et al. | Oct 2019 | B2 |
10437678 | Resch | Oct 2019 | B2 |
10452289 | Colgrove et al. | Oct 2019 | B1 |
10467107 | Abrol et al. | Nov 2019 | B1 |
10489256 | Hayes et al. | Nov 2019 | B2 |
10503598 | Trichardt et al. | Dec 2019 | B2 |
10521120 | Miller et al. | Dec 2019 | B1 |
10530862 | Isely et al. | Jan 2020 | B2 |
10534661 | Resch | Jan 2020 | B2 |
10572176 | Davis et al. | Feb 2020 | B2 |
10579450 | Khadiwala et al. | Mar 2020 | B2 |
10606700 | Alnafoosi et al. | Mar 2020 | B2 |
10613974 | Dreier et al. | Apr 2020 | B2 |
10656871 | Peake | May 2020 | B2 |
10657000 | Resch | May 2020 | B2 |
10671480 | Hayes et al. | Jun 2020 | B2 |
RE48222 | Colgrove et al. | Sep 2020 | E |
10776204 | Resch et al. | Sep 2020 | B2 |
10810083 | Colgrove et al. | Oct 2020 | B1 |
10817375 | Colgrove et al. | Oct 2020 | B2 |
10838834 | Sanvido et al. | Nov 2020 | B1 |
10860424 | Dhuse et al. | Dec 2020 | B1 |
10891192 | Brennan et al. | Jan 2021 | B1 |
RE48448 | Colgrove et al. | Feb 2021 | E |
11269778 | Kanteti | Mar 2022 | B1 |
11340987 | Gole et al. | May 2022 | B1 |
11442646 | Agarwal | Sep 2022 | B2 |
11698836 | Gole et al. | Jul 2023 | B2 |
20060129873 | Hafner et al. | Jun 2006 | A1 |
20060242539 | Kang et al. | Oct 2006 | A1 |
20100332401 | Prahlad et al. | Dec 2010 | A1 |
20120084506 | Colgrove et al. | Apr 2012 | A1 |
20120151118 | Flynn | Jun 2012 | A1 |
20140281227 | Herron et al. | Sep 2014 | A1 |
20150169244 | Asnaashari et al. | Jun 2015 | A1 |
20150199151 | Klemm et al. | Jul 2015 | A1 |
20160313943 | Hashimoto et al. | Oct 2016 | A1 |
20160342470 | Cudak et al. | Nov 2016 | A1 |
20170124345 | Christiansen et al. | May 2017 | A1 |
20170220264 | Sokolov | Aug 2017 | A1 |
20190004964 | Kanno | Jan 2019 | A1 |
20190018788 | Yoshida et al. | Jan 2019 | A1 |
20190278663 | Mehta et al. | Sep 2019 | A1 |
20200089407 | Baca et al. | Mar 2020 | A1 |
20200394112 | Gupta et al. | Dec 2020 | A1 |
20200409589 | Bennett et al. | Dec 2020 | A1 |
20200409601 | Helmick et al. | Dec 2020 | A1 |
20210081273 | Helmick et al. | Mar 2021 | A1 |
20210081330 | Bennett | Mar 2021 | A1 |
20210132827 | Helmick et al. | May 2021 | A1 |
20210303188 | Bazarsky | Sep 2021 | A1 |
20210334006 | Singh | Oct 2021 | A1 |
20220027051 | Kant et al. | Jan 2022 | A1 |
20220137844 | Goss et al. | May 2022 | A1 |
20220197553 | Benhanokh et al. | Jun 2022 | A1 |
20220229596 | Jung | Jul 2022 | A1 |
20220244869 | Kanteti | Aug 2022 | A1 |
20220283900 | Gole et al. | Sep 2022 | A1 |
20220291838 | Gorobets et al. | Sep 2022 | A1 |
20230082636 | Zhu et al. | Mar 2023 | A1 |
20230107466 | Gole | Apr 2023 | A1 |
20230161500 | Doucette et al. | May 2023 | A1 |
20230195382 | Gole et al. | Jun 2023 | A1 |
20230195383 | Gole et al. | Jun 2023 | A1 |
Number | Date | Country |
---|---|---|
1343087 | Sep 2003 | EP |
Entry |
---|
International Search Report and Written Opinion of the International Searching Authority, dated Aug. 5, 2021. |
International Preliminary Report on Patentability, dated Oct. 25, 2022. |
Dholakia A., et al., “A New Intra-disk Redundancy Scheme for High-Reliability RAID Storage Systems in the Presence of Unrecoverable Errors,” ACM Transactions on Storage, May 2008, vol. 4 (1), Article 1, 42 pages. |
European Search Report for Application No. EP22157793 dated Jul. 19, 2022, 16 pages. |
Mao, B., et al., “HPDA: A Hybrid Parity-Based Disk Array for Enhanced Performance and reliability,” ACM Transactions on Storage (TOS), vol. 8, No. 1, Publication [online), Feb. 2012 [retrieved Apr. 4, 2016). Retrieved from the Internet: URL: http://or.nsfc.gov.cn/bitstream/00001903-5/90177/1/1000003549834.pdf , 20 pages. |
Mao B., et al., “HPDA: A Hybrid Parity-Based Disk Array for Enhanced Performance and Reliability,” Retrieved from Internet URL: https://www.researchgate.net/publication/224140602, May 2020; 13 pages. |
Netapp, Inc., “Data ONTAP®7 .3 Data Protection Online Backup and Recovery Guide,” Feb. 22, 2011, Part No. 210-05212_A0, 432 pages. |
Netapp, Inc., “Data ONTAP®7.3 Active/Active Configuration Guide,” Jun. 16, 2011, Part No. 210-05247_A0, 214 pages. |
Netapp, Inc., “Data ONTAP®7.3 Archive and Compliance Management Guide,” Mar. 4, 2010, Part No. 210-04827_A0, 180 pages. |
Netapp, Inc., “Data ONTAP®7.3 Block Access Management Guide for iSCSI and FC,” Mar. 4, 2010, Part No. 210-04752_B0, 202 pages. |
Netapp, Inc., “Data ONTAP®7.3 Data Protection Tape Backup and Recovery Guide,” Jan. 15, 2010, Part No. 210-04762_A0, 142 pages. |
Netapp, Inc., “Data ONTAP®7.3 Documentation Roadmap,” Jul. 9, 2008, Part No. 210-04229_A0, 8 pages. |
Netapp, Inc., “Data ONTAP®7.3 File Access and Protocols Management Guide,” Sep. 10, 2009, Part No. 210-04505_B0, 382 pages. |
Netapp, Inc., “Data ONTAP®7.3 MultiStore Management Guide,” Mar. 4, 2010, Part No. 210-04855_A0, 144 pages. |
Netapp, Inc., “Data ONTAP®7.3 Network Management Guide,” Jan. 15, 2010, Part No. 210-04757_A0, 222 pages. |
Netapp, Inc., “Data ONTAP®7.3 Software Setup Guide,” Nov. 4, 2010, Part No. 210-05045_A0, 116 pages. |
Netapp, Inc., “Data ONTAP®7.3 Storage Efficiency Management Guide,” Mar. 4, 2010, Part No. 210-04856_A0, 76 pages. |
Netapp, Inc., “Data ONTAP®7.3 Storage Management Guide,” May 3, 2012, Part No. 210-04766_B0, 356 pages. |
Netapp, Inc., “Data ONTAP®7.3 System Administration Guide,” Nov. 11, 2010, Part No. 210-05043_A0, 350 pages. |
Netapp, Inc., “Data ONTAP®7.3 Upgrade Guide,” Nov. 11, 2010, Part No. 210-05042_A0, 200 pages. |
Netapp, Inc., “Data ONTAP®7.3.7 Release Notes,” May 31, 2012, Part No. 215-06916_A0, 182 pages. |
Netapp, Inc., “Date ONTAP®7.3 Core Commands Quick Reference,” Jun. 2008, Part No. 215-03893_A0, 1 page. |
Netapp, Inc., “Notices,” 2010, Part No. 215-05705_A0, 46 pages. |
Netapp, Inc., “V-Series Systems Hardware Maintenance Guide,” Jul. 2006, Part No. 210-00975_A0, 202 pages. |
Netapp, Inc., “V-Series Systems Implementation Guide for Hitachi® Storage,” Dec. 2009, Part No. 210-04694_A0, 66 pages. |
Netapp, Inc., “V-Series Systems Installation Requirements and Reference Guide,” Oct. 2010, Part No. 210-05064_A0, 214 pages. |
Netapp, Inc., “V-Series Systems MetroCiuster Guide,” Jul. 2009, Part No. 210-04515_A0, 80 pages. |
Notice of Allowance for Co-pending U.S. Appl. No. 17/192,606, dated Jan. 28, 2022. |
NVM Express Base Specification; Mar. 9, 2020; Revision 1.4a; NVM Express Workgroup; 405 pages. |
International Search Report and Written Opinion for Patent Application No. PCT/US2022/049431, dated Mar. 3, 2023, 13 pages. |
Non-Final Office Action for Co-pending U.S. Appl. No. 17/650,940 dated Feb. 16, 2023. |
Notice of Allowance on co-pending U.S. Appl. No. 16/858,019, dated Dec. 20, 2022, 11 pages. |
Notice of Allowance on co-pending U.S. Appl. No. 17/727,511, dated Dec. 14, 2022, 12 pages. |
Notice of Allowance dated Apr. 14, 2023 for U.S. Appl. No. 17/727,511, filed Apr. 22, 2022, 2 pages. |
Notice of Allowance dated Jun. 1, 2023 for U.S. Appl. No. 17/727,511, filed Apr. 22, 2022, 2 pages. |
Non-Final Office Action dated Mar. 30, 2023 for U.S. Appl. No. 17/494,684, filed Oct. 5, 2021, 8 pages. |
Non-Final Office Action dated Apr. 18, 2023 for U.S. Appl. No. 17/456,012, filed Nov. 22, 2021, 34 pages. |
Non-Final Office Action dated May 15, 2023 for U.S. Appl. No. 17/650,936, filed Feb. 14, 2022, 19 pages. |
Notice of Allowance dated Mar. 1, 2023 for U.S. Appl. No. 17/727,511, filed Apr. 22, 2022, 15 pages. |
Notice of Allowance dated May 24, 2023 for U.S. Appl. No. 17/727,511, filed Apr. 22, 2022, 2 pages. |
Notice of Allowance dated Aug. 3, 2023 for U.S. Appl. No. 17/494,684, filed Oct. 5, 2021, 02 pages. |
Notice of Allowance dated Jul. 19, 2023 for U.S. Appl. No. 17/650,940, filed Feb. 14, 2022, 9 pages. |
Notice of Allowance dated Jun. 26, 2023 for U.S. Appl. No. 17/494,684, filed Oct. 5, 2021, 8 pages. |
Number | Date | Country | |
---|---|---|---|
20210334006 A1 | Oct 2021 | US |