The subject disclosure relates generally to storage systems. More specifically, this disclosure relates to various embodiments for replication of data in a geographically distributed storage environment.
The large increase in amount of data generated by digital systems has created a new set of challenges for data storage environments. Traditional storage area network (SAN) and/or network-attached storage (NAS) architectures have not been designed to support data storage and/or protection at large multi-petabyte capacity levels. Object storage technology can be utilized to meet these requirements. By utilizing object storage technology, organizations can not only keep up with rising capacity levels, but can also store these new capacity levels at a manageable cost point.
Typically, a scale-out, cluster-based, shared-nothing object storage that employs a microservices architecture pattern, for example, an ECS™ (formerly known as Elastic Cloud Storage) can be utilized as a storage environment for a new generation of workloads. ECS™ utilizes the latest trends in software architecture and development to achieve increased availability, capacity use efficiency, and performance. ECS™ uses a specific method for disk capacity management, wherein disk space is partitioned into a set of blocks of fixed size called chunks. User data is stored in these chunks and the chunks are shared. One chunk can comprise fragments of several user objects (e.g., object segments). Chunk content is modified in an append mode. When chunks become full, they are sealed, and the content of sealed chunks is immutable.
The above-described background relating to ECS™ is merely intended to provide a contextual overview of some current issues, and is not intended to be exhaustive. Other contextual information may become further apparent upon review of the following detailed description.
One or more embodiments are now described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the various embodiments. It may be evident, however, that the various embodiments can be practiced without these specific details, e.g., without applying to any particular networked environment or standard. In other instances, well-known structures and devices are shown in block diagram form in order to facilitate describing the embodiments in additional detail.
The term “cloud” as used herein can refer to a cluster of nodes (e.g., set of network servers), for example, within a distributed object storage system, that are communicatively and/or operatively coupled to each other, and that host a set of applications utilized for servicing user requests. In general, the cloud computing resources can communicate with user devices via most any wired and/or wireless communication network to provide access to services that are based in the cloud and not stored locally (e.g., on the user device). A typical cloud-computing environment can include multiple layers, aggregated together, that interact with each other to provide resources for end-users.
Example systems and methods disclosed herein, in one or more embodiments, relate to cloud storage systems, such as, but not limited to an ECS™ platform. The ECS™ platform combines the cost advantages of commodity infrastructure with the reliability, availability and serviceability of traditional arrays. ECS™ supports geographically distributed setups comprising two or more sites (also referred to herein as zones). The corresponding feature is called GEO. GEO can be used to provide additional protection of user data by means of replication. At the GEO level, some ECS implementations can implement a model which can be called Active-Active, meaning any zone can update any object. This disclosure provides an alternative to Active-Active, referred to as an Active-Passive model. In an Active-Passive model, some zones are in active mode and other zones are in passive mode, as described in further detain herein. We provide an implementation of an Active-Passive model for a geographically distributed storage system like ECS.
With regard to the ECS™ platform in general, ECS™ can comprise a cluster of nodes (also referred to as “cluster” herein) that delivers scalable and simple public cloud services with the reliability and/or control of a private-cloud infrastructure. Moreover, the ECS™ platform comprises a scale-out, cluster-based, shared-nothing object storage, which employs a microservices architecture pattern. The ECS™ platform can support storage, manipulation, and/or analysis of unstructured data on a massive scale on commodity hardware. As an example, ECS™ can support mobile, cloud, big data, content-sharing, and/or social networking applications. ECS™ can be deployed as a turnkey storage appliance or as a software product that can be installed on a set of qualified commodity servers and/or disks.
The ECS™ scale-out and geo-distributed architecture is a cloud platform that can provide at least the following features: (i) lower cost than public clouds; (ii) unmatched combination of storage efficiency and data access; (iii) anywhere read/write access with strong consistency that simplifies application development; (iv) no single point of failure to increase availability and performance; (v) universal accessibility that eliminates storage silos and inefficient extract, transform, load (ETL)/data movement processes; etc.
In an aspect, ECS™ does not rely on a file system for disk capacity management. Instead, ECS™ partitions disk space into a set of blocks of fixed size called chunks (e.g., having a chunk size of 128 MB). All user data is stored in these chunks and the chunks can be shared. Typically, a chunk can comprise fragments of several different user objects. The chunk content can be modified in an append-only mode. When a chunk becomes full, it can be sealed and the content of a sealed chunk is immutable. In a further aspect, a data chunk can comprise a plurality of object segments having a size of 2 MB that are appended in the data chunk. Thus, a data chunk of size 128 MB can have 64 object segments appended together. In general operation, a read request comprises a request to read (e.g., extract) a single object segment.
In an aspect, ECS™ is a cloud storage system that supports geographically distributed setups comprising two or more zones (e.g. sites). The corresponding feature is called GEO. GEO can be used to provide an additional protection of user data by means of replication. The replication mechanism can work at the chunks level, although it can also serve object-oriented goals. In an aspect, a chunk with data can be replicated to multiple remote sites. ECS™ can use GEO erasure coding techniques to minimize capacity overhead associated with GEO data protection. Although the systems and methods disclosed herein have been described with respect to object storage systems (e.g., ECS™), it is noted that the subject specification is not limited to object storage systems and can be utilized for most any storage systems. Thus, any of the embodiments, aspects, concepts, structures, functionalities or examples described herein are non-limiting, and the technology may be used in various ways that provide benefits and advantages in computing and data storage in general.
The GEO system is capable of employing, e.g., a replication to single site mode, a replication to all sites mode, or a replication to one or more regions mode depending on geographical considerations (e.g., distance and government rules on storing data), site or region storage size considerations (e.g., a site limited to storing 100 million data chunks). In a GEO system according to some embodiments, data chunks are continuously added to sites. In a GEO setup according to an aspect, a backup copy of the data chunk is created.
In an aspect, a replication to one site mode comprises replicating a backup copy of a data chunk at a single site (discussed in
ECS™ supports temporary unavailability of GEO zones. When a zone is temporarily unavailable we say that the zone is in temporary site outage (TSO) mode. Support for TSOs is accomplished in part through object management. Geographically distributed ECS™ setups maintain a global namespace and assure strong consistency for user data. This is achieved via defining a primary zone for each object. Normally, the primary zone is a zone that created an object. Then, we say that the primary zone owns the object or that the primary zone is the owner of the object. All requests related to the object can be handled by the zone that owns it.
When a zone that owns some object enters TSO mode, a new owner can be elected for the object. Note that every available zone of the GEO setup can become the owner. In the general case, a new object owner can facilitate update of the object. This means a potential conflict after the original owner of the object becomes available again. The system runs through a reconciliation phase to resolve conflicts. The ECS's conflict resolution strategy is very simple: choose a latest object update and discard potentially conflicting object updates.
In an ECS™ GEO model referred to as Active-Active, as noted herein, any zone can update any object. There are some drawbacks to the Active-Active model, for example, it provides a great deal of freedom for applications, which can connect to any zone of a GEO setup for data access. There is a price to pay for this freedom. In order to keep object conflict resolution simple, the Active-Active model encounters difficulty in supporting simultaneous TSO for two or more zones of a GEO setup.
Aspects of this disclosure provide an Active-Passive model, which can be deployed in cloud systems such as ECS™. An example Active-Passive model setup can include two or more regions, with each region in either active mode or passive mode. Each region can comprise multiple sites (zones). Applications can primarily work with the active sites (the sites from the active regions), and applications can have limited or no access to passive sites. Applications are free to automatically switch between the active sites. Applications switches to sites in passive regions can be restricted, e.g., requiring reconfiguration by a system administrator in order to allow application switches to sites in passive regions.
Potential aspects of Active-Passive model GEO setups include, e.g., in case of a misconfiguration of an application, the application can fail to access a passive region. Furthermore, Active-Passive model GEO setups can support up to N sites in TSO mode, where N is the number of regions and no more than one site from each region is in TSO mode. Furthermore, Active-Passive model GEO setups support high availability/disaster recovery (HA/DR) replication (discussed in
In an example Active-Passive model, a mode a given site operates in can be explicitly specified. There are two modes: Active and Passive. Active sites, sites in active mode, can accept data connections from applications. Passive sites, sites in passive mode, can be promoted to active mode in order to accept data connections from applications. In some optional embodiments, passive sites can be allowed to accept data connections from applications, but passive sites are not allowed to serve data creation/update requests. This option can be used for example to organize read failover during a temporary region outage (TRO).
In the Active-Passive model, data handling during and after TSO can include refusing “eligibility” and “positive suffrage” to sites in passive mode. In other words, unless it switches to active mode, a passive site cannot become an owner of data created within an active site. A passive site also cannot participate in election of a new owner. As a result, the Active-Passive model can allow two sites from different regions to enter TSO mode. Indeed, a passive site in TSO mode need not impact ownership transition for user data. Furthermore, a passive site in TSO need not impact reconciliation procedures after an active site exits TSO. Passive sites cannot store unique data updates, so active sites can negotiate an effective data version between themselves using any applicable conflict resolution policy, e.g. the ECS's “latest update wins” policy.
To the accomplishment of the foregoing and related ends, the disclosed subject matter, then, comprises one or more of the features hereinafter more fully described. The following description and the annexed drawings set forth in detail certain illustrative aspects of the subject matter. However, these aspects are indicative of but a few of the various ways in which the principles of the subject matter can be employed. Other aspects, advantages, and novel features of the disclosed subject matter will become apparent from the following detailed description when considered in conjunction with the provided drawings.
Clients 108, and in particular, an application 109 at one or more of the clients 108, can send data system-related requests to the cluster 102, which in general is configured as one large object namespace; there can be on the order of billions of objects such as 109a maintained in a cluster 102, for example. Application 109 can for example create object 109a at any of nodes 104(1)-104(M), in which case object 109a may be referred to as a “new object 109a”, or application 109 can request a previously created object 109a from any of nodes 104(1)-104(M), in which case object 109a can be referred to as a “stored object 109a”. To this end, a node such as the node 104(2) generally comprises ports 112 by which clients connect to the cloud storage system. Example ports 112 are provided for requests via various protocols, including but not limited to SMB (server message block), FTP (file transfer protocol), HTTP/HTTPS (hypertext transfer protocol), and NFS (Network File System); further, SSH (secure shell) allows administration-related requests, for example.
Each node, such as the node 104(2), includes an instance of an object storage system 114 and data services. In general, and in one or more implementations, e.g., ECS™, disk space is partitioned into a set of large blocks of fixed size called chunks; user data is stored in chunks. Chunks are shared, that is, one chunk can contain segments of multiple user objects; e.g., one chunk can contain mixed segments of some number of (e.g., three) user objects. A chunk manager 120 can be utilized to manage the chunks and their protection.
A CPU 122 and RAM 124 are shown for completeness; note that the RAM 124 can comprise at least some non-volatile RAM. The node includes storage devices such as disks 126, comprising hard disk drives and/or solid-state drives. It is noted that the storage devices can comprise volatile memory(s) or nonvolatile memory(s), or both volatile and nonvolatile memory(s). Examples of suitable types of volatile and non-volatile memory are described below with reference to
For a cluster that comprises a “GEO” zone of a geographically distributed storage system, at least one node, such as the node 104(2), includes or is coupled to reference tracking asynchronous replication logic 116 that synchronizes the cluster/zone 102 with other remote GEO zones, such as 118. Note that ECS™ implements asynchronous low-level replication, however the Active-Passive replication techniques herein are applicable to object level replication as well.
According to some embodiments, the nodes 102(1), 102(2) . . . 102(M) can be switched between active and passive modes. Any of the nodes 102(1), 102(2) . . . 102(M) and the zone 102 as a whole may be described as a data storage system, comprising at least one processor and at least one memory that stores executable instructions that, when executed by the at least one processor, facilitate performance of operations, including determining whether the data storage system 102 is in an active data storage mode or a passive data storage mode in connection with data (such as object 109a) stored on behalf of an application 109. Such determining can for example include checking an administrator setting for the zone 102.
In response to determining that the data storage system 102 is in an active data storage mode, any of the nodes 102(1), 102(2) . . . 102(M) can accept connections from the application 109, permitting the application 109 to store one or more new data objects 109a at the data storage system 102, and replicating the one or more new data objects 109a to other data storage systems 118 other than the data storage system 102. The nodes 102(1), 102(2) . . . 102(M) can furthermore participate in object ownership management for the application 109, and accept replicated application 109 data from one or more of the other data storage systems 118. As described further in connection with
In response to determining that the data storage system 102 is in the passive data storage mode, any of the nodes 102(1), 102(2) . . . 102(M) can block at least storage of the one or more new data objects 109a by the application 109, while nonetheless accepting replicated data from one or more of the other data storage systems 118. Furthermore, nodes 102(1), 102(2) . . . 102(M) can optionally block connections from the application 109, thereby neither allowing creation of new objects 109a nor serving previously stored objects to application 109. Furthermore, any of the nodes 102(1), 102(2) . . . 102(M) in passive data storage mode can be omitted from object ownership management for the application 109, as described herein.
In an aspect, objects and corresponding chunks can be protected locally at a zone 102 or node 102(1), 102(2) . . . 102(M) by employing a mirroring, a parity, and/or an erasure coding protection measure. During erasure coding, a chunk can be divided into k data fragments of equal size. To encode the chunk, redundant m coding fragments are created so that the system can tolerate the loss of any m fragments. The process of generating the coding fragments is called encoding. The process of data fragments recovery using available data and coding fragments is called decoding. As an example, the encoding operation can be represented with the equation below:
Ci=Σj=1kCi,j (1)
wherein,
Ci,j=Xi,j*Dj (2)
and wherein, Xi,j is a defined coefficient from a coding matrix (e.g., wherein i, j, and/or k can be most any integer). Further, j is an index assigned to the data fragment. It is noted that Dj are independent data fragments and Ci are coding fragments.
Additionally the systems and methods disclosed herein support geographically distributed setups (GEO) comprising two or more zones. GEO can be used to provide an additional protection of user data by means of replication. Replication can store object data at other zones. Replication can optionally be implemented at the chunk level, wherein a backup copy of a chunk stored in a primary zone can be replicated to one or more secondary zones. Each zone protects the chunks it stores. If a copy of a chunk becomes inaccessible, it can be recovered using its other copy. This process is called GEO recovery. In case of GEO erasure coding, remote backup copies of data chunks can be used as data fragments and coding fragments created for such data fragments are stored as coding chunks.
In the replication to regions mode, a copy of the chunk is replicated in one or more regions. According to the exemplary embodiment, the data chunk is replicated at all the regions and at a single site within each region. For example, a backup copy of chunk A 520 is stored at region 2 514 and at region 3 516, and a backup copy of chunk B 522 is stored at region 2 514 and at region 3 516. According to an exemplary embodiment, when the site 1.1 512A of region 1 512 creates chunk A 520, a backup copy of the chunk (e.g., chunk A′ 520A is created in each remote region. Namely, backup chunk A′ 520A is created in site 2.1514A of region 2 514 and backup chunk A′ 520A is create in site 3.1 516A of region 3 516. When site 1.1 512A of region 1 512 creates chunk B 522, a backup copy of the chunk is created in each remote region. Namely, backup chunk B′ 522A is created in site 2.2 514B of region 2 514 and backup chunk B′ 522A is created in site 3.2 516B of region 3 516. Thus, 3 copies (the number of regions) of each chunk are available.
One advantage of dividing sites into regions and storing a copy in each region is, to save hardware and memory resources by reducing the footprint of user data in a GEO environment. The data can be retrieved from a region that could be geographically hundreds of miles away. It is unlikely that the event that caused a failure at a site will cause failure at site in a different region. Thus, replication by region provides reliability of data access during a site failure, while reducing overall resources needed to replicate data. With replication to one or more regions, the system can tolerate loss of all but one region without data loss. Another advantage is that handling read requests for data can be reduced. Since read requests can be directed to a site that is continents away (e.g., Europe site requesting data read from Australia), the handling of such requests can take considerably more time. However, in the exemplary embodiment, read requests can be handled by a site having the copy of data chunk and located in adjacent region.
In the replication to active and passive regions mode, a copy of the chunk is replicated in both active and passive regions. According to the exemplary embodiment, the data chunk is replicated at sites in active regions and at a single site within each passive region. For example, a backup copy of chunk A 620 is stored at active site 1.2 612B, and at passive region 2 614, and at passive region 3 616, and a backup copy of chunk B 622 is stored at active site 1.1 612A, and at passive region 2 614, and at passive region 3 616. According to an exemplary embodiment, when the active site 1.1 612A of region 1 612 creates chunk A 620, a backup copy of the chunk (e.g., chunk A′ 620A is created in active sites, as well as in passive regions). Namely, backup chunk A′ 620A is created in active site 1.2 612B of region 1 612, site 2.1 614A of region 2 614, and site 3.1 616A of region 3 616.
When site 1.2 612B of region 1 612 creates chunk B 622, a backup copy of the chunk is created at sites in active regions and at a single site within each passive region. Namely, backup chunk B′ 622A is created at site 1.1 612A of region 1 512, at site 2.2 614B of region 2 614, and at site 3.2 616B of region 3 616. Thus, in this example 3 copies of each chunk are available.
In the example illustrated by
In further discussion of
Similarly a primary site (active site 1.1 612A) can find a site in each passive region 614, 616 and replicate its chunk there. The destination passive sites (in this example, passive sites 2.1 614A and 3.1 616A) can also protect the chunk locally.
A geographically distributed storage system 600 which employs replication to active and passive regions can include features to handle permanent active site or active region failures. When an active site (active site 1.1 612A) is marked as a permanent site outage (PSO), the active region 612 is to recover the site's state using its region 612 resources. Region 612 resources can optionally be expanded via adding a new site. When an entire active region 612 fails, one of passive regions 614, 616 must be made active. The application 109 is switched to the new active region. The new active region (e.g., 614) becomes the rightful owner of all user data/application objects. The new active region 614 drives recovery of all user data/application objects. Besides, as a new active region, new active region 614 produces a required number of region-local copies of each chunk to meet the HA requirements, by replicating data to multiple sites 2.1 614A and 202614B.
In another aspect, a data storage system represented by active site 1.1 612A, and the other data storage systems (sites) illustrated in
In some examples, replicating new data objects (e.g., included in chunk A 620) to a first active data storage system can comprise replicating the new data objects to at most a defined number of the active data storage systems 612A, 612B, which need not necessarily include all of the active sites in active region 612. The replicating of new data objects to at most the defined number of the active data storage systems 612A, 612B can comprise applying a distribution protocol, such as round robin or otherwise, to select among the active data storage systems 612A, 612B.
Meanwhile, replicating the new data objects to a passive data storage system can comprise replicating the new data objects to at least one of the passive data storage systems in each of the passive regions, for example, by replicating to sites 614A, 616A in regions 614 and 616. It will be appreciated that more or fewer passive sites can also be selected for replication in some embodiments.
Active data storage systems can participate in object ownership management for the application 109. In circumstances wherein a data storage system, such as active site 1.1 612A is designated as an owner of one or more new data objects, i.e. objects which can be included in chunk A 620 stored at the data storage system 612A by the application 109, active site 101612A can handle requests related to the one or more new data objects (included in chunk A 620), as such requests are forwarded from, e.g., other active data storage systems such as 612B.
In response to a site outage being determined to have taken place at another active site, e.g., a first active data storage system represented by active site 1.2 612B, an active site 1.1 612A can participate in object ownership management by selecting, from among one or more other active data storage systems/sites, a new owner for one or more data objects owned by the at least one first active data storage system (active site 1.2 612B). In
Under some circumstances, a site or region can switch from passive to active. For example, in response to a failure of multiple data storage systems, e.g., failure of all of the active sites in the active region 612, a passive data storage system (passive site 614A) can switch the from the passive data storage mode to the active data storage mode. Passive site 614A can initiate replication of application data (e.g., all chunks including application data) to data storage systems, such as passive site 2.2 614B within a same region 614 as the data storage system 614A. This can allow passive region 614 to take over as the active region for an application.
In some embodiments, the primary site (e.g., replication component 710 of primary site) scans available regions to identify suitable active and passive regions to store copies of data chunks. In some embodiments, the replication component 710 identifies suitable sites within active and passive regions, and replicates the data chunk at the active and passive sites. The active and passive sites protect replicated data chunks locally using for example, but not limited to, mirroring, parity or erasure coding techniques.
Aspects of the processor 706 can constitute machine-executable component(s) embodied within machine(s), e.g., embodied in one or more computer readable mediums (or media) associated with one or more machines. Such component(s), when executed by the one or more machines, e.g., computer(s), computing device(s), virtual machine(s), etc. can cause the machine(s) to perform the operations described herein. In an aspect, memory 704 can store computer executable components and instructions. It is noted that the memory 704 can comprise volatile memory(s) or nonvolatile memory(s) or can comprise both volatile and nonvolatile memory(s). Examples of suitable types of volatile and non-volatile memory are described below with reference to
In some embodiments, the chunk manager 802 can further comprise a recovery component 814 that handles data chunk recovery in the event of a site failure that causes the data chunk to become inaccessible. In the event of site failure, several data chunks are lost. The recovery of impacted chunks can be handled one at a time by the recovery component 814. The recovery comprises recreating a site using available remaining available sites. For each chunk, a primary recovery site is found among the remaining sites to drive the recovery of the data chunk. Based on availability, the primary site, a secondary site or a tertiary site can serve as the primary recovery site to drive the recovery of the data chunk. In some embodiments, all missing chunk copies can be re-created within the original regions. In the event of region failure, a primary recovery region is found among the remaining regions to drive the recovery of the missing chunks. Based on availability, the primary region, secondary region or tertiary region can serve as the primary recovery region to drive the recovery of the data chunk. In the event the primary site is not available or inaccessible, the secondary site becomes the primary site, the tertiary site becomes the secondary site, and a new site is selected as the tertiary site.
In non-limiting example embodiments, a computing device (or system) (e.g., computer 1012) is provided, the device or system comprising one or more processors and one or more memories that stores executable instructions that, when executed by the one or more processors, can facilitate performance of the operations as described herein, including the non-limiting methods as illustrated in the flow diagrams of
Operation 902 depicts, determining, by a data storage system comprising a processor and a memory, whether the data storage system is in an active data storage mode or a passive data storage mode. For example, with reference to
If the data storage system determines it is in active mode, the illustrated method continues via arrow 903. If the data storage system determines it is in passive mode, the illustrated method continues via arrow 904. It is also noted that any given data storage system can operate in active mode occasionally and in passive mode occasionally.
Operation 905 depicts data protection 905. Regardless of whether the data storage system is in active or passive mode, the data storage system can perform data protection operations at block 905 or at any time in the illustrated method. The data storage system can protect data objects locally at the data storage system using, for example, one or more of a mirroring, a parity, and an erasure coding protection measure.
Operation 906 depicts, in response to determining that the data storage system is in the active data storage mode, such as site 1.1 612A in
Operation 908 depicts replicating data to active data storage systems, such as site 1.2 612B in
Operation 909 depicts replicating data to passive data storage systems, such as site 2.1 614A in
Referring back to operations 908 and 909, it will be appreciated from
For example, replicating the one or more new data objects 109a to the at least one first active data storage system such as site 1.2 612B can comprise replicating the one or more new data objects 109a to at most a defined number of the active data storage systems. Any predefined number can be used, and the number can be less than or equal to the total available sites in active mode. A distribution protocol, such as round robin or any desired protocol, can be applied to select which among the active data storage systems will receive replicated data for a particular application.
In contrast, replicating the one or more new data objects 109a, at operation 909 to at least one second passive data storage system (such as, but not limited to site 2.1 614A) can comprise replicating the one or more new data objects 109a to at least one of the passive data storage systems in each of the passive regions 614, 616. In other words, a replicate by region approach, introduced in
Operation 910 depicts owner processing, whereby an active data storage system can participate in object ownership management. For example, when the data storage system at site 1.1 612A is designated as an owner of the one or more new data objects 109a stored at the site 1.1 612A by the application 109, the site 1.1 612A can handle requests related to the one or more new data objects 109a forwarded from other active sites, e.g., the at least one first active data storage system a site 1.2 612B.
Operation 911 depicts new owner selection and conflict resolution. Operation 911 will generally be performed responsive to a TSO or PSO of a site that was previously the owner of an object. Operation 911 is another example aspect of participating in object ownership management. For example, if site 1.2 612B experiences a TSO, then in response to a site outage being determined to have taken place at the at least one first active data storage system 612B, site 1.1 612A can select, from among one or more other active data storage systems, e.g., in region 612, a new owner for one or more data objects owned by the at least one first active data storage system 612B. Selecting the new owner can include any selection method, e.g., site 1.1 612A can optionally participate in an election process with the one or more other active data storage systems (e.g., in region 612). The passive data storage systems need not be included in the election or other new owner selection process. In response to a return of the at least one first active data storage system at site 1.2 612B to an operational state, site 1.1 612A, if it served as a temporary owner, can reconcile conflicts between the new owner (site 1.1 612A) and the at least one first active data storage system (site 1.2 612B).
Operation 912 depicts, in response to determining that the data storage system at, e.g., site 2.1 614A is in the passive data storage mode, blocking at least storage of the one or more new data objects 109a by the application 109, and accepting replicated data 620A from one or more of the other data storage systems, such as site 1.1 612A. In further response to determining that the data storage system at site 2.1 614A is in the passive data storage mode, site 2.1 614A can block connections from the application 109. Site 2.1 614A is used in this discussion because it is shown in passive mode in
Operation 913 depicts passive to active switchover processing. Similar to operation 911, operation 913 will generally be performed responsive to a TSO or PSO of an active site that was previously the owner of an object. In response to a failure the owner site, or of multiple of the other data storage systems, e.g., all active sites for an application, a site 2.1 614A in passive mode can switch the data storage system at site 2.1 614A from the passive data storage mode to the active data storage mode. In such a circumstance, site 2.1 614A can also initiate replication of application data at data storage systems located at sites within a same region 614 as the data storage system 614A. The region 614 and all sites therein can optionally switch from passive to active mode and can begin replicating data as described herein for active regions.
Generally, program modules include routines, programs, components, data structures, etc., that perform particular tasks or implement particular abstract data types. Moreover, those skilled in the art will appreciate that the inventive methods can be practiced with other computer system configurations, including single-processor or multiprocessor computer systems, minicomputers, mainframe computers, as well as personal computers, hand-held computing devices, microprocessor-based or programmable consumer electronics, and the like, each of which can be operatively coupled to one or more associated devices. The illustrated aspects of the specification can also be practiced in distributed computing environments where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote memory storage devices.
Computing devices typically include a variety of media, which can include computer-readable storage media and/or communications media, which two terms are used herein differently from one another as follows. Computer-readable storage media can be any available storage media that can be accessed by the computer and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer-readable storage media can be implemented in connection with any method or technology for storage of information such as computer-readable instructions, program modules, structured data, or unstructured data. Computer-readable storage media can include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or other tangible and/or non-transitory media which can be used to store desired information. Computer-readable storage media can be accessed by one or more local or remote computing devices, e.g., via access requests, queries or other data retrieval protocols, for a variety of operations with respect to the information stored by the medium.
Communications media typically embody computer-readable instructions, data structures, program modules or other structured or unstructured data in a data signal such as a modulated data signal, (e.g., a carrier wave or other transport mechanism), and includes any information delivery or transport media. The term “modulated data signal” or signals refers to a signal that has one or more of its characteristics set or changed in such a manner as to encode information in one or more signals. By way of example, and not limitation, communication media include wired media, such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), infrared and other wireless media.
With reference to
System bus 1018 can be any of several types of bus structure(s) comprising a memory bus or a memory controller, a peripheral bus or an external bus, and/or a local bus using any variety of available bus architectures comprising, but not limited to, industrial standard architecture (ISA), micro-channel architecture (MSA), extended ISA (EISA), intelligent drive electronics (IDE), VESA local bus (VLB), peripheral component interconnect (PCI), card bus, universal serial bus (USB), advanced graphics port (AGP), personal computer memory card international association bus (PCMCIA), Firewire (IEEE 1394), small computer systems interface (SCSI), and/or controller area network (CAN) bus used in vehicles.
System memory 1016 comprises volatile memory 1020 and nonvolatile memory 1022. A basic input/output system (BIOS), comprising routines to transfer information between elements within computer 1012, such as during start-up, can be stored in nonvolatile memory 1022. By way of illustration, and not limitation, nonvolatile memory 1022 can comprise ROM, PROM, EPROM, EEPROM, or flash memory. Volatile memory 1020 comprises RAM, which acts as external cache memory. By way of illustration and not limitation, RAM is available in many forms such as SRAM, dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM).
Computer 1012 also comprises removable/non-removable, volatile/non-volatile computer storage media.
It is to be appreciated that
A user can enter commands or information into computer 1012 through input device(s) 1036. Input devices 1036 comprise, but are not limited to, a pointing device such as a mouse, trackball, stylus, touch pad, keyboard, microphone, joystick, game pad, satellite dish, scanner, TV tuner card, digital camera, digital video camera, web camera, cellular phone, user equipment, smartphone, and the like. These and other input devices connect to processing unit 1014 through system bus 1018 via interface port(s) 1038. Interface port(s) 1038 comprise, for example, a serial port, a parallel port, a game port, a universal serial bus (USB), a wireless based port, e.g., Wi-Fi, Bluetooth®, etc. Output device(s) 1040 use some of the same type of ports as input device(s) 1036.
Thus, for example, a USB port can be used to provide input to computer 1012 and to output information from computer 1012 to an output device 1040. Output adapter 1042 is provided to illustrate that there are some output devices 1040, like display devices, light projection devices, monitors, speakers, and printers, among other output devices 1040, which use special adapters. Output adapters 1042 comprise, by way of illustration and not limitation, video and sound devices, cards, etc. that provide means of connection between output device 1040 and system bus 1018. It should be noted that other devices and/or systems of devices provide both input and output capabilities such as remote computer(s) 1044.
Computer 1012 can operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s) 1044. Remote computer(s) 1044 can be a personal computer, a server, a router, a network PC, a workstation, a microprocessor based appliance, a peer device, or other common network node and the like, and typically comprises many or all of the elements described relative to computer 1012.
For purposes of brevity, only a memory storage device 1046 is illustrated with remote computer(s) 1044. Remote computer(s) 1044 is logically connected to computer 1012 through a network interface 1048 and then physically and/or wirelessly connected via communication connection 1050. Network interface 1048 encompasses wire and/or wireless communication networks such as local-area networks (LAN) and wide-area networks (WAN). LAN technologies comprise fiber distributed data interface (FDDI), copper distributed data interface (CDDI), Ethernet, token ring and the like. WAN technologies comprise, but are not limited to, point-to-point links, circuit switching networks like integrated services digital networks (ISDN) and variations thereon, packet switching networks, and digital subscriber lines (DSL).
Communication connection(s) 1050 refer(s) to hardware/software employed to connect network interface 1048 to bus 1018. While communication connection 1050 is shown for illustrative clarity inside computer 1012, it can also be external to computer 1012. The hardware/software for connection to network interface 1048 can comprise, for example, internal and external technologies such as modems, comprising regular telephone grade modems, cable modems and DSL modems, wireless modems, ISDN adapters, and Ethernet cards.
The computer 1012 can operate in a networked environment using logical connections via wired and/or wireless communications to one or more remote computers, cellular based devices, user equipment, smartphones, or other computing devices, such as workstations, server computers, routers, personal computers, portable computers, microprocessor-based entertainment appliances, peer devices or other common network nodes, etc. The computer 1012 can connect to other devices/networks by way of antenna, port, network interface adaptor, wireless access point, modem, and/or the like.
The computer 1012 is operable to communicate with any wireless devices or entities operatively disposed in wireless communication, e.g., a printer, scanner, desktop and/or portable computer, portable data assistant, communications satellite, user equipment, cellular base device, smartphone, any piece of equipment or location associated with a wirelessly detectable tag (e.g., scanner, a kiosk, news stand, restroom), and telephone. This comprises at least Wi-Fi and Bluetooth® wireless technologies. Thus, the communication can be a predefined structure as with a conventional network or simply an ad hoc communication between at least two devices.
The computing system 1000 is operable to communicate with any wireless devices or entities operatively disposed in wireless communication, e.g., desktop and/or portable computer, server, communications satellite, etc. This includes at least Wi-Fi and Bluetooth® wireless technologies. Thus, the communication can be a predefined structure as with a conventional network or simply an ad hoc communication between at least two devices.
Wi-Fi, or Wireless Fidelity, allows connection to the Internet from a couch at home, a bed in a hotel room, or a conference room at work, without wires. Wi-Fi is a wireless technology similar to that used in a cell phone that enables such devices, e.g., computers, to send and receive data indoors and out; anywhere within the range of a base station. Wi-Fi networks use radio technologies called IEEE 1002.11 (a, b, g, n, etc.) to provide secure, reliable, fast wireless connectivity. A Wi-Fi network can be used to connect computers to each other, to the Internet, and to wired networks (which use IEEE 1002.3 or Ethernet). Wi-Fi networks operate in the unlicensed 5 GHz radio band at a 54 Mbps (802.11a) data rate, and/or a 2.4 GHz radio band at an 11 Mbps (802.11b), a 54 Mbps (802.11g) data rate, or up to a 600 Mbps (802.11n) data rate for example, or with products that contain both bands (dual band), so the networks can provide real-world performance similar to the basic 11BaseT wired Ethernet networks used in many offices.
As it employed in the subject specification, the term “processor” can refer to substantially any computing processing unit or device comprising, but not limited to comprising, single-core processors; single-processors with software multithread execution capability; multi-core processors; multi-core processors with software multithread execution capability; multi-core processors with hardware multithread technology; parallel platforms; and parallel platforms with distributed shared memory in a single machine or multiple machines. Additionally, a processor can refer to an integrated circuit, a state machine, an application specific integrated circuit (ASIC), a digital signal processor (DSP), a programmable gate array (PGA) including a field programmable gate array (FPGA), a programmable logic controller (PLC), a complex programmable logic device (CPLD), a discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. Processors can exploit nano-scale architectures such as, but not limited to, molecular and quantum-dot based transistors, switches and gates, in order to optimize space usage or enhance performance of user equipment. A processor may also be implemented as a combination of computing processing units. One or more processors can be utilized in supporting a virtualized computing environment. The virtualized computing environment may support one or more virtual machines representing computers, servers, or other computing devices. In such virtualized virtual machines, components such as processors and storage devices may be virtualized or logically represented. In an aspect, when a processor executes instructions to perform “operations”, this could include the processor performing the operations directly and/or facilitating, directing, or cooperating with another device or component to perform the operations
In the subject specification, terms such as “data store,” data storage,” “database,” “cache,” and substantially any other information storage component relevant to operation and functionality of a component, refer to “memory components,” or entities embodied in a “memory” or components comprising the memory. It is noted that the memory components, or computer-readable storage media, described herein can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. By way of illustration, and not limitation, nonvolatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in many forms such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM). Additionally, the disclosed memory components of systems or methods herein are intended to comprise, without being limited to comprising, these and any other suitable types of memory.
The illustrated aspects of the disclosure can be practiced in distributed computing environments where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote memory storage devices.
The systems and processes described above can be embodied within hardware, such as a single integrated circuit (IC) chip, multiple ICs, an application specific integrated circuit (ASIC), or the like. Further, the order in which some or all of the process blocks appear in each process should not be deemed limiting. Rather, it should be understood that some of the process blocks can be executed in a variety of orders that are not all of which may be explicitly illustrated herein.
As used in this application, the terms “component,” “module,” “system,” “interface,” “cluster,” “server,” “node,” or the like are generally intended to refer to a computer-related entity, either hardware, a combination of hardware and software, software, or software in execution or an entity related to an operational machine with one or more specific functionalities. For example, a component can be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, computer-executable instruction(s), a program, and/or a computer. By way of illustration, both an application running on a controller and the controller can be a component. One or more components may reside within a process and/or thread of execution and a component may be localized on one computer and/or distributed between two or more computers. As another example, an interface can include input/output (I/O) components as well as associated processor, application, and/or API components.
Furthermore, the terms “user,” “consumer,” “client,” and the like are employed interchangeably throughout the subject specification, unless context warrants particular distinction(s) among the terms. It is noted that such terms can refer to human entities or automated components/devices supported through artificial intelligence (e.g., a capacity to make inference based on complex mathematical formalisms), which can provide simulated vision, sound recognition and so forth.
Further, the various embodiments can be implemented as a method, apparatus, or article of manufacture using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof to control a computer to implement one or more aspects of the disclosed subject matter. An article of manufacture can encompass a computer program accessible from any computer-readable device or computer-readable storage/communications media. For example, computer readable storage media can include but are not limited to magnetic storage devices (e.g., hard disk, floppy disk, magnetic strips . . . ), optical disks (e.g., compact disk (CD), digital versatile disk (DVD) . . . ), smart cards, and flash memory devices (e.g., card, stick, key drive . . . ). Of course, those skilled in the art will recognize many modifications can be made to this configuration without departing from the scope or spirit of the various embodiments.
Artificial intelligence based systems, e.g., utilizing explicitly and/or implicitly trained classifiers, can be employed in connection with performing inference and/or probabilistic determinations and/or statistical-based determinations as in accordance with one or more aspects of the disclosed subject matter as described herein. For example, an artificial intelligence system can be used to dynamically perform operations as described herein.
A classifier can be a function that maps an input attribute vector, x=(x1, x2, x3, x4, xn), to a confidence that the input belongs to a class, that is, f(x)=confidence (class). Such classification can employ a probabilistic and/or statistical-based analysis (e.g., factoring into the analysis utilities and costs) to infer an action that a user desires to be automatically performed. In the case of communication systems, for example, attributes can be information received from access points, servers, components of a wireless communication network, etc., and the classes can be categories or areas of interest (e.g., levels of priorities). A support vector machine is an example of a classifier that can be employed. The support vector machine operates by finding a hypersurface in the space of possible inputs, which the hypersurface attempts to split the triggering criteria from the non-triggering events. Intuitively, this makes the classification correct for testing data that is near, but not identical to training data. Other directed and undirected model classification approaches include, e.g., naïve Bayes, Bayesian networks, decision trees, neural networks, fuzzy logic models, and probabilistic classification models providing different patterns of independence can be employed. Classification as used herein can also be inclusive of statistical regression that is utilized to develop models of priority.
In accordance with various aspects of the subject specification, artificial intelligence based systems, components, etc. can employ classifiers that are explicitly trained, e.g., via a generic training data, etc. as well as implicitly trained, e.g., via observing characteristics of communication equipment, e.g., a server, etc., receiving reports from such communication equipment, receiving operator preferences, receiving historical information, receiving extrinsic information, etc. For example, support vector machines can be configured via a learning or training phase within a classifier constructor and feature selection module. Thus, the classifier(s) can be used by an artificial intelligence system to automatically learn and perform a number of functions.
In addition, the word “example” or “exemplary” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs. Rather, use of the word exemplary is intended to present concepts in a concrete fashion. As used in this application, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless specified otherwise, or clear from context, “X employs A or B” is intended to mean any of the natural inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, then “X employs A or B” is satisfied under any of the foregoing instances. In addition, the articles “a” and “an” as used in this application and the appended claims should generally be construed to mean “one or more” unless specified otherwise or clear from context to be directed to a singular form.
What has been described above includes examples of the present specification. It is, of course, not possible to describe every conceivable combination of components or methods for purposes of describing the present specification, but one of ordinary skill in the art may recognize that many further combinations and permutations of the present specification are possible. Accordingly, the present specification is intended to embrace all such alterations, modifications and variations that fall within the spirit and scope of the appended claims. Furthermore, to the extent that the term “includes” is used in either the detailed description or the claims, such term is intended to be inclusive in a manner similar to the term “comprising” as “comprising” is interpreted when employed as a transitional word in a claim.
Number | Name | Date | Kind |
---|---|---|---|
4675869 | Driessen | Jun 1987 | A |
5675802 | Allen et al. | Oct 1997 | A |
5805788 | Johnson | Sep 1998 | A |
5950225 | Kleiman | Sep 1999 | A |
6065020 | Dussud | May 2000 | A |
6073218 | DeKoning et al. | Jun 2000 | A |
6108684 | DeKoning et al. | Aug 2000 | A |
6233696 | Kedem | May 2001 | B1 |
6240527 | Schneider et al. | May 2001 | B1 |
6502243 | Thomas | Dec 2002 | B1 |
6549921 | Ofek | Apr 2003 | B1 |
7007044 | Rafert et al. | Feb 2006 | B1 |
7103884 | Fellin et al. | Sep 2006 | B2 |
7389393 | Karr et al. | Jun 2008 | B1 |
7577091 | Antal et al. | Aug 2009 | B2 |
7631051 | Fein et al. | Dec 2009 | B1 |
7636814 | Karr et al. | Dec 2009 | B1 |
7664839 | Karr et al. | Feb 2010 | B1 |
7680875 | Shopiro et al. | Mar 2010 | B1 |
7694191 | Bono et al. | Apr 2010 | B1 |
7721044 | Chatterjee et al. | May 2010 | B1 |
7653792 | Shimada et al. | Jun 2010 | B2 |
7752403 | Weinman, Jr. | Jul 2010 | B1 |
7895394 | Nakajima et al. | Feb 2011 | B2 |
8037391 | Jung et al. | Oct 2011 | B1 |
8125406 | Jensen et al. | Feb 2012 | B1 |
8261033 | Slik et al. | Sep 2012 | B1 |
8370542 | Lu et al. | Feb 2013 | B2 |
8429514 | Goel | Apr 2013 | B1 |
8479037 | Chatterjee et al. | Jul 2013 | B1 |
8495465 | Anholt et al. | Jul 2013 | B1 |
8504518 | Ghemawat et al. | Aug 2013 | B1 |
8540625 | Miyoshi | Sep 2013 | B2 |
8683205 | Resch et al. | Mar 2014 | B2 |
8725986 | Goel | May 2014 | B1 |
8751599 | Tran et al. | Jun 2014 | B2 |
8751740 | De Forest et al. | Jun 2014 | B1 |
8751897 | Borthakur et al. | Jun 2014 | B2 |
8799746 | Baker et al. | Aug 2014 | B2 |
8832234 | Brooker et al. | Sep 2014 | B1 |
8856619 | Cypher | Oct 2014 | B1 |
8856624 | Paniconi | Oct 2014 | B1 |
8892938 | Sundaram et al. | Nov 2014 | B1 |
8972478 | Storer et al. | Mar 2015 | B1 |
9003086 | Schuller et al. | Apr 2015 | B1 |
9021296 | Kiselev et al. | Apr 2015 | B1 |
9037825 | Donlan et al. | May 2015 | B1 |
9052942 | Barber et al. | Jun 2015 | B1 |
9063838 | Boyle et al. | Jun 2015 | B1 |
9098447 | Donlan et al. | Aug 2015 | B1 |
9128910 | Dayal et al. | Sep 2015 | B1 |
9208009 | Resch et al. | Dec 2015 | B2 |
9218135 | Miller et al. | Dec 2015 | B2 |
9244761 | Yekhanin et al. | Jan 2016 | B2 |
9268783 | Shilane et al. | Feb 2016 | B1 |
9274903 | Garlapati et al. | Mar 2016 | B1 |
9280430 | Sarfare et al. | Mar 2016 | B2 |
9405483 | Wei et al. | Aug 2016 | B1 |
9411717 | Goss et al. | Aug 2016 | B2 |
9442802 | Hung | Sep 2016 | B2 |
9477682 | Bent et al. | Oct 2016 | B1 |
9495241 | Flynn et al. | Nov 2016 | B2 |
9619256 | Natanzon et al. | Apr 2017 | B1 |
9641615 | Robins et al. | May 2017 | B1 |
9665428 | Vairavanathan et al. | May 2017 | B2 |
9747057 | Ramani et al. | Aug 2017 | B1 |
9817713 | Gupta et al. | Nov 2017 | B2 |
9864527 | Srivastav et al. | Jan 2018 | B1 |
9942084 | Sorenson, III | Apr 2018 | B1 |
9971649 | Dhuse et al. | May 2018 | B2 |
10001947 | Chatterjee et al. | Jun 2018 | B1 |
10007561 | Pudipeddi et al. | Jun 2018 | B1 |
10055145 | Danilov et al. | Aug 2018 | B1 |
10061668 | Lazier et al. | Aug 2018 | B1 |
10089026 | Puhov et al. | Oct 2018 | B1 |
10097659 | Rao | Oct 2018 | B1 |
10108819 | Donlan et al. | Oct 2018 | B1 |
10127234 | Krishnan et al. | Nov 2018 | B1 |
10216770 | Kulesza et al. | Feb 2019 | B1 |
10242022 | Jain et al. | Mar 2019 | B1 |
10282262 | Panara et al. | May 2019 | B2 |
10289488 | Danilov et al. | May 2019 | B1 |
10331516 | Danilov et al. | Jun 2019 | B2 |
10361810 | Myung et al. | Jul 2019 | B2 |
10387546 | Duran et al. | Aug 2019 | B1 |
10496330 | Bernat et al. | Dec 2019 | B1 |
10503611 | Srivastav et al. | Dec 2019 | B1 |
10567009 | Yang et al. | Feb 2020 | B2 |
10579490 | Danilov et al. | Mar 2020 | B2 |
10613780 | Naeni et al. | Apr 2020 | B1 |
10628043 | Chatterjee et al. | Apr 2020 | B1 |
10644408 | Sakai et al. | May 2020 | B2 |
10671431 | Dolan et al. | Jun 2020 | B1 |
10705911 | Vishnumolakala et al. | Jul 2020 | B2 |
10733053 | Miller et al. | Aug 2020 | B1 |
10740183 | Blaum et al. | Aug 2020 | B1 |
10754845 | Danilov et al. | Aug 2020 | B2 |
10761931 | Goyal et al. | Sep 2020 | B2 |
10797863 | Chen et al. | Oct 2020 | B2 |
10846003 | Danilov et al. | Nov 2020 | B2 |
10924543 | Chen | Feb 2021 | B1 |
10951236 | Chen et al. | Mar 2021 | B2 |
11023331 | Danilov et al. | Jun 2021 | B2 |
11150995 | Dhoolam | Oct 2021 | B1 |
20020049883 | Schneider et al. | Apr 2002 | A1 |
20020166026 | Ulrich et al. | Nov 2002 | A1 |
20020191311 | Ulrich et al. | Dec 2002 | A1 |
20030016596 | Chiquoine et al. | Jan 2003 | A1 |
20050027938 | Burkey | Feb 2005 | A1 |
20050050131 | Lawrow | Mar 2005 | A1 |
20050071546 | Delaney et al. | Mar 2005 | A1 |
20050080982 | Vasilevsky et al. | Apr 2005 | A1 |
20050088318 | Liu et al. | Apr 2005 | A1 |
20050108775 | Bachar et al. | May 2005 | A1 |
20050140529 | Choi et al. | Jun 2005 | A1 |
20050234941 | Watanabe | Oct 2005 | A1 |
20060047896 | Nguyen et al. | Mar 2006 | A1 |
20060075007 | Anderson et al. | Apr 2006 | A1 |
20060143508 | Mochizuki et al. | Jun 2006 | A1 |
20060212744 | Benner et al. | Sep 2006 | A1 |
20060265211 | Canniff et al. | Nov 2006 | A1 |
20070076321 | Takahashi et al. | Apr 2007 | A1 |
20070239759 | Shen et al. | Oct 2007 | A1 |
20070250674 | Findberg et al. | Oct 2007 | A1 |
20080222480 | Huang et al. | Sep 2008 | A1 |
20080222481 | Huang et al. | Sep 2008 | A1 |
20080228828 | Teodorescu | Sep 2008 | A1 |
20080244353 | Dholakia et al. | Oct 2008 | A1 |
20080320061 | Aszmann et al. | Dec 2008 | A1 |
20090070771 | Yuyitung et al. | Mar 2009 | A1 |
20090113034 | Krishnappa et al. | Apr 2009 | A1 |
20090132543 | Chatley et al. | May 2009 | A1 |
20090172464 | Byrne et al. | Jul 2009 | A1 |
20090183056 | Aston | Jul 2009 | A1 |
20090204959 | Anand et al. | Aug 2009 | A1 |
20090240880 | Kawaguchi | Sep 2009 | A1 |
20090259882 | Shellhamer | Oct 2009 | A1 |
20100031060 | Chew et al. | Feb 2010 | A1 |
20100094963 | Zuckerman et al. | Apr 2010 | A1 |
20100174968 | Charles et al. | Jul 2010 | A1 |
20100218037 | Swartz et al. | Aug 2010 | A1 |
20100293348 | Ye et al. | Nov 2010 | A1 |
20100332748 | Van der Goot et al. | Dec 2010 | A1 |
20110029836 | Dhuse et al. | Feb 2011 | A1 |
20110040937 | Augenstein | Feb 2011 | A1 |
20110066882 | Walls et al. | Mar 2011 | A1 |
20110106972 | Grube et al. | May 2011 | A1 |
20110107165 | Resch et al. | May 2011 | A1 |
20110138148 | Friedman et al. | Jun 2011 | A1 |
20110161712 | Athalye et al. | Jun 2011 | A1 |
20110191536 | Mizuno et al. | Aug 2011 | A1 |
20110196833 | Drobychev et al. | Aug 2011 | A1 |
20110246503 | Bender et al. | Oct 2011 | A1 |
20110292054 | Boker et al. | Dec 2011 | A1 |
20120023291 | Zeng et al. | Jan 2012 | A1 |
20120096214 | Lu et al. | Apr 2012 | A1 |
20120191675 | Kim et al. | Jul 2012 | A1 |
20120191901 | Norair | Jul 2012 | A1 |
20120204077 | D'Abreu et al. | Aug 2012 | A1 |
20120233117 | Holt et al. | Sep 2012 | A1 |
20120311395 | Leggette et al. | Dec 2012 | A1 |
20120317234 | Bohrer et al. | Dec 2012 | A1 |
20120321052 | Morrill et al. | Dec 2012 | A1 |
20130013564 | Ben-Or et al. | Jan 2013 | A1 |
20130047187 | Frazier et al. | Feb 2013 | A1 |
20130054822 | Mordani et al. | Feb 2013 | A1 |
20130067159 | Mehra | Mar 2013 | A1 |
20130067187 | Moss et al. | Mar 2013 | A1 |
20130088501 | Fell | Apr 2013 | A1 |
20130097470 | Hwang et al. | Apr 2013 | A1 |
20130145208 | Yen et al. | Jun 2013 | A1 |
20130238932 | Resch | Sep 2013 | A1 |
20130246876 | Manssour et al. | Sep 2013 | A1 |
20130290482 | Leggette | Oct 2013 | A1 |
20130305365 | Rubin et al. | Nov 2013 | A1 |
20140006850 | Aliev et al. | Jan 2014 | A1 |
20140040417 | Galdwin et al. | Feb 2014 | A1 |
20140064048 | Cohen et al. | Mar 2014 | A1 |
20140082414 | Olster | Mar 2014 | A1 |
20140115182 | Sabaa et al. | Apr 2014 | A1 |
20140122745 | Singh et al. | May 2014 | A1 |
20140149794 | Shetty et al. | May 2014 | A1 |
20140164430 | Hadjieleftheriou et al. | Jun 2014 | A1 |
20140164694 | Storer | Jun 2014 | A1 |
20140172930 | Molaro et al. | Jun 2014 | A1 |
20140250450 | Yu et al. | Sep 2014 | A1 |
20140280375 | Rawson et al. | Sep 2014 | A1 |
20140281804 | Resch | Sep 2014 | A1 |
20140297955 | Yamazaki et al. | Oct 2014 | A1 |
20140304460 | Carlson, Jr. et al. | Oct 2014 | A1 |
20140331100 | Dhuse et al. | Nov 2014 | A1 |
20140351633 | Grube et al. | Nov 2014 | A1 |
20140358972 | Guarrier et al. | Dec 2014 | A1 |
20140359244 | Chambliss et al. | Dec 2014 | A1 |
20140380088 | Bennett et al. | Dec 2014 | A1 |
20140380093 | Molaro | Dec 2014 | A1 |
20140380125 | Calder et al. | Dec 2014 | A1 |
20140380126 | Yekhanin et al. | Dec 2014 | A1 |
20150006846 | Youngworth | Jan 2015 | A1 |
20150074065 | Christ et al. | Mar 2015 | A1 |
20150112951 | Narayanamurthy et al. | Apr 2015 | A1 |
20150134626 | Theimer et al. | May 2015 | A1 |
20150142863 | Yuen et al. | May 2015 | A1 |
20150160872 | Chen | Jun 2015 | A1 |
20150178007 | Moisa et al. | Jun 2015 | A1 |
20150186043 | Kesselman et al. | Jul 2015 | A1 |
20150254150 | Gordon et al. | Sep 2015 | A1 |
20150269025 | Krishnamurthy et al. | Sep 2015 | A1 |
20150303949 | Jafarkhani et al. | Oct 2015 | A1 |
20150331766 | Sarfare et al. | Nov 2015 | A1 |
20150370656 | Tsafrir et al. | Dec 2015 | A1 |
20150378542 | Saito et al. | Dec 2015 | A1 |
20160011935 | Luby | Jan 2016 | A1 |
20160011936 | Luby | Jan 2016 | A1 |
20160055054 | Patterson, III et al. | Feb 2016 | A1 |
20160085645 | Buzzard et al. | Mar 2016 | A1 |
20160162378 | Garlapati et al. | Jun 2016 | A1 |
20160169692 | Gupta | Jun 2016 | A1 |
20160170668 | Mehra | Jun 2016 | A1 |
20160217104 | Kamble et al. | Jul 2016 | A1 |
20160232055 | Vairavanathan et al. | Aug 2016 | A1 |
20160239384 | Slik | Aug 2016 | A1 |
20160253400 | McAlister et al. | Sep 2016 | A1 |
20160277497 | Bannister et al. | Sep 2016 | A1 |
20160292429 | Bannister et al. | Sep 2016 | A1 |
20160294419 | Sandell et al. | Oct 2016 | A1 |
20160328295 | Baptist et al. | Nov 2016 | A1 |
20160357443 | Li et al. | Dec 2016 | A1 |
20160357649 | Karrotu et al. | Dec 2016 | A1 |
20160371145 | Akutsu et al. | Dec 2016 | A1 |
20160378624 | Jenkins, Jr. et al. | Dec 2016 | A1 |
20160380650 | Calder et al. | Dec 2016 | A1 |
20170003880 | Fisher et al. | Jan 2017 | A1 |
20170004044 | Tormasov et al. | Jan 2017 | A1 |
20170010944 | Saito et al. | Jan 2017 | A1 |
20170017671 | Baptist et al. | Jan 2017 | A1 |
20170031945 | Sarab et al. | Feb 2017 | A1 |
20170097875 | Jess et al. | Apr 2017 | A1 |
20170102993 | Hu et al. | Apr 2017 | A1 |
20170115903 | Franke et al. | Apr 2017 | A1 |
20170116088 | Anami et al. | Apr 2017 | A1 |
20170123914 | Li et al. | May 2017 | A1 |
20170153946 | Baptist et al. | Jun 2017 | A1 |
20170185331 | Gao et al. | Jun 2017 | A1 |
20170187398 | Trusov | Jun 2017 | A1 |
20170187766 | Zheng et al. | Jun 2017 | A1 |
20170206025 | Viswanathan | Jul 2017 | A1 |
20170206135 | Zeng | Jul 2017 | A1 |
20170212680 | Waghulde | Jul 2017 | A1 |
20170212845 | Conway | Jul 2017 | A1 |
20170220662 | Barton et al. | Aug 2017 | A1 |
20170235507 | Sinha et al. | Aug 2017 | A1 |
20170262187 | Manzanares et al. | Sep 2017 | A1 |
20170268900 | Nicolaas et al. | Sep 2017 | A1 |
20170272209 | Yanovsky et al. | Sep 2017 | A1 |
20170285952 | Danilov et al. | Oct 2017 | A1 |
20170286009 | Danilov et al. | Oct 2017 | A1 |
20170286436 | Neporada et al. | Oct 2017 | A1 |
20170286516 | Horowitz et al. | Oct 2017 | A1 |
20170288701 | Slik et al. | Oct 2017 | A1 |
20170344285 | Choi et al. | Nov 2017 | A1 |
20180032279 | Davis | Feb 2018 | A1 |
20180052744 | Chen et al. | Feb 2018 | A1 |
20180063213 | Bevilacqua-Linn et al. | Mar 2018 | A1 |
20180074753 | Ober | Mar 2018 | A1 |
20180074881 | Burden | Mar 2018 | A1 |
20180088857 | Gao et al. | Mar 2018 | A1 |
20180107415 | Motwani et al. | Apr 2018 | A1 |
20180121286 | Sipos | May 2018 | A1 |
20180129417 | Sivasubramanian et al. | May 2018 | A1 |
20180129600 | Ishiyama et al. | May 2018 | A1 |
20180181324 | Daniliv et al. | Jun 2018 | A1 |
20180181475 | Danilov et al. | Jun 2018 | A1 |
20180181612 | Daniliv et al. | Jun 2018 | A1 |
20180217888 | Colgrove | Aug 2018 | A1 |
20180246668 | Sakashita et al. | Aug 2018 | A1 |
20180267856 | Hayasaka et al. | Sep 2018 | A1 |
20180267985 | Badey et al. | Sep 2018 | A1 |
20180293017 | Curley | Oct 2018 | A1 |
20180306600 | Nicolaas et al. | Oct 2018 | A1 |
20180307560 | Vishnumolakala et al. | Oct 2018 | A1 |
20180341662 | He | Nov 2018 | A1 |
20180375936 | Chirammal et al. | Dec 2018 | A1 |
20190028179 | Kalhan | Jan 2019 | A1 |
20190034084 | Nagarajan et al. | Jan 2019 | A1 |
20190043201 | Strong et al. | Feb 2019 | A1 |
20190043351 | Yang et al. | Feb 2019 | A1 |
20190050301 | Juniwal et al. | Feb 2019 | A1 |
20190065092 | Shah et al. | Feb 2019 | A1 |
20190065310 | Rozas | Feb 2019 | A1 |
20190102103 | Ari et al. | Apr 2019 | A1 |
20190114223 | Pydipaty et al. | Apr 2019 | A1 |
20190129644 | Gao et al. | May 2019 | A1 |
20190188079 | Kohli | Jun 2019 | A1 |
20190205437 | Larson et al. | Jul 2019 | A1 |
20190215017 | Danilov et al. | Jul 2019 | A1 |
20190220207 | Lingarajappa | Jul 2019 | A1 |
20190266062 | Borlick et al. | Aug 2019 | A1 |
20190342418 | Eda et al. | Nov 2019 | A1 |
20190356416 | Yanovsky et al. | Nov 2019 | A1 |
20190384500 | Danilov et al. | Dec 2019 | A1 |
20190386683 | Danilov et al. | Dec 2019 | A1 |
20200004447 | Danilov et al. | Jan 2020 | A1 |
20200026810 | Subramaniam et al. | Jan 2020 | A1 |
20200034339 | Gershaneck et al. | Jan 2020 | A1 |
20200034471 | Danilov et al. | Jan 2020 | A1 |
20200042178 | Danilov et al. | Feb 2020 | A1 |
20200050510 | Chien et al. | Feb 2020 | A1 |
20200081778 | Varanasi | Mar 2020 | A1 |
20200104377 | Earnesty, Jr. et al. | Apr 2020 | A1 |
20200117547 | Danilov et al. | Apr 2020 | A1 |
20200117556 | Zou et al. | Apr 2020 | A1 |
20200145511 | Gray et al. | May 2020 | A1 |
20200151353 | Struttmann | May 2020 | A1 |
20200204198 | Danilov et al. | Jun 2020 | A1 |
20200218450 | Danilov et al. | Jul 2020 | A1 |
20200241759 | Danilov et al. | Jul 2020 | A1 |
20200382141 | Wang et al. | Dec 2020 | A1 |
20210019067 | Miller et al. | Jan 2021 | A1 |
20210019093 | Karr et al. | Jan 2021 | A1 |
20210019237 | Karr et al. | Jan 2021 | A1 |
20210034268 | Hara et al. | Feb 2021 | A1 |
20210096754 | Danilov et al. | Apr 2021 | A1 |
20210132851 | Danilov et al. | May 2021 | A1 |
20210133049 | Danilov et al. | May 2021 | A1 |
20210218420 | Danilov et al. | Jul 2021 | A1 |
20210255791 | Shimada et al. | Aug 2021 | A1 |
20210273660 | Danilov et al. | Sep 2021 | A1 |
Entry |
---|
Non-Final Office Action received for U.S. Appl. No. 16/399,897 dated Feb. 19, 2021, 56 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/670,715 dated Mar. 31, 2021, 60 pages. |
Final Office Action received for U.S. Appl. No. 16/177,278 dated Feb. 24, 2021, 109 pages. |
EMC; “EMC ECS (Elastic Cloud Storage) Architectural Guide v2.x;” EMC; Jun. 2015; available at: https://www.dell.com/community/s/vjauj58549/attachments/vjauj58549/solutions-ch/477 /1/h14071-ecs-architectural-guide-wp.pdf, Jun. 2015, 21 pages. |
Mohan et al., “Geo-aware erasure coding for high-performance erasure-coded storage clusters”, Springer Link, URL: https://link.springer.com/article/10.1007/s 12243-017-0623-2, Jan. 18, 2018. |
Final Office Action received for U.S. Appl. No. 16/179,486 dated Jan. 28, 2021, 55 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/670,746 dated Feb. 16, 2021, 55 pages. |
Dell Technologies, “ECS Overview and Architecture”, h14071.18, Feb. 2021, 21 Pages. |
Non-Final Office Action received for U.S. Appl. No. 16/177,285 dated Apr. 9, 2021, 41 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/779,208 dated Apr. 20, 2021, 71 pages. |
Notice of Allowance received for U.S. Appl. No. 16/570,657 dated Sep. 7, 2021, 65 pages. |
Ma et al., “An Ensemble of Replication and Erasure Codes for Cloud File Systems”, Proceedings—IEEE INFOCOM, Apr. 2013, pp. 1276-1284. |
Final Office Action received for U.S. Appl. No. 16/698,096 dated Sep. 7, 2021, 24 pages. |
Final Office Action received for U.S. Appl. No. 16/177,285 dated Sep. 14, 2021, 65 pages. |
Final Office Action received for U.S. Appl. No. 16/670,715 dated Sep. 7, 2021, 35 pages. |
Final Office Action received for U.S. Appl. No. 16/179,486 dated Oct. 20, 2021, 46 pages. |
Non-Final Office Action received for U.S. Appl. No. 15/651,504 dated Mar. 21, 2019, 10 pages. |
Non-Final Office Action received for U.S. Appl. No. 15/662,273 dated Nov. 16, 2018, 19 pages. |
Final Office Action received for U.S. Appl. No. 15/662,273 dated May 15, 2019, 33 pages. |
Non-Final Office Action received for U.S. Appl. No. 15/965,479 dated Apr. 15, 2019, 21 pages. |
Final Office Action dated Feb. 12, 2020 for U.S. Appl. No. 16/024,314, 29 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/010,255 dated Jan. 9, 2020, 31 pages. |
Office Action dated Feb. 5, 2020 for U.S. Appl. No. 16/261,551, 30 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/228,612 dated Feb. 27, 2020, 49 pages. |
Final Office Action received for U.S. Appl. No. 16/010,246 dated Mar. 16, 2020, 33 pages. |
Final Office Action received for U.S. Appl. No. 15/656,382 dated Apr. 6, 2020, 31 pages. |
Notice of Allowance received for U.S. Appl. No. 16/240,193, dated May 4, 2020, 46 pages. |
Final Office Action received for U.S. Appl. No. 16/177,278, dated May 11, 2020, 53 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/231,018 dated May 8, 2020, 78 pages. |
Notice of Allowance dated May 11, 2020 for U.S. Appl. No. 16/240,193, 24 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/228,624 dated Jun. 24, 2020, 65 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/240,272 dated Jun. 29, 2020, 64 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/228,612 dated Jun. 29, 2020, 62 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/526,182 dated Nov. 24, 2021, 83 pages. |
Notice of Allowance received for U.S. Appl. No. 16/888,144 dated Nov. 22, 2021, 71 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/209,185 dated Jun. 18, 2020, 22 pages. |
Martin Hosken, Developing a Hyper-Converged Storage Strategy for VMware vCloud Director with VMware vSAN, Jan. 2018 (Year: 2018). |
Non-Final Office Action received for U.S. Appl. No. 16/261,549 dated Apr. 15, 2020, 22 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/374,726 dated Jun. 2, 2020, 47 pages. |
Natarajan, RAID 0, RAID 1, RAID 5, RAID 10 Explained with Diagrams, Aug. 10, 2010, thegeekstuff.com (18 pages). |
Non-Final Office Action received for U.S. Appl. No. 16/177,285 dated Jul. 22, 2020, 31 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/261,547 dated Sep. 3, 2020, 26 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/261,548 dated Aug. 21, 2020, 42 pages. |
Notice of Allowance received for U.S. Appl. No. 16/261,549 dated Jul. 17, 2020, 40 pages. |
Qiang et al., “Dynamics Process of Long-running Allocation/Collection in Linear Storage Space”, International Conference on Networking, Architecture, and Storage (NAS 2007), Guilin, 2007, pp. 209-216. |
Non-Final Office Action received for U.S. Appl. No. 16/374,725 dated Aug. 19, 2020, 50 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/511,161 dated Jul. 10, 2020, 24 pages. |
Notice of Allowance received for U.S. Appl. No. 15/862,547 dated Mar. 29, 2019 27 pages. |
Non-Final Office Action received for U.S. Appl. No. 15/792,714 dated Apr. 4, 2019, 20 pages. |
Final Office Action received for U.S. Appl. No. 15/792,714 dated Sep. 12, 2019, 43 pages. |
Wikipedia “Garbage Collection”, URL: https://en.wikipedia.org/wiki/Garbage_collection_(computer science) #Availability (Year: 2017) retrieved using the WayBackMachine, Sep. 8, 2017, 8 pages. |
Wikipedia “Erasure code”, URL: https://web.archive.org/web/20170908171158/https://en.wikipedia.org/wiki/Erasure_code (Year: 2017), retrieved using the WayBackMachine, Sep. 8, 2017, 5 pages. |
Wikipedia “Front and back ends” URL: https://en.wikipedia.org/wiki/Front_and_back_ends (Year:2019), Sep. 6, 2019, 4 pages. |
Notice of Allowance received for U.S. Appl. No. 15/792,714 dated Nov. 8, 2019, 31 pages. |
Non-Final Office Action received for U.S. Appl. No. 15/791,390 dated Sep. 20, 2019, 27 pages. |
Final Office Action received for U.S. Appl. No. 15/791,390 dated Feb. 6, 2020, 29 pages. |
Non-Final Office Action received for U.S. Appl. No. 15/791,390 dated Apr. 30, 2020, 48 pages. |
Huang et al., “Scale-RS: An Efficient Scaling Scheme for RS-Coded Storage Clusters,” in IEEE Transactions on Parallel and Distributed Systems, vol. 26, No. 6, pp. 1704-1717, Jun. 1, 2015. |
Non-Final Office Action received for U.S. Appl. No. 16/457,615 dated Jul. 20, 2020, 34 pages. |
Final Office Action received for U.S. Appl. No. 16/010,255 dated Jul. 23, 2020, 36 pages. |
Office Action received for U.S. Appl. No. 16/010,246 dated Jul. 27, 2020, 36 pages. |
Office Action received for U.S. Appl. No. 16/177,278, dated Aug. 21, 2020, 53 pages. |
Office Action received for U.S. Appl. No. 16/179,486, dated Aug. 13, 2020, 64 pages. |
Guo et al., “GeoScale: Providing Geo-Elasticity in Distributed Clouds” 2016 IEEE International Conference on Cloud Engineering, 4 pages. |
Guo et al., “Providing Geo-Elasticity in Geographically Distributed Clouds”. ACM Transactions on Internet Technology, vol. 18, No. 3, Article 38. Apr. 2018. 27 pages. |
Office Action received for U.S. Appl. No. 16/254,073, dated Aug. 18, 2020, 62 pages. |
Wikipedia, “Standard Raid Levels—RAID 6”, URL: https://en.wikipedia.org/wiki/Standard_RAID_levels#RAID_6, Oct. 18, 2019, 11 pages. |
Non-Final Office Action received for U.S. Appl. No. 15/656,382 dated Nov. 1, 2019, 47 pages. |
Final Office Action received for U.S. Appl. No. 15/952,179 dated Nov. 26, 2019, 53 pages. |
Non Final Office Action received for U.S. Appl. No. 16/024,314 dated Nov. 25, 2019, 42 pages. |
Non- Final Office Action received for U.S. Appl. No. 16/177, 278 dated Dec. 2, 2019, 55 pages. |
Non-Final Office Action received for U.S. Appl. No. 15/651,504 dated Dec. 31, 2019, 18 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/010,246 dated Dec. 5, 2019, 67 pages. |
Stonebreaker et al. “Distributed RAID—A New Multiple Copy Algorithm.”, IEEE ICDE, 1990, pp. 430-437. |
Muralidhar et al. “f4: Facebook's Warm BLOB Storage System”, USENIX. OSDI, Oct. 2014, pp. 383-398. |
Non-Final Office Action received for U.S. Appl. No. 16/526,142 dated Oct. 15, 2020, 21 pages. |
Notice of Allowance received U.S. Appl. No. 16/228,612 dated Oct. 20, 2020, 84 pages. |
Zhou, et al. “Fast Erasure Coding for Data Storage: A Comprehensive Study of the Acceleration Techniques” Proceedings of the 17th Usenix Conference on File and Storage Technologies (FAST '19), [https://www.usenix.org/conference/fast19/presentation/zhou], Feb. 2019, Boston, MA, USA. 14 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/010,255 dated Oct. 29, 2020, 65 pages. |
Final Office Action received for U.S. Appl. No. 16/240,272 dated Oct. 27, 2020, 42 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/399,902 dated Oct. 28, 2020, 83 pages. |
Notice of Allowance received for U.S. Appl. No. 16/374,726 dated Nov. 20, 2020, 78 pages. |
Final Office Action received for U.S. Appl. No. 16/228,624 dated Dec. 1, 2020, 63 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/570,657 dated Nov. 27, 2020, 75 pages. |
Final Office Action received for U.S. Appl. No. 16/177,285 dated Dec. 30, 2020, 61 pages. |
Final Office Action received for U.S. Appl. No. 16/511,161 dated Dec. 30, 2020, 61 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/399,895 dated Jan. 4, 2021, 64 pages. |
Notice of Allowance received for U.S. Appl. No. 16/374,726 dated Jan. 6, 2021, 56 pages. |
Non-Final Office Action received for U.S. Appl. No. 15/794,950 dated Jul. 9, 2019, 29 pages. |
Final Office Action received for U.S. Appl. No. 15/651,504 dated Sep. 18, 2019, 15 pages. |
Non-Final Office Action received for U.S. Appl. No. 15/952,179 dated Sep. 10, 2019, 42 pages. |
Notice of Allowance received for U.S. Appl. No. 16/726,428 dated Jun. 14, 2021, 34 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/698,096 dated May 24, 2021, 62 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/745,855 dated May 13, 2021, 71 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/834,649 dated Jun. 24, 2021, 61 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/179,486 dated May 12, 2021, 50 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/570,657 dated May 12, 2021, 79 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/670,765 dated Jul. 20, 2021, 79 pages. |
Thomasian et al., “Hierarchical RAID: Design, performance, reliability, and recovery”, J. Parallel Distrib. Comput. vol. 72 (2012) pp. 1753-1769. |
Non-Final Office Action received for U.S. Appl. No. 15/582,167 dated Sep. 7, 2018, 19 pages. |
Non-Final Office Action received for U.S. Appl. No. 15/952,179 dated Apr. 20, 2020, 68 pages. |
Notice of Allowance received for U.S. Appl. No. 16/584,800 dated Jun. 27, 2022, 33 pages. |
Notice of Allowance received for U.S. Appl. No. 16/179,486 dated Jun. 8, 2022, 67 pages. |
Final Office Action received for U.S. Appl. No. 16/986,222 dated Jun. 17, 2022, 76 pages. |
Final Office Action received for U.S. Appl. No. 17/153,602 dated Jul. 14, 2022, 34 pages. |
Final Office Action received for U.S. Appl. No. 16/538,984 dated Jun. 1, 2022, 114 pages. |
Notice of Allowance received for U.S. Appl. No. 17/333,815 dated Jun. 27, 2022, 27 pages. |
Wu et al., “Improving I/O Performance of Clustered Storage Systems by Adaptive Request Distribution,” 2006 15th IEEE International Conference on High Performance Distributed Computing, 2006, pp. 207-217. |
Notice of Allowance received for U.S. Appl. No. 16/526,182 dated May 16, 2022, 54 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/584,800 dated Mar. 3, 2022, 90 pages. |
Notice of Allowance received for U.S. Appl. No. 17/333,793 dated Mar. 9, 2022, 39 pages. |
Non-Final Office Action received for U.S. Appl. No. 17/153,602 dated Mar. 16, 2022, 40 pages. |
Sun et al., “Data Management across Geographically-Distributed Autonomous Systems: Architecture, Implementation, and Performance Evaluation,” IEEE Transactions on Industrial Informatics, 2019, 9 pages. |
Office Action dated Sep. 3, 2021 for U.S. Appl. No. 16/803,913, 23 pages. |
Office Action dated Jan. 25, 2022 for U.S. Appl. No. 16/803,913, 25 pages. |
Office Action dated May 27, 2022 for U.S. Appl. No. 16/803,913, 24 pages. |
RAID vs. non-RAID Storage—Difference & Comparison, https://www.fromdev.com/2014/01/raid-vs-non-raid-storage-difference.html, pp. 1-4, 2014. (Year: 2014). |
Office Action dated Apr. 13, 2021 for U.S. Appl. No. 16/781,316, 21 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/670,715 dated Jan. 5, 2022, 22 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/834,649 dated Jan. 28, 2022, 26 pages. |
Office Action dated Feb. 8, 2022 for U.S. Appl. No. 16/986,222, 62 pages. |
Office Action dated Nov. 24, 2021 for U.S. Appl. No. 16/538,984, 30 pages. |
Non-Final Office Action received for U.S. Appl. No. 16/698,096 dated Jan. 5, 2022, 21 pages. |
Notice of Allowance received for U.S. Appl. No. 16/745,855 dated Sep. 10, 2021, 37 pages. |
Non- Final Office Action received for U.S. Appl. No. 17/153,602 dated Oct. 25, 2022, 40 pages. |
Notice of Allowance received for U.S. Appl. No. 16/986,222 dated Oct. 13, 2022, 37 pages. |
Final Office Action received for U.S. Appl. No. 16/803,913 dated Oct. 26, 2022, 102 pages. |
Files Controlling User Accounts and Groups, https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/4, 2012, pp. 1-2. (Year: 2012). |
Notice of Allowance dated Feb. 8, 2023 for U.S. Appl. No. 17/153,602, 27 pages. |
Notice of Allowance dated Mar. 6, 2023 for U.S. Appl. No. 17/083,135, 111 pages. |
Office Action dated Mar. 31, 2023 for U.S. Appl. No. 17/152,537, 95 pages. |
Number | Date | Country | |
---|---|---|---|
20200348865 A1 | Nov 2020 | US |