Patent Application
20200034474
Maintaining synchronized snap set pairs (also referred to as “identical snap set pairs) between a source system and a target system is useful in remote replication environments. The identical snap set pairs can be used for data verification, fast recovery after replications session termination or disaster, or efficient synchronized restore/rollback operations between the source and target.
Identical snap set pairs are easy to create in asynchronous snap-based replication. Since the read only snap sets are replicated to the target in each replication cycle, at the end of each cycle, the result is the same snap set is stored on the target as on the source. In sync replication, however, it is more challenging as data gets replicated constantly from the source consistency group to the target consistency group. At any given time, there are always IOs inflight that might make the source system and target system different. Conventionally, due to constant inflight IO changes between the source and target in sync replication, to create synchronized snap set one has to suspend and drain source host IOs to safely create a synchronized snap set pair.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described herein in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
One aspect may provide a method for creating identical snap pairs in synchronous replication environment. The method includes creating a snap set (S-base) on a source site, marking, in a journal, valid sync replication IO journal entries at time of snap set creation, and tracking journal entries. Upon determining all marked sync replication IO journal entries are removed from the journal indicating completion of inflight IOs, the method further includes creating a snap set (S-base′) on the target site, creating a local snap set Sn against the source and a remote snap set against the S-base, transferring a data difference between Sn and S-base to the target site, and writing the difference to Sn′ on the target site.
Another aspect may provide a system for creating identical snap pairs in synchronous replication environment. The system includes a memory having computer-executable instructions. The system also includes a processor operated by a storage system. The processor executes the computer-executable instructions. When executed by the processor, the computer-executable instructions cause the processor to perform operations. The operations include creating a snap set (S-base) on a source site, marking, in a journal, valid sync replication IO journal entries at time of snap set creation, and tracking journal entries. Upon determining all marked sync replication IO journal entries are removed from the journal indicating completion of inflight IOs, the operations further include creating a snap set (S-base′) on the target site, creating a local snap set Sn against the source and a remote snap set against the S-base, transferring a data difference between Sn and S-base to the target site, and writing the difference to Sn′ on the target site.
Another aspect may provide a computer program product embodied on a non-transitory computer readable medium. The computer program product includes instructions that, when executed by a computer at a storage system, causes the computer to perform operations. The operations include creating a snap set (S-base) on a source site, marking, in a journal, valid sync replication IO journal entries at time of snap set creation, and tracking journal entries. Upon determining all marked sync replication IO journal entries are removed from the journal indicating completion of inflight IOs, an aspect further includes creating a snap set (S-base′) on the target site, creating a local snap set Sn against the source and a remote snap set against the S-base, transferring a data difference between Sn and S-base to the target site, and writing the difference to Sn′ on the target site.
Objects, aspects, features, and advantages of embodiments disclosed herein will become more fully apparent from the following detailed description, the appended claims, and the accompanying drawings in which like reference numerals identify similar or identical elements. Reference numerals that are introduced in the specification in association with a drawing figure may be repeated in one or more subsequent figures without additional description in the specification in order to provide context for other features. For clarity, not every element may be labeled in every figure. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating embodiments, principles, and concepts. The drawings are not meant to limit the scope of the claims included herewith.
Embodiments described herein provide a way to create identical snap set pairs in a synchronous replication environment of a storage system. The identical snap set pair creation process provides a way to create synchronized identical snap set pairs on a source and target system with minimum interruption of both ongoing host IO operations and sync replication IO activities, and without the need to suspend and drain IO before snap set creation.
Turning now to
Source site 102 may include a host 104, storage application 106, and data storage 108. In some embodiments, storage 108 may include one or more storage volumes (not shown), that operate as active or production volumes.
Host 104 may perform I/O operations on storage 108 (e.g., read data from and write data to storage 108). In some embodiments, the I/O operations may be intercepted by and controlled by the storage application 106. As changes are made to data stored on storage 108 via the I/O operations from host 104, or over time as storage system 100 operates, storage application 106 may perform data replication from the source site 102 to the target site 112 over a communication network 110. In some embodiments, the communication network 110 may include internal (e.g., short distance) communication links (not shown) to transfer data between storage volumes for storing replicas 107 and 118 (also referred to herein as snap sets), such as an InfiniBand (IB) link or Fibre Channel (FC) link. In other embodiments, the communication link 110 may be a long-distance communication network of a storage area network (SAN), e.g., over an Ethernet or Internet (e.g., TCP/IP) link that may employ, for example, the iSCSI protocol.
In illustrative embodiments, storage system 100 may employ a snap set (or replication) mechanism to replicate data between source site 102 and target site 112. A snap set (or replica) may be created from data within storage 108 and transferred to the target site 112 during a data replication cycle by data replication.
Data replication may be performed based on data replication policies that may define various settings for data recovery operations, shown as policy 114 in target site 112. For example, policy 114 may define a plurality of attributes, such as a frequency with which replicas are generated and how long each replica 118 is kept at target site 112. In some embodiments, policy 114 defines metrics for use in snap set creation and replication process determinations. For example, metrics include a minimum snap set creation interval, a maximum snap set creation interval, and a recovery time threshold.
As described herein, in example embodiments, data replication may be synchronous data replication with snap sets created in dynamic intervals during operation of storage system 100. The timing of synchronous replication cycles and the retention of the replicas 118 may be managed by replica manager 116 of target site 112.
In addition to managing replicas 118 according to a policy 114 (e.g., a replication and/or retention policy), the replica manager 116 may also include a cycle counter 117 to track generations of snap sets over time, as will be described further herein.
It will be understood that the roles of the source site 102 and the target site 112 may be reversed in instances, e.g., in which an event occurring on the source site 102 causes the target site 112 to intercept I/Os and take on the role of snap set creation and replication to the source site. This role reversal is referred to as a failover event. In this manner, the processes described herein apply equally to the target site.
In embodiments, the identical snap set pair creation process leverages the use of a sync replication IO journal. The journal may be stored at any dedicated location in the storage system of
Referring to
Turning now to
In block 302, the process 300 creates a snap set (S-base) on the source site.
In block 304, the process 300 marks the valid sync replication IO journal entries at the time of snap creation. Each journal entry represents an inflight 10 request. When a journal entry is allocated to track inflight sync IO, the entry becomes valid. If marking is needed due to snap set creation, then all the existing valid journal entries are marked. Once the IO is complete, the entry becomes invalid and unmarked. A sample journal table is shown below in a non-limiting embodiment.
In block 306, it is determined whether all marked sync replication IO journal entries are removed (e.g., all marked inflight IOs are completed). By completed, this means that they have been successfully transmitted to the source or target.
If so, in block 308, the process 300 creates a snap set S-base′ on the target site. The S-base′ contains all of the content of the S-base, since all of the inflight IOs at the time of S-base creation have completed before the S-base′ creation. However, there may be other IOs completed while waiting for inflight IO completion (all marked journal entries cleared), which is why S-base′>=S-base. Otherwise, if not all marked sync replication IO journal entries have been removed, the process 300 continues to track the journal entries.
In block 310, the process 300 creates a local snap set Sn against the source and a remote snap set Sn′ against the S-base′. The remote target snap set Sn′ is a paired object of Sn created on the local source. In sync replication, data updates are replicated from a source storage group to a target. If a snap set is created against a source, it means that a snap set of the point in time content of the replication source storage object is created. The S-base′ is created prior to Sn and Sn′ so its content is less than Sn and is used as a base for Sn′. If a remote snap set Sn is created against the S-base′, and S-base′>=S_base, add (Sn-S-base) on top of S-base′, the resulting Sn′ will be equivalent to Sn. In other words, Sn′=S-base′+(Sn-S-base)=Sn.
In block 312, the process 300 transfers the data difference (D-delta) between Sn and S-base to the target. In block 314, the process 300 writes the difference to Sn′ on the target. Since the S-base<=S-base′, and Sn=S-base+D-delta, once the data difference transfer is complete, the result is Sn′=S-base′+D-delta==Sn.
Since the source IO is not suspended during this process 300, the data D-delta is essentially transferred twice, once to the target site via sync replication IO, and once to S-n′ via the special async delta transfer described above. With the capability of marking inflight IOs through sync replication IO journal, and creating a snap set right after the short window of marked inflight IOs complete, the data needed to retransmit is kept at a minimum.
Referring to
Process 300 shown in
The processes described herein are not limited to the specific embodiments described. For example, process 300 is not limited to the specific processing order shown in
Processor 402 may be implemented by one or more programmable processors executing one or more computer programs to perform the functions of the system. As used herein, the term “processor” is used to describe an electronic circuit that performs a function, an operation, or a sequence of operations. The function, operation, or sequence of operations can be hard coded into the electronic circuit or soft coded by way of instructions held in a memory device. A “processor” can perform the function, operation, or sequence of operations using digital values or using analog signals. In some embodiments, the “processor” can be embodied in an application specific integrated circuit (ASIC). In some embodiments, the “processor” can be embodied in a microprocessor with associated program memory. In some embodiments, the “processor” can be embodied in a discrete electronic circuit. The “processor” can be analog, digital or mixed-signal.
While illustrative embodiments have been described with respect to processes of circuits, described embodiments may be implemented as a single integrated circuit, a multi-chip module, a single card, or a multi-card circuit pack. Further, as would be apparent to one skilled in the art, various functions of circuit elements may also be implemented as processing blocks in a software program. Such software may be employed in, for example, a digital signal processor, micro-controller, or general purpose computer. Thus, described embodiments may be implemented in hardware, a combination of hardware and software, software, or software in execution by one or more processors.
Some embodiments may be implemented in the form of methods and apparatuses for practicing those methods. Described embodiments may also be implemented in the form of program code, for example, stored in a storage medium, loaded into and/or executed by a machine, or transmitted over some transmission medium or carrier, such as over electrical wiring or cabling, through fiber optics, or via electromagnetic radiation. A non-transitory machine-readable medium may include but is not limited to tangible media, such as magnetic recording media including hard drives, floppy diskettes, and magnetic tape media, optical recording media including compact discs (CDs) and digital versatile discs (DVDs), solid state memory such as flash memory, hybrid magnetic and solid state memory, non-volatile memory, volatile memory, and so forth, but does not include a transitory signal per se. When embodied in a non-transitory machine-readable medium, and the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the method.
When implemented on a processing device, the program code segments combine with the processor to provide a unique device that operates analogously to specific logic circuits. Such processing devices may include, for example, a general purpose microprocessor, a digital signal processor (DSP), a reduced instruction set computer (RISC), a complex instruction set computer (CISC), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a programmable logic array (PLA), a microcontroller, an embedded controller, a multi-core processor, and/or others, including combinations of the above. Described embodiments may also be implemented in the form of a bitstream or other sequence of signal values electrically or optically transmitted through a medium, stored magnetic-field variations in a magnetic recording medium, etc., generated using a method and/or an apparatus as recited in the claims.
Various elements, which are described in the context of a single embodiment, may also be provided separately or in any suitable subcombination. It will be further understood that various changes in the details, materials, and arrangements of the parts that have been described and illustrated herein may be made by those skilled in the art without departing from the scope of the following claims.
In the above-described flow chart of
Some embodiments may be implemented in the form of methods and apparatuses for practicing those methods. Described embodiments may also be implemented in the form of program code, for example, stored in a storage medium, loaded into and/or executed by a machine, or transmitted over some transmission medium or carrier, such as over electrical wiring or cabling, through fiber optics, or via electromagnetic radiation. A non-transitory machine-readable medium may include but is not limited to tangible media, such as magnetic recording media including hard drives, floppy diskettes, and magnetic tape media, optical recording media including compact discs (CDs) and digital versatile discs (DVDs), solid state memory such as flash memory, hybrid magnetic and solid state memory, non-volatile memory, volatile memory, and so forth, but does not include a transitory signal per se. When embodied in a non-transitory machine-readable medium and the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the method.
When implemented on one or more processing devices, the program code segments combine with the processor to provide a unique device that operates analogously to specific logic circuits. Such processing devices may include, for example, a general purpose microprocessor, a digital signal processor (DSP), a reduced instruction set computer (RISC), a complex instruction set computer (CISC), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a programmable logic array (PLA), a microcontroller, an embedded controller, a multi-core processor, and/or others, including combinations of one or more of the above. Described embodiments may also be implemented in the form of a bitstream or other sequence of signal values electrically or optically transmitted through a medium, stored magnetic-field variations in a magnetic recording medium, etc., generated using a method and/or an apparatus as recited in the claims.
For example, when the program code is loaded into and executed by a machine, such as the computer of
In some embodiments, a storage medium may be a physical or logical device. In some embodiments, a storage medium may consist of physical or logical devices. In some embodiments, a storage medium may be mapped across multiple physical and/or logical devices. In some embodiments, storage medium may exist in a virtualized environment. In some embodiments, a processor may be a virtual or physical embodiment. In some embodiments, a logic may be executed across one or more physical or virtual processors.
For purposes of illustrating the present embodiment, the disclosed embodiments are described as embodied in a specific configuration and using special logical arrangements, but one skilled in the art will appreciate that the device is not limited to the specific configuration but rather only by the claims included with this specification. In addition, it is expected that during the life of a patent maturing from this application, many relevant technologies will be developed, and the scopes of the corresponding terms are intended to include all such new technologies a priori.
The terms “comprises,” “comprising”, “includes”, “including”, “having” and their conjugates at least mean “including but not limited to”. As used herein, the singular form “a,” “an” and “the” includes plural references unless the context clearly dictates otherwise. Various elements, which are described in the context of a single embodiment, may also be provided separately or in any suitable subcombination. It will be further understood that various changes in the details, materials, and arrangements of the parts that have been described and illustrated herein may be made by those skilled in the art without departing from the scope of the following claims.
