Patent Application
20150012492
This application claims priority to and the benefit of Korean Patent Application No. 10-2013-0077380 filed in the Korean Intellectual Property Office on Jul. 2, 2013, the entire contents of which are incorporated herein by reference.
(a) Field of the Invention
The present invention relates to a method and apparatus for replicating data.
(b) Description of the Related Art
For storage of important computing data, data duplication that maintains the same data in real-time in two or more machines or apparatuses may be necessary. For duplication of data, it is necessary to replicate data of different computing apparatuses.
An easiest method of synchronizing a file in two apparatuses is to transmit entire data of a changed file from a transmitting apparatus to a receiving apparatus. However, because the transmitting apparatus should transmit entire files every time, the method is very inefficient.
As a method of solving such a drawback, there is a method of equally maintaining data by giving and receiving a change log in which only a changed portion of a file is recorded. The transmitting apparatus transmits only changed data instead of entire data, and the receiving apparatus, having received the changed data updates only changed data, thereby maintaining synchronization. In this case, in order to find changed data, the receiving apparatus should compare files corresponding to a size of a specific unit (several bytes) from a first portion to an end portion of two files. Therefore, there is a problem that much time is consumed in finding a changed portion of a file.
The present invention has been made in an effort to provide a method and apparatus for replicating data having advantages of efficiently replicating data using a minimum amount of computing resources and time.
An exemplary embodiment of the present invention provides a method of replicating data in a transmitting node that changes a file so as to synchronize a file between the transmitting node and a receiving node. The method includes: providing a change file and change information about the change file as a hint, when the file is changed; generating a change log with reference to the change file and the hint; and transmitting the change log to the receiving node.
The hint may include hint type information, the hint type information may include one of a first type representing that the change file has stored new data; a second type representing that a changed portion has been continued in the change file; and a third type representing that a size of actual data that is included in the change file is a threshold value or less while the change file stores new data.
A hint of the first type may include only the hint type information.
The generating of a change log may include generating a change log including data of the change file when receiving the hint of the first type.
A hint of the second type may include a position of a change start point and a size of change data.
The generating of a change log may include reading data corresponding to a size of the change data from a position of the change start point in the change file, when receiving the hint of the second type, and generating a change log including the read data.
A hint of the third type may include a position and a data size of a data start point in which actual data is written in the change file.
The generating of a change log may include reading data corresponding to the data size from a position of the data start point in the change file, when receiving the hint of the third type, and generating a change log including the read data.
Another embodiment of the present invention provides a data replication apparatus for notifying a change of a file in a computing node that changes the file. The data replication apparatus includes a hint provider, a change log generator, and a change log transmitter. The hint provider provides a change file and change information about the change file as a hint, when the file is changed. The change log generator generates a change log with reference to the change file and the hint. The change log transmitter transmits the change log to another computing node.
The hint may include hint type information, and the hint provider may represent a first type with the hint type information when the change file stores new data and may represent a second type with the hint type information when a change portion of the change file is continued.
The change log generator may generate a change log including data of the change file, when receiving a hint of the first type.
A hint of the second type may include a position of a change start point and a size of change data, and the change log generator may generate a change log by reading data corresponding to a size of the change data from a position of the data start point in the change file, when receiving a hint of the second type.
The hint provider may represent a third type with the hint type information, when a size of actual data that is included in the change file is a threshold value or less while the change file stores new data.
A hint of the third type may include a position and a data size of a data start point in which actual data is written in the change file, and the change log generator may generate the change log by reading data corresponding to the data size from a position of the data start point in the change file, when receiving the hint of the third type.
The hint provider may include a change file routine that provides a change file, when the file is changed, and a hint generator that provides change information about the change file as the hint.
In the following detailed description, only certain exemplary embodiments of the present invention have been shown and described, simply by way of illustration. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. Accordingly, the drawings and description are to be regarded as illustrative in nature and not restrictive.
In addition, in the entire specification and claims, unless explicitly described to the contrary, the word “comprise” and variations such as “comprises” or “comprising” will be understood to imply the inclusion of stated elements but not the exclusion of any other elements.
Hereinafter, a method and apparatus for replicating data according to an exemplary embodiment of the present invention will be described in detail with reference to the drawings.
A unit that stores and changes computing data may include a data block, which is a set of one byte to several hundred bytes and several thousand bytes, and when such data blocks are gathered and stored at a disk, a file may be used. A method and apparatus for replicating data that are suggested in an exemplary embodiment of the present invention can be applied to entire data units according to a range that performs synchronization. For convenience of description, in an exemplary embodiment of the present invention, it is assumed that a target unit of synchronization is a file, and a method and apparatus for replicating data according to an exemplary embodiment of the present invention will be described. It is assumed that an apparatus, which is a subject that performs synchronization, is a computing node that is connected to a network for convenience, and the computing node is briefly referred to as a node.
Referring to
When the receiving node 200 receives the change log 150, by analyzing the change log 150, the receiving node 200 adjusts a file F2 and changes the file F2 to a file F3, thereby synchronizing the file F1 of the transmitting node 100 and the file F3 of the receiving node 200.
Referring to
Next, the transmitting node 100 generates a change log 150 in which a position 152 of a changed portion, a size 154 of changed contents, and changed contents 156 are displayed.
The receiving node 200 receives the change log 150 and writes the changed contents 156 at a corresponding position of the file F2 and thus the file F2 is changed to the file F3. Therefore, the receiving node 200 maintains the file F3 equally to the file F1 of the transmitting node 100.
However, because a method of generating a change log by comparing entire data of the files F0 and F1 generates the change log 150 without any information about the changed contents 156, in order to find the changed portion, entire files F0 and F1 should be compared. Particularly, when the changed portion is only a portion, most time is used for comparing unchanged data.
In an exemplary embodiment of the present invention, when changing a file, change information of the file is used when generating a change log and thus a method that can reduce a change log generation time is suggested. In a file change routine, when changing a file, change information about a file change is provided, and the change information is referred to as a hint. Further, this technique is called a hint-based data replicating technique.
Hereinafter, each hint that is used in an exemplary embodiment of the present invention will be described based on an exemplary embodiment.
Referring to
At least one file change routine 300 transfers a change file 302 to the change log generator 320 whenever changing a file 301. Further, whenever the file is changed in the file change routine 300, the hint generator 310 transfers a hint 303 of the changed file to the change log generator 320.
When writing a change log of the change file 302, the change log generator 320 generates a change log 304 with reference to the hint 303 together with the change file 302.
When the change log generator 320 transfers the generated change log 304 to the change log transmitter 330, the change log transmitter 330 transfers the change log to the receiving node 200, and the receiving node 200 synchronizes a corresponding file, as in the transmitting node 100. Because a file change of a plurality of files may simultaneously occur in a plurality of file change routines 300, the change log generator 320 receives a plurality of hints to write a change log of each file.
Hints that may be provided by the hint generator 310 of the transmitting node 100 may be various according to a file change routine, and in an exemplary embodiment of the present invention, a method of three cases that may be most efficiently applied when writing a change log is suggested.
A first case is one in which the file after change 302 newly stores data without using contents of the file before change 301 as a base. In this case, the hint generator 310 provides such information as a hint to the change log generator 320, and such a hint corresponds to a hint type 1.
Referring to
When the change log generator 320 receives the hint 400 of the hint type 1, the change log generator 320 compares contents of the file before change 301 and the file after change 302, does not write a change log, writes a change log 304 including only contents of the file after change 302, and transfers the change log 304 to the change log transmitter 330.
A second case is one in which a changed portion is not distributed but is continuously formed. In this case, the hint generator 310 provides such information as a hint to the change log generator 320, and in this case, the provided hint corresponds to a hint type 2.
Referring to
When the change log generator 320 receives the hint 500 of the hint type 2, the change log generator 320 does not compare the file before change 301 and the file after change 302, reads data of the file after change 302 with the position 520 of a change start point that is provided as the hint 500 and the size 530 of change data, and writes the change log 304. The hint 500 of the hint type 2 may be applied even when there are multiple changed portions of the file.
That is, the change log generator 320 reads data corresponding to a size of the change data from a position of a change start point in the change file 302 and generates a change log.
Referring to
Information about one change portion 601/602 includes a position 620/640 of a change start point and a size of change data 630/650, as shown in
The change log generator 320, having received the hint of
Next, a hint type 3, which is a third case, is a case that newly writes contents of an entire file, as in the hint type 1, and is an effective hint type when a size of actual data that is included in a file is not large, compared with the entire size of the file. For example, there is a file having a size of 100 bytes, but when a size of a portion having actual data is only 10 bytes, if the file is transferred to a receiving node, data of 100 bytes, which is an entire size, should be sent. In this case, an area of a 90 byte size in which data is not written is filled with “0” and is sent. The hint type 3 represents a case in which such a file is generated, and in this case, the hint generator 310 notifies a portion in which actual data is written in the file as a hint and provides the hint to the change log generator 320.
Referring to
Information about each data portion 701/702 includes a position 720/740 of a data start point and a data size 730/750, as shown in
As shown in
The change log generator 320, having received a hint of the hint type 3, reads only a portion in which actual data is written without reading the entire file after change 302 and generates a change log.
A case in which a hint is not provided may be advantageous. This indicates a wide change range. That is, this is a case in which the side that changes a file changes many portions of the file, and in this case, a log file is written using an existing method, and at this time, a hint is not provided.
When change contents of a file are not estimated, it is difficult to determine whether giving a hint is effective. However, when a file change is performed along a determined routine in a specific software code, it may be estimated how to change in a corresponding code and thus it may be determined whether a hint is effective.
An exemplary embodiment thereof may include duplication of metadata of a distribution file system. Metadata is data having information about a file system and is a very important element in a file system. Therefore, in order to safely keep metadata, duplication is necessary. For duplication, metadata that is changed whenever performing a service should be replicated to another node in real time. In this case, a change of metadata is performed in a metadata server and thus a metadata server knows operation of each service routine such that how metadata is changed may be estimated. In this case, an exemplary embodiment of the present invention generates a change log file based on a hint that is provided in a routine that changes a file instead of generating a log file by comparing with an existing file without any hint like an existing method, and thus can reduce a load and a time that are consumed when writing a log file.
Referring to
The data replication apparatus 800 of the transmitting node 100 includes a hint provider 810, a change log generator 820, and a change log transmitter 830.
The data replication apparatus 900 of the receiving node 200 includes a change log receiver 910 and a replicator 920.
In the data replication apparatus 800 of the transmitting node 100, the hint provider 810 may include the file change routine 300 and the hint generator 310 that are described with reference to
In the data replication apparatus 900 of the receiving node 200, the change log receiver 910 receives a change log from the data replication apparatus 800 of the transmitting node 100. By analyzing the change log, the replicator 920 changes the file F2 of the receiving node 200 to the file F3. Thereby, the changed file F3 of the receiving node 200 becomes identical to the changed file F1 of the transmitting node 100.
A function of two transmitting and receiving nodes may be changed, as needed. Therefore, the node may include the data replication apparatus 800 of the transmitting node 100 and the data replication apparatus 900 of the receiving node 200.
According to an exemplary embodiment of the present invention, by using a hint-based data replicating technique, data can be efficiently replicated in two or more different apparatuses using a minimum amount of computing resources and time.
An exemplary embodiment of the present invention may be not only embodied through the above-described apparatus and/or method, but may also be embodied through a program that executes a function corresponding to a configuration of the exemplary embodiment of the present invention or through a recording medium on which the program is recorded, and can be easily embodied by a person of ordinary skill in the art from a description of the foregoing exemplary embodiment.
While this invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2013-0077380 | Jul 2013 | KR | national |
