1. Technical Field
The embodiments of the present disclosure relate to management technology, and particularly to a data block saving system and method.
2. Description of Related Art
A data center is a facility which houses a large number of computers and stores huge amounts of data. By using cloud computing, the files are uploaded into a data center. However, at present, a file stored in the data center may include duplicates or duplicated portions, which waste a lot of storage spaces. Therefore, there is room for improvement in the art.
The disclosure is illustrated by way of examples and not by way of limitation in the figures of the accompanying drawings in which like references indicate similar elements. It should be noted that references to “an” or “one” embodiment in this disclosure are not necessarily to the same embodiment, and such references mean “at least one.”
In general, the word “module”, as used herein, refers to logic embodied in hardware or firmware, or to a collection of software instructions, written in a programming language, such as, Java, C, or assembly. One or more software instructions in the modules may be embedded in firmware, such as in an EPROM. The modules described herein may be implemented as either software and/or hardware modules and may be stored in any type of non-transitory computer-readable medium or other storage device. Some non-limiting examples of non-transitory computer-readable media include CDs, DVDs, BLU-RAY, flash memory, and hard disk drives.
In one embodiment, the client 1 divides each file into two or more data blocks, and uploads the two or more data blocks of the file into the assignment server 2. The assignment server 2 sends the two or more data blocks of the file to the storage server 3. Additionally, before uploading the two or more data blocks of the file into the assignment server 2, the client 1 further calculates a hash value of each data block and saves the hash value of each data block into a hash list. The client 1 also includes information of the files. The information of each file includes a name of the file and an attribute of the file. Furthermore, each file corresponds to a hash list. In other words, the data blocks of the file are saved into the hash list corresponding to the file. Each data block includes a name. The name of each data block is generated in order and also saved into the hash list. In detail, the name of each data block is generated in an alphabetical order (e.g., “a,” “b,” “c,” “d,” “d,” or “f”) or in a numerical order (e.g., “1,” “2,” “3,” or “4”). For example, the file is divided to three data blocks, namely data block “a,” data block “b,” and data block “c.” Each data block may include a storage capacity predetermined by a user, such as 16 KB, 32 KB, 64 KB, 128 KB, or 256 KB. For example, if the storage capacity is predetermined as 32 KB, the file is divided into a plurality of data blocks, and each data block is 32 KB.
The receiving module 2000 receives a hash list corresponding to a file and information of the file uploaded from the client 1, and saves the hash list corresponding to the file and information of the file into the database 4. For example, the receiving module 2000 receives the hash list corresponding to the file A and information of the file A from the client A1, the receiving module 2000 receives the hash list corresponding to the file B and information of the file A from the client B1, and the receiving module 2000 receives the hash list corresponding to the file B and information of the file B from the client B1.
The setting module 2002 sets a sequence number of each data block. In one embodiment, the sequence number of each data block may be in the alphabetical order (e.g., “a,” “b,” “c,” “d,” “d,” or “f”) or in the numerical order (e.g., “1,” “2,” “3,” or “4”). As shown in
The determination module 2004 obtains a data block according to the sequence number of the data block and determines if the obtained data block is a repetitive data block. For example, the determination module 2004 obtains the data blocks of the file A stored in the client A1 in the alphabetical order, for example, from the data block “a” to the data block “f”). The determination module 2004 searches for repetitive data blocks according to the hash value of each data block.
The obtained data block is determined as the repetitive data block upon the condition that the hash values of the obtained data block is the same as the hash values of other data blocks in the storage server 3. For example, as shown in
The obtained data block is further determined as the repetitive data block upon the condition that the hash values of the obtained data block is the same as the hash values of other data blocks which is in a process of uploading into the storage server 3. In one embodiment, for example, as shown in
The removing module 2006 skips the obtained data block and obtains next data block according to the sequence number of each data block. For example, if the obtained data block “a” from the client A1 is determined as the repetitive data block, the removing module 2006 skips the obtained data block “a,” and obtains next data block “b.”
The uploading module 2008 uploads the obtained data block from the client 1 into the storage server 3. Additionally, the uploading module 2008 sends a pointer of the obtained data block to the client 1. The pointer of the obtained data block is received from the assignment server 2 and displayed on a display device of the client 1. The obtained data block corresponds to a pointer that points to a storage space of the storage server 3. In other words, a user uses the pointer to find the storage space and know where the obtained data block is saved in the storage server 3. The storage space may store one or more data blocks in the server 2. Furthermore, even the repetitive data blocks are skipped in the client 1, however, each repetitive data block is also assigned to one pointer, and the pointer corresponding to the repetitive data block is the same as the pointer corresponding to the data block in the storage server 3.
In step S100, each client 1 divides a file stored in the client 1 into two or more data blocks, saves a name of each data block and a hash value of each data block into a hash list. For example, as shown in
In step S102, each client 1 uploads information of the file into an assignment server 2 and uploads the hash list corresponding to the file into a database 4. The receiving module 2000 receives the information of the file and the hash list from each client 1.
In step S104, the setting module 2002 sets a sequence number of each data block. As shown in
In step S106, the determination module 2004 obtains a data block according to the sequence number of the data block and determines if the obtained data block is a repetitive data block. In one embodiment, the determination module 2004 searches for repetitive data blocks according to the hash value of each data block. If the obtained data block is the repetitive data block, the procedure goes to step S108. Otherwise, if the obtained data block is not the repetitive data block, the procedure goes to step S110.
In step S108, the removing module 2006 skips the obtained data block and obtains next data block according to the sequence number of each data block, then the procedure returns to step S106.
In step S110, the uploading module 2008 uploads the obtained data block from the client 1 into the storage server 3. The uploading module 2008 also obtains a pointer of the obtained data block when the obtained data block is saved into the storage server 3, and the uploading module 2008 sends the pointer of the obtained data block to the client 1.
In step S112, the available server 3 receives the obtained data block from the assignment server 2, and determines if the obtained data block is correct. In one embodiment, when the storage server 3 receives the obtained data block from the assignment server 2, the storage server 3 also calculates the hash value of the obtained data block, and verifies if the hash value of the obtained data block exists in the hash list. If the hash value of the obtained data block exists in the hash list, the data block is determined to be correct, the procedure goes to step S114. If the hash value of the obtained data block does not exist in the hash list, the obtained data block is determined to not be correct, the procedure goes to step S116.
In step S114, the storage server 3 sends the pointer of the obtained data block to the client 1.
In step S116, the storage server 3 notifies the client 1 to upload the obtained file again.
In step S200, the client 1 obtains a hash value of each data block of a file from a hash list stored in a database 4.
In step S202, the client 1 downloads each data block of the file according to a pointer of each data block from the storage server.
In step S204, the download module 2012 calculates a hash value of each downloaded data block and determines if the hash value of each downloaded data block exists in the hash list stored in the database 4. In one embodiment, if the calculated hash value of each downloaded data block exists in the database 4, the procedure goes to step S206. Otherwise, if one calculated hash value of the downloaded data block does not exist in the hash list, the procedure returns to step S200.
In step S206, the client 1 combines all downloaded data blocks to generate the file in the temporary storage space of the client 1 according to the sequence number of each downloaded data block. The temporary storage space of the client 1 may be, but is not limited to, a random access memory (RAM). In one embodiment, the sequence number of each downloaded data block is generated in order, and the client 1 combines all downloaded data blocks to generate the file in order of the sequence number of each downloaded data block.
In step S208, the client 1 calculates the hash value of the generated file and determines if the calculated hash value of the generated file exists in the hash list stored in the database 4. If the calculated hash value of the generated file exists in the hash list, the procedure goes to step S210. If the calculated hash value of the generated file does not exists in the hash list, the client 1 displays fail information (e.g., display “FAIL”) on the display device of the client 1, and the procedure returns to step S200.
In step S210, the client 1 displays the generated file and success information (e.g., display “SUCCESS”) on a display device of the client 1
Although certain inventive embodiments of the present disclosure have been specifically described, the present disclosure is not to be construed as being limited thereto. Various changes or modifications may be made to the present disclosure without departing from the scope and spirit of the present disclosure.
Number | Date | Country | Kind |
---|---|---|---|
201210534098X | Dec 2012 | CN | national |