Patent Application
20160232055
This technology generally relates to data storage management and, more particularly, methods for distributing erasure-coded fragments in a geo-distributed storage system and devices thereof.
Geographically distributed storage systems are used in the current technologies to store files or data by the clients due to the high end-to-end performance and reliability. To use these geographically distributed storage systems, existing technologies use erasure coding techniques. With erasure coding the data is divided into numerous data fragments and parity fragments are created. These data fragments and the created parity fragments are then distributed across geographically distributed storage systems.
However, while distributing the data fragments and parity fragments, prior technologies fails to properly address issues, such as the failure of storage systems, inefficiencies during ingestion, retrieval and repair of the data, and issues with parity fragments. Since these prior technology fail to address these issues, the storage systems are not as efficient as they could be resulting in utilization of a higher number of input/output resources. Additionally, because of these inefficiencies, client devices experience unnecessary delay while performing basic operations, such as ingesting data into the geographically distributed storage systems.
A method for distributing erasure coded fragments in geo-distributed storage nodes comprising receiving, by a storage management computing device, an object and a storage reliability requirement from a client computing device. Erasure coding is performed by the storage management computing device on the received object using an erasure coding scheme. An erasure coding group from a plurality of erasure coding groups present in a plurality of geographically distributed storage nodes is determined by the storage management computing device based on the received storage reliability requirement and the erasure coding scheme. The erasure coded object is distributed by the storage management computing device to the determined erasure coding group from the plurality of erasure coding groups in the plurality of geographically distributed storage nodes.
A non-transitory computer readable medium having stored thereon instructions for distributing erasure coded fragments in geo-distributed storage nodes comprising executable code which when executed by a processor, causes the processor to perform steps includes receiving an object and a storage reliability requirement from a client computing device. Erasure coding is performed on the received object using an erasure coding scheme. An erasure coding group from a plurality of erasure coding groups present in a plurality of geographically distributed storage nodes is determined based on the received storage reliability requirement and the erasure coding scheme. The erasure coded object is distributed to the determined erasure coding group from the plurality of erasure coding groups in the plurality of geographically distributed storage nodes.
A storage management computing device includes a processor and a memory coupled to the processor which is configured to be capable of executing programmed instructions comprising and stored in the memory to receive an object and a storage reliability requirement from a client computing device. Erasure coding is performed on the received object using an erasure coding scheme. An erasure coding group from a plurality of erasure coding groups present in a plurality of geographically distributed storage nodes is determined based on the received storage reliability requirement and the erasure coding scheme. The erasure coded object is distributed to the determined erasure coding group from the plurality of erasure coding groups in the plurality of geographically distributed storage nodes.
This technology provides a number of advantages including providing methods, non-transitory computer readable medium and devices for more effectively and efficiently distributing erasure-coded fragments in a geo-distributed storage system. Additionally, this technology is able to provide a user with a reliability requirement and then store the erasure coded objects in an EC group based on the selected reliability requirement and EC scheme. Further, this technology ranks the chunk services thereby balancing ingest, repair, and retrieval load across the plurality of storage nodes.
An environment 10 with a plurality of client computing devices 12(1)-12(n), an exemplary storage management computing device 14, a plurality of storage nodes 16(1)-16(n) is illustrated in
Referring to
The processor 18 of the storage management computing device 14 may execute one or more programmed instructions stored in the memory 20 for distributing erasure-coded fragments in a geo-distributed storage system as illustrated and described in the examples herein, although other types and numbers of functions and/or other operation can be performed. The processor 18 of the storage management computing device 14 may include one or more central processing units (“CPUs”) or general purpose processors with one or more processing cores, such as AMD® processor(s), although other types of processor(s) could be used (e.g., Intel®).
The memory 20 of the storage management computing device 14 stores the programmed instructions and other data for one or more aspects of the present technology as described and illustrated herein, although some or all of the programmed instructions could be stored and executed elsewhere. A variety of different types of memory storage devices, such as a random access memory (RAM) or a read only memory (ROM) in the system or a floppy disk, hard disk, CD ROM, DVD ROM, or other computer readable medium which is read from and written to by a magnetic, optical, or other reading and writing system that is coupled to the processor 18, can be used for the memory 20.
The communication interface 24 of the storage management computing device 14 operatively couples and communicates with the plurality of client computing devices 12(1)-12(n) and the plurality of storage nodes 16(1)-16(n), which are all coupled together by the communication network 30, although other types and numbers of communication networks or systems with other types and numbers of connections and configurations to other devices and elements. By way of example only, the communication network 30 can use TCP/IP over Ethernet and industry-standard protocols, including NFS, CIFS, SOAP, XML, LDAP, and SNMP, although other types and numbers of communication networks, can be used. The communication networks 30 in this example may employ any suitable interface mechanisms and network communication technologies, including, for example, any local area network, any wide area network (e.g., Internet), teletraffic in any suitable form (e.g., voice, modem, and the like), Public Switched Telephone Network (PSTNs), Ethernet-based Packet Data Networks (PDNs), and any combinations thereof and the like. In this example, the bus 26 is a universal serial bus, although other bus types and links may be used, such as PCI-Express or hyper-transport bus.
Each of the plurality of client computing devices 12(1)-12(n) includes a central processing unit (CPU) or processor, a memory, an interface device, and an I/O system, which are coupled together by a bus or other link, although other numbers and types of network devices could be used. The plurality of client computing devices 12(1)-12(n) communicates with the storage management computing device 14 to storing files and data in the plurality of storage nodes 16(1)-16(n), although the client computing devices 12(1)-12(n) can interact with the storage management computing device 14 for other purposes. By way of example, the plurality of client computing devices 12(1)-12(n) may run interface application(s) that may provide an interface to make requests to access, modify, delete, edit, read or write data within storage management computing device 14 or the plurality of storage nodes 16(1)-16(n) via the communication network 30.
Each of the plurality of storage nodes 16(1)-16(n) includes a central processing unit (CPU) or processor, storage volumes within a memory, an interface device, and an I/O system, which are coupled together by a bus or other link, although other numbers and types of network devices could be used. Each of the plurality of storage nodes 16(1)-16(n) assist with storing of files and data from the plurality of client computing devices 12(1)-12(n) or the storage management computing device 14, although the plurality of storage nodes 16(1)-16(n) can assist with other types of operations. In this example, each of the plurality of storage nodes 16(1)-16(n) can be spread across different geographical locations. In another example, all of the plurality of storage nodes 16(1)-16(n) can be present in one geographical location. Additionally in this example, an erasure coding group relates to a collection of a fixed number of virtual chunk services, although the erasure coding group can include other types or amounts of information. Further, data and parity fragments of erasure coded objects are stored in the virtual chunk services contained in an erasure coding group. By way of example only, the number of virtual chunk services in an erasure coding group usually corresponds to the number of fragments required by an erasure coding scheme; for example in the case of a 6+3 scheme, the erasure coding group will have nine virtual chunk services. The virtual chunk services in an erasure coding group are allocated across distinct chunk services. Various network processing applications, such as CIFS applications, NFS applications, HTTP Web Data storage device applications, and/or FTP applications, may be operating on the plurality of storage nodes 16(1)-16(n) and transmitting data (e.g., files or web pages) in response to requests from the storage management computing device 14 and the plurality of client computing devices 12(1)-12(n). It is to be understood that the plurality of storage nodes 16(1)-16(n) may be hardware or software or may represent a system with multiple external resource servers, which may include internal or external networks. In this example the plurality of storage nodes 16(1)-16(n) may be any version of Microsoft® IIS servers or Apache® servers, although other types of servers may be used.
Although the exemplary network environment 10 includes the plurality of client computing devices 12(1)-12(n), the storage management computing device 14, and the plurality of storage nodes 16(1)-16(n) described and illustrated herein, other types and numbers of systems, devices, components, and/or other elements in other topologies can be used. It is to be understood that the systems of the examples described herein are for exemplary purposes, as many variations of the specific hardware and software used to implement the examples are possible, as will be appreciated by those of ordinary skill in the art.
In addition, two or more computing systems or devices can be substituted for any one of the systems or devices in any example. Accordingly, principles and advantages of distributed processing, such as redundancy and replication also can be implemented, as desired, to increase the robustness and performance of the devices and systems of the examples. The examples may also be implemented on computer system(s) that extend across any suitable network using any suitable interface mechanisms and traffic technologies, including by way of example only teletraffic in any suitable form (e.g., voice and modem), wireless traffic media, wireless traffic networks, cellular traffic networks, G3 traffic networks, Public Switched Telephone Network (PSTNs), Packet Data Networks (PDNs), the Internet, intranets, and combinations thereof.
The examples also may be embodied as a non-transitory computer readable medium having instructions stored thereon for one or more aspects of the present technology as described and illustrated by way of the examples herein, as described herein, which when executed by the processor, cause the processor to carry out the steps necessary to implement the methods of this technology as described and illustrated with the examples herein. An example of a method for distributing erasure-coded fragments in a geo-distributed storage system will now be described herein with reference to
Referring more specifically to
Next in step 310, the storage management computing device 14 groups the obtained chunk services based on the data centers. In this example, chunk services are present within the plurality of storage nodes 16(1)-16(n) and each data center includes multiple storage nodes of the plurality of storage nodes 16(1)-16(n). Accordingly, when the storage management computing device 14 obtains the chunk services in the plurality of storage nodes 16(1)-16(n), the storage management computing device 14 may not be arranged or grouped based on the data center in which the chunk services are present. Therefore, the storage management computing device 14 groups the obtained chunk services based on the data centers, although the storage management computing device 14 can group the obtained chunk services based on other parameters.
In step 315, the storage management computing device 14 assigns a reliability requirement to each of the grouped chunk services within the plurality of storage nodes 16(1)-16(n) and the corresponding data centers. In this example, memory 20 includes a list of reliability requirements that can be provided to the plurality of client computing devices 12(1)-12(n), although the list of reliability requirements can be present at other memory locations. By way of example only, the list of reliability requirements that can be provided to the plurality of client computing devices 12(1)-12(n) includes accessing objects when a data center or volume has failed, although the list of reliability requirements can include other types of options that can be provided to plurality of client computing devices 12(1)-12(n). Accordingly in this example, each reliability requirement includes one or more group of chunk services and the corresponding data centers assigned to it.
Next in step 320, the storage management computing device 14 assigns a rank to each of the chunk service within the plurality of storage nodes 16(1)-16(n) in the assigned data centers based on the space of the chunk service, computing capacity of the chunk service and the memory of the chunk service, although the storage management computing device 14 can assign the rank based on other parameters or criteria. In this example, the storage management computing device 14 assigns a higher rank to the chunk service within the plurality of storage nodes 16(1)-16(n) that has space that is above a memory threshold, computing capacity above a computing capacity threshold and memory above a memory threshold. By way of example only, the rank assigned to the chunk service by the storage management computing device 14 is a numerical value from one to hundred (one being the highest and hundred being the lowest), although the rank assigned to the chunk service by the storage management computing device 14 can be other values.
Next in step 325, the storage management computing device 14 creates erasure coded (EC) groups in the plurality of storage nodes 16(1)-16(n) including the higher ranked chunk services to store erasure coded objects, although the storage management computing device 14 can. Prior to creating the EC groups, the storage management computing device 14 first identifies one or more EC groups with a higher rank to create the EC groups in the plurality of storage nodes 16(1)-16(n). By way of example only, the storage management computing device 14 creates EC groups in each of the plurality of storage nodes 16(1)-16(n) that includes top fifty ranked chunked services, although the storage management computing device 14 can create EC groups in the plurality of storage nodes 16(1)-16(n) including lower ranked chunk services.
In step 330, the storage management computing device 14 assigns erasure coded (EC) schemes for each of the created erasure coded groups in the plurality of storage nodes 16(1)-16(n). As it would be appreciated by a person having ordinary skill in the art, an object in erasure coding can be divided into k number of data fragments and m number of parity fragments are created by encoding the k data fragments. Therefore, the original data can be recovered from any number of k data fragments of the m+k fragments (i.e., the EC scheme can tolerate m number of failures). In this example, an EC scheme relates to different numerical values that can be assigned to k data fragments and m parity fragments. By way of example only and for purpose of further illustration, 9+3 is an EC scheme where there are nine (k=9) data fragments and three (m=3) parity fragments. Accordingly in this example, the storage management computing device 14 assigns different EC schemes such as 6+3 scheme, 12+4 scheme, or other m+k EC schemes to each of the created erasure coded groups in the plurality of storage nodes 16(1)-16(n), although other types of EC schemes can be assigned to the created erasure coded groups by the storage management computing device 14 and the exemplary method of creating EC groups end at step 335.
Next,
Next in step 410, the storage management computing device 14 receives a selection of one of the reliability requirement from a provided list of reliability requirements from the requesting one of the plurality of client computing devices 12(1)-12(n), although the storage management computing device 14 can receive multiple selection of the reliability requirements from the requesting one of the plurality of client computing devices 12(1)-12(n). As previously illustrated above in step 315 of
Next in step 415, the storage management computing device 14 selects one of plurality of EC groups that was previously created in step 325 of
In step 420, the storage management computing device 14 determines one of plurality of erasure coding (EC) schemes based on the received selection of the reliability requirement, although the storage management computing device 14 can select the EC scheme based on other parameters such as type of received object, size of the received object or format of the received object. As previously illustrated in step 330 of
In step 425, the storage management computing device 14 performs erasure coding on the received object from the requesting one of the plurality of client computing devices 12(1)-12(n) using the determined erasure coding (EC) scheme, although the storage management computing device 14 can perform erasure coding using other types of EC schemes. In this example, the storage management computing device 14 performs erasure coding using Reed Solomon erasure coding technique, which is incorporated here by reference in its entirety, although the storage management computing device 14 can use other techniques to perform erasure coding.
In step 430, the storage management computing device 14 distributes and stores the erasure coded object to the selected EC group of the plurality of EC groups in the plurality of storage nodes 16(1)-16(n), although the storage management computing device 14 can distribute and store the EC object in another EC groups. In this example, the selected EC groups to which the EC object is distributed and stored includes higher ranked chunk services, although the storage management computing device 14 can distribute and store the EC objects in EC groups having lower ranked chunk services. Additionally, while distributing and storing the EC object, if the selected EC group is full, the storage management computing device 14 proceeds to select another EC group. In this example, the storage management computing device 14 determines the selected EC group to be full when the selected EC group does not have enough memory to store erasure coded received object in the form of data fragments or parity fragments. Accordingly, when the storage management computing device 14 determines that the selected EC group is full another EC group is identified by the storage management computing device 14. Alternatively in another example, the storage management computing device 14 can create a new EC group using the technique illustrated in steps 305-330 of
Accordingly, as illustrated and described with reference to the examples herein, this technology provides methods, non-transitory computer readable medium and devices that are able to more effectively and efficiently distributing erasure-coded fragments in a geo-distributed storage system. By using the techniques illustrated above, the technology disclosed herein is able to provide the user with the reliability requirement and then store to erasure coded objects in the EC group based on the selected reliability requirement and EC scheme. Additionally, the technology disclosed ranks the chunk services thereby balancing ingest, repair and retrieval load across the plurality of storage nodes. Further, this technology also increases the input/output efficiency and other computing resources of the storage management computing device thereby increasing the overall performance of the storage management computing device. This technology also is able to hide the failure in the storage volumes or storage nodes by distributing and storing the erasure coded objects in the EC groups that prevent these failures.
Having thus described the basic concept of the disclosed technology, it will be rather apparent to those skilled in the art that the foregoing detailed disclosure is intended to be presented by way of example only, and is not limiting. Various alterations, improvements, and modifications will occur and are intended to those skilled in the art, though not expressly stated herein. These alterations, improvements, and modifications are intended to be suggested hereby, and are within the spirit and scope of the invention. Additionally, the recited order of processing elements or sequences, or the use of numbers, letters, or other designations therefore, is not intended to limit the claimed processes to any order except as may be specified in the claims. Accordingly, the invention is limited only by the following claims and equivalents thereto.
