Patent Application
20150134625
Modern data centers extensively use server virtualization techniques. Server virtualization techniques enable better utilization of hardware resources and therefore reduce data center cost. By separating the physical computing system from the operating system and application software, virtualization enables dynamic allocation of resources, e.g., hardware or virtual machines. Data center administrators have the flexibility to move workloads from one server to another for balancing load, maintaining hardware, and enabling high availability.
Modern data centers also extensively use shared storage technology, including, e.g., network attached storage (NAS) and storage area networks (SAN). Both NAS and SAN technologies enable unified and centralized data management, which makes it easier for data center administrators to manage data. The administrators can choose the level of data protection (e.g. using redundant array of independent disks, “RAID”), enable mirroring for disaster recovery, and carefully configure the backup policies. Shared storage systems enable additional storage space to be dynamically added and reassigned. Centralization of the shared storage provides opportunities for deduplication to achieve greater storage efficiency.
Recently, flash-based solid-state drives (“SSDs”) are employed in environments associated with the virtualized, shared storage data centers. Flash is treated as an additional tier in the memory hierarchy between DRAM and magnetic hard disks. In terms of cost per gigabyte (“GB”), dynamic random-access memory (“DRAM”) capacity is more expensive than flash capacity, which in turn is more expensive than hard disk capacity. At the same time, DRAM latencies are less than flash, and flash latencies are less than magnetic hard disk. As a result, flash's cost per input/output (“I/O”) operation is between DRAM and magnetic hard disks. Recently, large caches have become increasingly common with the emergence of flash devices. As a result, companies have released flash caching products including, e.g., NetApp Flash Accel, EMC VFCache, and Fusion-io ioTurbine.
Objects, features and characteristics of the disclosed technology will become more apparent to those skilled in the art from a study of the following detailed description in conjunction with the appended claims and drawings, all of which form a part of this specification. In the drawings:
References in this specification to “an embodiment,” “one embodiment,” or the like, mean that the particular feature, structure, or characteristic being described is included in at least one embodiment of the disclosed technology. Occurrences of such phrases in this specification do not all necessarily refer to the same embodiment or all embodiments, however.
The technology disclosed herein employs duplication lists to improve the storage efficiency and communication efficiency for a storage client device by maximizing the cache hit rate and minimizing data requests to the storage server. Storage clients access data stored at storage services. The storage server provides a duplication list to the storage client device. The duplication list contains references (e.g. storage addresses) to data blocks that contain duplicate data content. The storage client uses the duplication list to improve the cache hit rate.
For example, the storage client can insert the information of the duplication list into an address map associated with its cache. When the storage client tries to read a data block, the storage client first consult the address map to determine whether there is a cached data block for the data block associated with the reference. If such a data block exists in the cache, the storage client device is able to retrieve the data to satisfy the read request directly from the cache, without sending any data request to the storage server over a network.
The duplication list can be pruned to contain references to data blocks relevant to the storage client device. The storage server can prune the duplication list based on a working set of storage objects for a client. Alternatively, the storage server can prune the duplication list based on content characteristics, e.g. duplication degree and access frequency. A duplication degree of a data chunk is, or can be calculated from, a number of duplicate data chunks that have the same content. Duplicate blocks to which the client does not have access can also be excluded from the duplication list.
The technology disclosed herein allows a flash-based SSD cache (or other type of hardware media such as storage class memory) in a storage client device to exploit the duplication information maintained by a storage server. The storage client cache can employ this duplication information to efficiently utilize the cache space and reduce or eliminate unnecessary communications with the storage server. For example, suppose there is a cache storing a single copy of data for disparate chunks of data whose contents are exactly duplicated in separate storage server locations. Such a cache may be able to store twice the data in the same amount of storage as would be possible without taking advantage of the storage server maintained duplication information.
Turning now to the Figures,
Referring to
The storage host devices (or servers) 108 may be, for example, one of the storage server products available from NetApp, Inc., the assignee of the present application, or available from other vendors. The client systems 104A-104N may access the storage host device 108 via network 106, which can be a packet-switched network, for example, a local area network (LAN), a wide area network (WAN), the Internet, or any other type of network. The client systems 104A-104N can include caches 170A-170N to store data has been written or read so that future write or read requests for that data can be served directly from the caches 170A-170N. The caches 170A-170N can be, e.g., flash-based SSDs.
The client systems 104A-104N can further include cache managers 175A-175N to manage the data stored in the cache 170A-170N. The cache managers 175A-175N can be implemented as applications or services running on the client systems 104A-104N, firmware in the cache 170A-170N, or a combination thereof. The cache managers 175A-715N can maintain a server duplication address list for duplicate data stored in the network system 102. For example, if the client system 104A needs data associated with server address Lx, the client system 104A first requests the cache manager 175A to determine whether the cache 170A stores the data associated with Lx. If the cache 170A does not store the data associated with Lx, the cache manager 175A further checks the server duplication address list to determine if there is any server address associated with data that duplicates the data associated with Lx. The server duplication address list may be generated and sent by the storage system 102, for example. If there is such duplicate server address, the cache manager 175A determines whether the cache 170A stores the data associated with the duplicate server address. If so, the cache manager 175A returns the data to the client system 104A directly from the cache 170A to satisfy the local data request for Lx. If the cache 170A does not store the data associated with any duplicate server address, the cache manager 175A or the client system 104A sends a data request for the data associated with Lx to the storage system 102.
The caches 170A-170N improve I/O performance of the storage clients 104A-104N. The technology described herein can function with different types of caches, including those stored in volatile memory, non-volatile memory (e.g., storage class memory, or battery-backed DRAM), flash, disk, or some combination of these technologies.
The storage host device 108 may be connected to the storage devices 112A-112M via a storage network switch 110, which can be a Serial Attached SCSI (SAS) storage network switch or a fiber distributed data interface (FDDI), for example. It is noted that, within the network data storage environment, any other suitable numbers of storage servers and/or mass storage devices, and/or any other suitable network technologies, may be employed. Although
The storage host device 108 can make some or all of the storage space on the mass storage devices 112A-112M available to the client systems 104A-104N e.g., in a conventional manner. For example, a mass storage device (one of 112A-112M) can be implemented as an individual disk, multiple disks (e.g., a RAID group) or any other suitable mass storage device(s). The storage host device 108 can communicate with the client systems 104A-104N according to well-known protocols, e.g., the Network File System (NFS) protocol or the Common Internet File System (CIFS) protocol, to make data stored at storage devices 112A-112M available to users and/or application programs.
The storage host device 108 can present or export data stored at mass storage device 112A-112M as volumes (also referred to herein as storage volumes) to one or more of the client systems 104A-104N. On or more volumes can be managed as a single Serial Attached SCSI (SAS) domain, for example. In various embodiments, a “file system” does not have to include or be based on “files” per se as its units of data storage. For example, the units of storage can be objects.
Various functions and configuration settings of the storage host device 108 and the mass storage subsystem 114 can be controlled from a management console 116 coupled to the network 106.
The hosts 208A-208N can be configured to include several modules, including an N-module 214, a D-module 216, and an M-host 218 (each of which can be implemented by using a separate processor executable module) and an instance of a replicated database (RDB) 220. In the illustrated embodiment, host 208A includes an N-module 214A, a D-module 216A, and an M-host 218A; host 208N includes an N-module 214N, a D-module 216N, and an M-host 218N; and so forth. The N-modules 214A-214N include functionality that enables hosts 208A-208N, respectively, to connect to one or more of the client systems 204 over the network 206, while the D-modules 216A-216N provide access to the data stored at storage devices in storage devices 212A-212L. The M-hosts 218 provide management functions for the clustered storage system 202 including, e.g., snapshotting, deduplication, and encryption. Accordingly, the hosts 208A-208N in the clustered storage system can provide the functionality of a storage server.
In various embodiments, RDBs 220A-220N are instances of a database that are replicated throughout the cluster. For example, hosts 208A-208N can include instances of the RDBs 220A-220N. The RDBs 220A-220N can provide cluster-wide storage information used by hosts 208A-208N, including a volume location database (VLDB) (not illustrated). The VLDB is a database that indicates the location within the cluster of volumes in the cluster and is used by the hosts 208A-208N to identify the appropriate mass storage devices in storage devices 212A-212L for any given volume to which access is requested. The various instances of the RDBs 220A-220N can be updated regularly to bring them into synchronization with each other.
A switched virtualization layer including multiple virtual interfaces (VIFs) 222A-222N can be provided between the respective N-modules 214A-214N and the client systems 204A-204M, enabling the storage devices in storage devices 212A-212L associated with the hosts 208A-208N to be presented to the client systems as a single shared storage pool.
The clustered storage system 202 can be organized into any suitable number of virtual servers (also referred to as “vservers”), in which one or more vservers represent a single storage system namespace with separate network access. In various embodiments, each vserver has a user domain and a security domain that are separate from the user and security domains of other vservers. In some other embodiments, two or more vservers can have a common user domain and a common security domain. Moreover, a vserver can be associated with one or more VIFs 222A-222N and can span one or more physical hosts, each of which can hold one or more VIFs 222A-222N and storage associated with one or more vservers. Client systems can access the data on a vserver from any host of the clustered system, but generally access vservers via the VIFs 222A-222N associated with that vserver. It is noteworthy that the embodiments described herein are not limited to the use of vservers.
The hosts 208A-208N and the storage devices can be interconnected by a cluster switching fabric 210, which can be embodied as one or more storage network switches, for example. The N-modules 214A-214N and D-modules 216A-216N cooperate to provide highly-scalable storage system architecture implementing various embodiments of the technology. Although an equal number of N-modules and D-modules are illustrated in
In a shared storage environment as illustrated in
To improve efficiency, a storage server may communicate only relevant duplication information that is relevant to the storage clients. Storage servers typically manage many times more data than storage clients. A storage server may be aware of many data duplicates that are of no interest to a storage client (e.g., because that storage client does not issue read requests for some of the duplicated content). As an example, consider a zero block (a block containing all zero bits). In modestly sized data sets with several gigabytes of data, the zero block can be duplicated hundreds of thousands of times. Storage servers can manage petabytes of data. It is inefficient for a storage client to receive a large amount of information about the data duplications that are irrelevant to the storage client.
The technology can also be employed with server virtualization, which stores operating systems of the virtual servers in storage servers. Multiple storage servers can contain the same or similar versions of operating systems or portions (e.g., files or blocks) thereof. Storing the same or similar versions of an operating system of the virtual servers results in a large number of duplicated data because a large amount of the operating system data (e.g., files) is common to them all. A storage client may request data via multiple virtual servers from the storage server, and therefore attempt to retrieve duplicate data.
Storage servers can select and communicate lists of duplicate blocks (or other units of duplicate data, e.g., logical unit number) to storage clients. For example, a storage server can include or “piggyback” a duplication list with the response to a read request. When a storage client 104A reads location Lx, the response from the storage server 102 can contain the contents of Lx and a list of locations where those contents are duplicated (e.g. Ly). As another example, a storage client can query the storage server for the duplication. The storage server can determine which data is relevant to the storage client and communicate a duplication list for the relevant data to the storage client.
The computing device 300 can further include a memory 304, a network adapter 310, a cluster access adapter 312 and a storage adapter 314, all interconnected by an interconnect 308. Interconnect 308 may include, for example, a system bus, a Peripheral Component Interconnect (PCI) bus, a HyperTransport or industry standard architecture (ISA) bus, a small computer system interface (SCSI) bus, a universal serial bus (USB), or an Institute of Electrical and Electronics Engineers (IEEE) standard 1394 bus (sometimes referred to as “Firewire”) or any other data communication system.
The cluster access adapter 312 includes multiple ports adapted to couple the computing device 300 to other host devices. In the illustrated embodiment, Ethernet can be used as the clustering protocol and interconnect media, although other types of protocols and interconnects may be utilized within the cluster architecture described herein. In various alternative embodiments in which the N-modules and D-modules are implemented on separate storage systems or computers, the cluster access adapter 312 can be utilized by the N-module and/or D-module for communicating with other N-modules and/or D-modules of the cluster.
The computing device 300 can be embodied as a single- or multi-processor storage system executing a storage operating system 306 that can implement a high-level module, e.g., a storage manager, to logically organize the information as a hierarchical structure of named directories, files and special types of files called virtual disks (hereinafter generally “blocks”) at the storage devices. For example, one processor 302 can execute the functions of an N-module on a node while another processor 302 executes the functions of a D-module on the node.
The memory 304 can comprise storage locations that are addressable by the processor(s) 302 and adapters 310, 312, and 314 for storing processor-executable instructions and/or data structures. The processor 302 and adapters 310, 312, and 314 may, in turn, comprise processing elements and/or logic circuitry configured to execute the software code and manipulate the data structures. The storage operating system 306, portions of which are typically resident in memory and executed by the processors(s) 302, functionally organizes the computing device 300 by (among other things) configuring the processor(s) 302 to invoke storage operations in support of the storage service provided by a node. It will be apparent to those skilled in the art that other processing and memory implementations, including various computer readable storage media, may be used for storing and executing program instructions pertaining to the technology.
The network adapter 310 can include multiple ports to couple the computing device 300 to one or more clients over point-to-point links, wide area networks, virtual private networks implemented over a public network (e.g. the Internet) or a shared local area network. The network adapter 310 thus can include the mechanical, electrical and signaling circuitry needed to connect the computing device 300 to the network. Illustratively, the network can be embodied as an Ethernet network or a Fibre Channel (FC) network. A client can communicate with a node over the network by exchanging discrete frames or packets of data according to pre-defined protocols, e.g., TCP/IP.
The storage adapter 314 can cooperate with the storage operating system 306 to access information requested by a client. The information may be stored on any type of attached array of writable storage media, e.g., magnetic disk or tape, optical disk (e.g., CD-ROM or DVD), flash memory, solid-state disk (SSD), electronic random access memory (RAM), micro-electro mechanical and/or any other similar media adapted to store information, including data and parity information. For example, as illustrated in
Storage of information on mass storage devices in storage devices 212A-212L can be implemented as one or more storage volumes that include a collection of physical storage disks cooperating to define an overall logical arrangement of volume block number (VBN) space on the volume(s). The mass storage devices in storage devices 212A-212L can be organized as a RAID group. One or more RAID groups can form an aggregate. An aggregate can contain one or more volumes and/or file systems.
The storage operating system 306 facilitates clients' access to data stored on the storage devices. In various embodiments, the storage operating system 306 implements a write-anywhere file system that cooperates with one or more virtualization modules to “virtualize” the storage space provided by storage devices. For example, a storage manager (e.g. as illustrated in
In the illustrated embodiment, the storage operating system 306 includes multiple functional layers organized to form an integrated network protocol stack or, more generally, a multi-protocol engine 416 that provides data paths for clients to access information stored on the mass storage devices using block and file access protocols. The multi-protocol engine 416 in combination with underlying processing hardware also forms an N-module 430. The multi-protocol engine 416 includes a network access layer 404 that includes one or more network drivers that implement one or more lower-level protocols to enable the processing system to communicate over the network 206, e.g., Ethernet, Internet Protocol (IP), Transport Control Protocol/Internet Protocol (TCP/IP), Fibre Channel Protocol (FCP) and/or User Datagram Protocol/Internet Protocol (UDP/IP). The multi-protocol engine 416 can also include a protocol layer 402 that implements various higher-level network protocols, e.g., Network File System (NFS), Common Internet File System (CIFS), Hypertext Transfer Protocol (HTTP), Internet small computer system interface (iSCSI), etc. Further, the multi-protocol engine 416 can include a cluster fabric (CF) interface module 400A that implements intra-cluster communication with other D-modules and/or N-modules.
In addition, the storage operating system 306 includes a set of layers organized to form a backend server 412 that provides data paths for accessing information stored on the storage devices in storage devices. The backend server 412 in combination with underlying processing hardware also forms a D-module 440. To that end, the backend server 412 includes a storage manager module 406 that can manage a number of storage volumes, a RAID system module 408 and a storage driver system module 410.
The storage manager 406 can manage a file system (or multiple file systems) and serve client-initiated read and write requests. The RAID system 408 manages the storage and retrieval of information to and from the volumes/disks in accordance with a RAID redundancy protocol, e.g., RAID-4, RAID-5, or RAID-DP, while the storage driver system 410 implements a disk access protocol e.g., SCSI protocol, Serial Attached SCSI (SAS) protocol or FCP.
The backend server 412 also includes a CF interface module 400B to implement intra-cluster communication 414 with other N-modules and/or D-modules. In various embodiments, the CF interface modules 400A and 400B can cooperate to provide a single domain across the storage system. Thus, a network port of an N-module that receives a client request can access any data within the single domain located on any mass storage device in any storage device.
The CF interface modules 400A and 400B implement the CF protocol to communicate file system commands among the modules of cluster over the cluster switching fabric (e.g. 210 in
In operation of a storage host device, a request from a client can be forwarded as a packet over a network to the node, where it is received at a network adapter (e.g. 310 in
The data request/response “path” through the storage operating system 306 as described above can be implemented in general-purpose programmable hardware executing the storage operating system 306 as software or firmware. Alternatively, it can be implemented at least partially in specially designed hardware. That is, in an alternate embodiment of the technology, some or all of the storage operating system 306 is implemented as logic circuitry embodied within a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC), for example.
The N-module 430 and D-module 440 can be implemented as processing hardware configured by separately-scheduled processes of storage operating system 306. However, in an alternate embodiment, the modules may be implemented as processing hardware configured by code within a single operating system process. Communication between an N-module 430 and a D-module 440 can thus be effected through the use of a message passing between the modules although, in the case of remote communication between an N-module and D-module of different nodes, such message passing occurs over a cluster switching fabric. The message-passing mechanism provided by the storage operating system to transfer information between modules (processes) can be the Inter Process Communication (IPC) mechanism. The protocol used with the IPC mechanism is illustratively a generic file and/or block-based “agnostic” CF protocol that comprises a collection of methods/functions constituting a CF API.
In various embodiments, the cache can generate or update an address map based on the received duplication list, e.g., at a server computing device, when performing I/O commands.
At decision block 510 after determining the command is a write command, the cache determines whether there is cache hit, which means the requested data is contained in the cache. If there is a cache hit, the old data in the cache will be invalidated at block 515, because the old data cannot be used for future read commands after the write command changes the content of the data (e.g. data block) associated with the write command. If there is no cache hit, the process continues to block 530 to perform the I/O command (e.g., write the data).
At decision block 520 after determining the command is a read command, the cache determines whether there is cache hit, which means the requested data is contained in the cache. If there is a cache hit, the cache fulfills the read command by reading the requested data from the cache itself at block 590. If there is no cache hit, the process 500 continues to block 530 to perform the I/O command (e.g., read the data from storage).
For both read miss and write, at block 530, the process 500 performs the I/O command by, e.g., sending a data request to a storage server. Once the requested data is retrieved from, e.g., the storage server, the process 500 determines whether the same data is already in the cache at decision block 540. If the data is not in the cache, the process 500 inserts the requested data into the cache at block 550. If the data has already been in the cache, at block 560, the process 500 adds a reference to the duplicated data in the address map.
The address map data structure records the cache location of a storage address. Multiple storage addresses can reference the same cache location when deduplication is enabled (shown in
Those skilled in the art will appreciate that the logic illustrated in
In various embodiments, the server duplication list can be implemented in the cache data structures.
The cache header array 610 can include multiple header fields 612A-612N. Each of the header fields 612A-612N can include, e.g., a reference to block addresses that have the same (e.g., duplicate) content, the number of the block addresses, and optionally a fingerprint of the block data (e.g. a SHA256 fingerprint or Adler-32 fingerprint). A storage server can communicate duplication information to the storage client. Accordingly, block addresses associated with the duplicate content can be saved in the cache header array 610, regardless of whether the duplicate content has been stored in the cache 680 or not. In various embodiments, the cache header array 610 may only store block addresses associated with the duplicate data that are stored in the cache 680.
The cache 680 can be a deduplicated cache. In deduplicated cache, all blocks on the cache may be unique. All or at least some of the block addresses that point to each unique block can be in an efficient way to save memory consumption. Data fingerprints can be used to detect duplication. The fingerprint map 630 keeps track of unique fingerprints of data stored in the cache 680. When data is read from or written to a storage server, the data is inserted into the cache 680. The fingerprint of the data is calculated and compared to the entries of the fingerprint map 630. If the fingerprint of the data exists in the fingerprint map 630, a copy of the same data is already in the cache 680.
Fingerprints can be further used to validate data consistency. When there is a read hit and data is retrieved from the cache 680, fingerprints saved in the header are used to validate the data consistency by comparing it to a fingerprint generated from the retrieved data.
As illustrated in
The process 700 continues to block 720, where the storage server determines a working set of storage objects for a client. The working set includes one or more storage objects stored at the storage server. For example, the storage objects of the working set can be objects that are referenced (e.g., for reading or writing) by the client. The working set can be determined by monitoring data access pattern of the client. For example, the working set can be determined by tracking input and output operations on the storage server for the client.
After determining the working set, at block 730, the storage server generates a duplication list, wherein the data chunks associated with the storage addresses of the duplication list are included in the working set of storage objects for the client. In other words, duplicate data chunks that are irrelevant to the working set of the client are excluded (also referred to as pruned) from the duplication list.
At block 740, responding to the read request, the storage server transfers to the storage client device data responding to the read request and the duplication list such that the storage client device therefore can avoid requesting duplicate data chunks from the storage server by checking the duplication list. The duplication list identifies data chunks that have the same data content. If the same data content is stored in a cache of the storage client device, the storage client device can locally satisfied data read operation for these data chunks using the data content stored in its cache. In other words, the storage client device satisfies a read operation for one of the multiple data chunks by retrieving the same data content from the cache of the storage client device, without sending a data request to the storage server.
Although the sample process 700 uses data chunks as the data granularity for the duplication list, the data granularity can be data block, Logic Unit Numbers (LUNs) or other types of data units.
Once the storage client receives the duplication list, the storage client improves the local cache hit rate based on the duplication list.
Optionally at block 812, the storage client identifies a policy regarding insertion of entries of duplicate addresses from the duplication list to the address map of the storage client. For example, the default policy can be that all addresses from the duplication list should be inserted to the address map. Another policy can be, e.g., that the storage client only records addresses from the duplication list to the address map if the address map is occupying less than an amount of memory determined by a threshold value. At block 814, the storage client insert the entries of duplicate addresses from the duplication list to the address map of the storage client according to the policy.
At block 820, the storage client receives a read operation including a storage address. The storage address identifies a data chunk on the storage server that the read operation is to retrieve. At decision block 830, the storage client determines whether the data chunk is stored in the local cache component, e.g., by comparing the storage address to data structures for cached data chunks. This can be performed, e.g., by an operation module of the storage client. The cache component can include a solid-state drive (e.g. a flash-based solid-state drive) or other hardware media such as storage class memory. If the data chunk is cached, at decision block 840, the storage client retrieves the data chunk for the read operation directly from the cache instead of sending a request to the storage server.
If the data chunk is not cached, at decision block 850, the storage client further determines whether there is at least one storage address from the address map that is associated with a data chunk duplicate to the data chunk associated with the storage address of the read operation. If so, these two data chunks are duplicates, and the storage client retrieves the cached data chunk for the read operation directly from the cache at block 860.
If there is no duplicate data chunk in the cache identified based on the duplication list, at block 870, the storage client sends a read request to the storage server to retrieve data for the read operation from the storage server.
The duplication list can be pruned by other criteria besides a working set of a client.
At block 920, the storage server reduces (“prunes”) the duplication list based on a content character of the duplicate data chunks. The content character can be a duplication degree. A duplication degree of a data chunk is, or can be calculated from, a number of duplicate data chunks that have the same content. The duplication list can be reduced by removing from the duplication list storage addresses of data chunks having duplication degrees less than a specified duplication degree threshold.
Alternatively, the content character can be a data access frequency. A data access frequency is, or can be calculated from, a number of times that common content of duplicate data chunks has been accessed. The duplication list can be reduced by removing from the duplication list storage addresses of data chunks having data access frequency less than a specified threshold access frequency.
For example, the storage server can use estimate values or average values of the duplication degree and access frequency (e.g. degree and frequency values accurate as of the last hour). By using the estimate values or average values based on historical data, the hardware burden for computing the values is lighter than the burden for computing these on the fly for every request. Pre-computing the duplication list is performed in a way that the list is guaranteed to only contain addresses that are still duplicates of the data chunk and subsequent write operations do break the duplication relationship.
Optionally at block 930, the storage server can further prune the duplication list by excluding storage addresses of data chunks containing content that is not in a working set of storage objects for a client, e.g., in a way similar to the process 700. Optionally at block 940, the storage server can further prune the duplication list by excluding storage addresses of data chunks belonging to storage objects to which the client does not have access, in a way similar to the process 1000 disclosed in following paragraphs.
At block 950, the storage server transfers to a storage client device the reduced or pruned duplication list such that the storage client device, by using the duplication list avoids requesting duplicate data chunks from the storage server.
At block 1020, the storage server reduces the duplication list based on an access control profile of a client. The access control profile of the client can include a list of access rights to storage objects stored in the storage server. For example, the duplication list can be reduced by excluding references to data chunks belonging to storage objects to which the client does not have access. The storage objects can include data files and/or directories. The client can be represented by at least one storage account from the storage client device. The access control profile of the client may include multiple access rights of the storage account to access storage objects stored at the storage server.
At block 1030, the storage server transfers to a storage client device the duplication list such that a cache of the storage client device can serve a read request for a first reference by providing a duplicate data chunk associated with a second reference, wherein the first and the second references are identified by the duplication list as references for duplicate data chunks.
Those skilled in the art will appreciate that the logic illustrated in
The network protocols used to communicate duplication lists between a storage client and a storage server can be based on standardized protocols or specialized protocols. The technology described herein is independent of specific protocol mechanisms. Industry standard protocols or proprietary protocols can be used to implement the technology.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims. Accordingly, the invention is not limited except as by the appended claims.
