Patent Application
20200034452
Data deduplication (also referred to simply as “deduplication”) is a space-saving technology intended to eliminate redundant (duplicate) data (such as, files) on a data storage system. By saving only one instance of a file, disk space can be significantly reduced. For example, if a file of size 10 megabytes (MB) is stored in ten folders of each employee in an organization that has ten employees. Thus, 100 megabytes (MB) of the disk space is consumed to maintain the same file of size 10 megabytes (MB). Deduplication ensures that only one complete copy is saved to a disk. Subsequent copies of the file are only saved as references that point to the saved copy, such that end-users still see their own files in their respective folders. Similarly, a storage system may retain 200 e-mails, each with an attachment of size 1 megabyte (MB). With deduplication, the disk space needed to store each attachment of size 1 megabyte (MB) is reduced to just 1 megabyte (MB) from 200 megabyte (MB) because deduplication only stores one copy of the attachment.
Data deduplication can operate at a file or a block level. File deduplication eliminates duplicate files (as in the example above), but block deduplication processes blocks within a file and saves unique copy of each block. For example, if only a few bytes of a document or presentation or a file are changed, only the changed blocks are saved. The changes made to few bytes of the document or the presentation or the file does not constitute an entirely new file.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described herein in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
One aspect may provide a method for dual layer deduplication for a file system running over a deduplication block storage system. The method includes accessing a file by a file system driver of the file system. The file is stored as one of a plurality of files in the file system. The method further includes breaking the file down into multiple blocks and generating chunks from the blocks. For at least one of the chunks, a trail of zeros is added until a size of a respective one of the chunks is a multiple of a block size of the deduplication block storage system
Another aspect may provide a system for dual layer deduplication for a file system running over a deduplication block storage system. The system includes a memory having computer-executable instructions and a processor. The processor executes the computer-executable instructions. When executed by the processor, the computer-executable instructions cause the processor to perform operations. The operations include accessing a file by a file system driver of the file system. The file is stored as one of a plurality of files in the file system. The operations further include breaking the file down into multiple blocks and generating chunks from the blocks. For at least one of the chunks, a trail of zeros is added until a size of a respective one of the chunks is a multiple of a block size of the deduplication block storage system.
Another aspect may provide a computer program product embodied on a non-transitory computer readable medium. The computer program product includes instructions that, when executed by a computer, causes the computer to perform operations. The operations include accessing a file by a file system driver of the file system. The file is stored as one of a plurality of files in the file system. A further aspect includes breaking the file down into multiple blocks and generating chunks from the blocks. For at least one of the chunks, a trail of zeros is added until a size of a respective one of the chunks is a multiple of a block size of the deduplication block storage system.
Objects, aspects, features, and advantages of embodiments disclosed herein will become more fully apparent from the following detailed description, the appended claims, and the accompanying drawings in which like reference numerals identify similar or identical elements. Reference numerals that are introduced in the specification in association with a drawing figure may be repeated in one or more subsequent figures without additional description in the specification in order to provide context for other features. For clarity, not every element may be labeled in every figure. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating embodiments, principles, and concepts. The drawings are not meant to limit the scope of the claims included herewith.
Before describing embodiments of the concepts, structures, and techniques sought to be protected herein, some terms are explained. The following description includes a number of terms for which the definitions are generally known in the art. However, the following glossary definitions are provided to clarify the subsequent description and may be helpful in understanding the specification and claims.
As used herein, the term “storage system” is intended to be broadly construed so as to encompass, for example, private or public cloud computing systems for storing data as well as systems for storing data comprising virtual infrastructure and those not comprising virtual infrastructure. As used herein, the terms “client,” “host,” and “user” refer, interchangeably, to any person, system, or other entity that uses a storage system to read/write data. In some embodiments, the term “storage device” may also refer to a storage array including multiple storage devices. In certain embodiments, a storage medium may refer to one or more storage mediums such as a hard drive, a combination of hard drives, flash storage, combinations of flash storage, combinations of hard drives, flash, and other storage devices, and other types and combinations of computer readable storage mediums including those yet to be conceived. A storage medium may also refer both physical and logical storage mediums and may include multiple level of virtual to physical mappings and may be or include an image or disk image. A storage medium may be computer-readable, and may also be referred to herein as a computer-readable program medium.
In certain embodiments, the term “I/O request” or simply “I/O” may be used to refer to an input or output request, such as a data read or data write request.
In certain embodiments, a storage device may refer to any non-volatile memory (NVM) device, including hard disk drives (HDDs), solid state drivers (SSDs), flash devices (e.g., NAND flash devices), and similar devices that may be accessed locally and/or remotely (e.g., via a storage attached network (SAN) (also referred to herein as storage array network (SAN)).
In certain embodiments, a storage array (sometimes referred to as a disk array) may refer to a data storage system that is used for block-based, file-based or object storage, where storage arrays can include, for example, dedicated storage hardware that contains spinning hard disk drives (HDDs), solid-state disk drives, and/or all-flash drives (e.g., the XtremIO all flash drive, available from DELL/EMC of Hopkinton Mass.). In certain embodiments, a data storage entity may be any one or more of a file system, object storage, a virtualized device, a logical unit, a logical unit number, a logical volume, a logical device, a physical device, and/or a storage medium.
In certain embodiments, a physical storage unit may be a physical entity, such as a disk or an array of disks, for storing data in storage locations that can be accessed by address, where physical storage unit is used interchangeably with physical volume. In certain embodiments, a data storage entity may be any one or more of a file system, object storage, a virtualized device, a logical unit, a logical unit number, a logical volume, a logical device, a physical device, and/or a storage medium.
In certain embodiments, an image may be a copy of a logical storage unit at a specific point in time. In certain embodiments, a clone may be a copy or clone of the image or images, and/or drive or drives of a first location at a second location. In some embodiments, a clone may be made up of a set of objects.
In certain embodiments, a snapshot may refer to differential representations of an image, i.e. the snapshot may have pointers to the original volume, and may point to log volumes for changed locations. In certain embodiments, a snapshot may refer to differential representations of the state of a system. Snapshots may be combined into a snapshot array, which may represent different images over a time period or different states of a system over a time period.
In certain embodiments, a journal may be a record of write transactions (e.g., I/O data) issued to a storage system, which may be used to maintain a duplicate storage system, and to roll back the duplicate storage system to a previous point in time. In some embodiments, each entry in a journal contains, apart from the I/O data itself, I/O metadata that can include information such as a volume identifier (ID), the I/O block offset within the volume, the I/O length, and a time stamp of the I/O.
In certain embodiments, XtremIO, available from Dell EMC of Hopkinton, Mass.) is a type of content addressable storage array that uses all flash technology. Flash, as is understood, is a solid-state (SS) random access media type that can read any address range with no latency penalty, in comparison to a hard disk drive (HDD) which has physical moving components which require relocation when reading from different address ranges and thus significantly increasing the latency for random I/O data.
In certain embodiments, an X-page is a predetermined-size aligned chunk as the base unit for memory and disk operations. In certain embodiments described in the present description, the X-Page size is referred to as having 4 KB; however other smaller or larger values can be used as well, and nothing in the design is limited to a specific value.
In certain embodiments, a logical X-page address is the logical address of an X-page, containing a LUN identifier as well as the offset of the X-page within the LUN.
In certain embodiments, a data protection strategy that can be advantageous for use with computer systems, especially networked storage systems, is checkpointing. A checkpoint, as used herein, contains a consistent point in time image of an entire system, including configuration, logical volume mapping metadata, physical on disk layout metadata, and actual user data. In certain embodiments, a checkpoint preserves the state of a system at a given point in time by saving one or more snapshots of, for example, a file system, or an application at one or more points in time. A checkpoint can preserve a snapshot of an application's state, so that it can restart from that point in case of failure, which can be useful for long running applications that are executed in failure-prone computing systems. If a checkpoint is used, an application periodically writes large volumes of snapshot data to persistent storage in an attempt to capture its current state. Thus, if there is a failure, the application can recover by rolling-back its execution state to a previously saved checkpoint.
In certain embodiments, a “checkpoint” refers at least to an entity created by a checkpoint process, where the checkpoint process performs actions to preserve the state of an apparatus, system, or other entity (including software entities) at a particular time. Advantageously, a checkpoint includes information such as user data, the configuration of the apparatus, user metadata, and other information related to the internal state of the apparatus or system. For example, some storage systems (including XtremIO), in accordance with certain embodiments herein, also provide some kind of checkpoint feature, to provide an ability to preserve system state including user data and metadata at some defined point in time in order to restore this state after system malfunction or corruption. In certain embodiments, the checkpoint corresponds to a frozen, immutable re representation of the state of a system or apparatus at certain point in time, including user data, metadata, and the system configuration. In certain embodiments, the checkpoint is stored in a dedicated, reserved location within the system. In certain embodiments, the checkpoint is able to be created in an online, dynamic environment, where the checkpoint creation is transparent to entities having I/O interactions with the system.
For a file system, the accuracy and consistency of a file system is necessary to relate applications and data, so a checkpoint provides a way to provide periodic backup of file server state to allow system recovery in the event of faults or failures. When data corruption is detected, one of the checkpoints can be used for file system recovery. Similarly, a checkpoint, in a virtualization context, is a snapshot of the state of a virtual machine. Like a restore point in MICROSOFT WINDOWS operating systems, a checkpoint allows an administrator to restore an entity (e.g., a computer system, a file system, an application, a virtual machine, etc.) to a previous state. Checkpoints also can be used to create backups before conducting updates. Should an update fail or cause problems, an administrator can return the virtual machine to its state prior to the update. A recover action is used to return the system to the checkpoint state.
It is envisioned that at least some embodiments herein are usable with one or more of the embodiments described in certain commonly owned U.S. patents, including: U.S. Pat. No. 8,990,495 (“Method and System for Storing Data in RAID Memory Devices”); U.S. Pat. No. 9,104,326 (“Scalable Block Data Storage Using Content Addressing”) (hereinafter “'326 patent”); and U.S. Pat. No. 9,606,870 (“Data Reduction Techniques in a Flash-Based Key/Value Cluster Storage”), each of which is hereby incorporated by reference.
While vendor-specific terminology may be used herein to facilitate understanding, it is understood that the concepts, techniques, and structures sought to be protected herein are not limited to use with any specific commercial products. In addition, to ensure clarity in the disclosure, well-understood methods, procedures, circuits, components, and products are not described in detail herein.
The phrases, “such as,” “for example,” “e.g.,” “exemplary,” and variants thereof, are used herein to describe non-limiting embodiments and are used herein to mean “serving as an example, instance, or illustration.” Any embodiments herein described via these phrases and/or variants is not necessarily to be construed as preferred or advantageous over other embodiments and/or to exclude the incorporation of features from other embodiments. In addition, the word “optionally” is used herein to mean that a feature or process, etc., is provided in some embodiments and not provided in other embodiments.” Any particular embodiment of the invention may include a plurality of “optional” features unless such features conflict.
The data and/or control systems 114, 116 may retain extracts of the data stored in the storage devices 112. In embodiments, the data extracts may be generated by cryptographic hashing of the data content in the data blocks. In embodiments, the extracts may be used for content addressing of the data blocks to the physical storage devices 112.
The second node 104 can include a hash system 117 to generate the hash/extract, which can be referred to as a content fingerprint for the data blocks. The second node 104 can also include a routing system 118, along with a switch interface 111 and a SAN interface 115. The routing system 118 may terminate storage and retrieval operations and distribute commands to control systems 114 that may be selected for the operation in such a way as to retain balanced usage within the system. In the illustrated embodiment, the third node 106 can be similar to the first node 102 and the fourth node 108 can be similar to the second node 108.
The routing systems 118 may use the hash values calculated from data blocks to select control systems 114 for distribution. More particularly, selection of the control system 114 may use hash values, or may rely on the user address and not on the content (hash). The hash value may, however, be used for selecting the data system 116, and for setting the physical location for data storage within the data system.
In example embodiments, control modules 114 can include a C cache 115 and the data modules 116 can include a D cache 117. As explained more fully below, the C cache 115 can include addresses, address hashes, and physical data location information and the D cache 117 can include, for each bucket, a filter, a hash to address, and bucket information.
In some examples, the system 100 may employ more than a single type of memory technology, including a mix of more than one Flash technology (e.g., single level cell (SLC) flash and multilevel cell (MLC) flash), and a mix of Flash and DRAM technologies. In certain embodiments, data mapping may optimize performance and life span by taking advantage of the different access speeds and different write/erase cycle limitations of the various memory technologies.
For a write operation from a host, the routing system 218 can receive the write data and can segment the data stream into data blocks and generate hash values for the data blocks. The hash value can be provided to the control system 214 to determine if the write data is unique. If unique, the hash value can be placed in an address mapping. The control system 214 can pass the hash value to the data system 216, which can assign the hash value to a physical address and write the data block(s) to the SSD at the physical address. In embodiments, the write reference information and/or the access reference information, can be modified, e.g., incremented,
If the hash value generated by the routing system 218 is not unique, the control system 214 can determine that data already exists at the physical address for the hash value. Since the data already exists, the data system 216 can increment the write reference information for the data block. In embodiments, the access reference information can also be modified. The data may not be written to the SSD. Deduplication may refer to a write operation where a hash for a data block is found not be unique and the non-unique data block is not written to physical storage. The reference count for the non-unique hash may be incremented.
As described further herein, the storage system 300 also can include a C (also called logical) cache 317 and a D (also called physical) cache 323. The C cache 317 and/or the D cache 323 can, in certain embodiments, be physical devices configured to store certain data so that future requests for that data can be served faster. Although the C cache 317 and D cache 323 are shown as being part of the storage system, it is understood that the C cache 317 and/or D cache 323 can be located anywhere such that they are accessible quickly to the storage system. Data that is stored within a cache might include data values that have been computed earlier or duplicates of original values that are stored elsewhere. If the requested data is contained in the cache (herein referred to as a cache hit), this request can be served by simply reading the cache, which is comparatively faster than going to other types of memory. On the other hand, if the requested data is not contained in the cache (herein referred to as a cache miss), the data may have to be to be recomputed or fetched from its original storage location, which is comparatively slower. Hence, the greater the number of requests that can be served from the cache, the faster the overall system performance becomes.
The primary memory 318 can be any type of memory having access times that are faster compared to the storage devices 308. In some embodiments, primary memory 318 may be provided as dynamic random-access memory (DRAM). In certain embodiments, primary memory 318 may be provided as synchronous DRAM (SDRAM). In one embodiment, primary memory 318 may be provided as double data rate SDRAM (DDR SDRAM), such as DDR3 SDRAM.
As described above, the control subsystem 302b may be configured to maintain a mapping between I/O addresses associated with data and the corresponding chunk hashes. As shown in
As also described above, the data subsystem 302c may be configured to maintain a mapping between chunk hashes and physical storage addresses (i.e., storage locations within the storage array 306 and/or within individual storage devices 308). This mapping may be maintained using a data structure 314, referred to herein as a “hash to physical address mapping table” or “H2P table,” or “H→P table,” according to some embodiments, where this table, in certain embodiments, includes information similar to that of the aforementioned HMD (hash metadata) and PL (physical layout) tables. In certain embodiments, as described, for example, in the incorporated by reference patents, there also may be a mapping referred to as the H2D or H→D table, where D stands for disk physical layout. In certain embodiments, the H2P table is maintained to route data with different hashes to different D modules. The data subsystem 302c may be also be configured to read and write data from/to the storage array 306 (and/or to individual storage devices 308 therein).
As described above, in a content addressable storage system, data is stored in blocks, for example 16 KB, 8 KB, 4 KB, etc., where each block has a universally unique large hash signature, for example of 20 bytes, which can be saved to disk, e.g., Flash memory. As described herein, hash signatures may be accessed by small in-memory handles (referred to herein as short hash handles, hash handles, or short hashes), for example of 6 bytes. These short hashes may be unique to each volume/array, but not necessarily unique across volumes/arrays. Additional information relating to hash-based replication, computation of hashes, generation and use of short hash handles can be found in U.S. Pat. No. 9,378,106 (“Hash Based Replication”); U.S. Pat. No. 9,208,162 (“Generating a Short Hash Handle”) and U.S. Pat. No. 9,396,243 (“Hash-Based Replication Using Short Hash Handle and Identity Bit”), each of which is hereby incorporated by reference.
In embodiments, address to hash mapping (A2H) maps an address inside a volume to the short hash value of its data. In embodiments, meta data can include for each address the hash value of the content. If the basis for deduplication is 16 KB, then the meta data holds for each address the short hash value of the data to which the address points. In cases where access to the volume is in larger chunks than the size of the basic hash value, the meta data for the address space can be readily cached.
As also noted above, hash to physical disk locations can include for each hash key (e.g., 6 bytes) the location on the disk, and the reference count. Where a storage system uses hash keys of 6 bytes, there may be collisions of data generating the same hash. If there is a collision, a new hash key from a different hash address space is generated for the data when the data is written. This means that the hash to physical disk location table may search for a hash value every time a new write arrives. If the write has the same hash value, there is a need to check the long hash value, and verify if there is a hash collision, or whether it is actually the same data. This means that during every write if the hash to physical disk location table is not in the system memory, there may a need to fetch the meta data of the hash from the disk to verify if such a hash exists. It will be appreciated that meta data structures may consume most of system memory, e.g., DRAM, in the storage system, so that the meta data limits the total size of the storage system.
Modern storage systems, such as XtremIO, leverage flash drives to provide fast reliable de-duplicated storage. One of the challenges of a block based inline deduplication storage system is the fact that they use block aligned deduplication, which is less effective than byte aligned de duplication used by secondary storage arrays like data domain. For block storage it is more difficult to create byte aligned deduplication as IOs arrive in blocks, and it is hard to chunk the data correctly compared to a file system. Chunking refers to a process of splitting a file into pieces called chunks. In data deduplication applications, the process of chunking facilitates duplicate detection performance of the system.
A deduplication storage system, such as XtremIO has two layers for accessing the volume data. As described above, there is a layer mapping from address in the volume to the hash value of the data, and a second layer mapping from the hash value to a location on the disk. The address to hash mapping maps an address inside a volume to the hash value of its data. The metadata includes for each address the hash value of the content. If the basic use of deduplication is 16 KB, then the metadata holds for each address the short hash value (6 bytes) of the data the address points to. In many cases, access to the volume is in larger chunks than the size of the basic hash value. This means that the metadata for the address space can be easily cached, and standard prefetching algorithms work efficiently.
The hash to physical disk location includes for each hash key (6 bytes) the location on the disk and the reference count. Since the system is architected to keep hash keys of 6 bytes, there may be collision of data generating the same hash, if there is a collision a new hash key from a different hash address space is generated for that data, when the data is written. This means that the hash to physical disk location table, must search for a hash value every time a new write arrives, if the write has the same hash value, there is a need to check the long hash value, and verify if there is a hash collision, or it is actually the same data. This means that during every write if the table is not in the system memory, there is a need to fetch the meta data of the hash from the disk and verify if such hash exists.
Many systems today use a file system over the block devices that a block storage exposes. A file system element is a file which is mapped to a list of blocks. In many cases there are files having many parts of data being identical. One good example is a large customer that generates very large video files, and the files are identical except that each video has embedded subtitles which are part of the video. As there may be many countries to which the high quality video is targeted, this means that there is a lot of duplications in the data. The differences in the subtitles cause a drift in the block alignment between the video resulting in little or no deduplication. A byte aligned deduplication system can manage such assignment problems much better than block level deduplication and achieve effective deduplication ratios.
Block aligned deduplication systems typically support compression, which means trailing zeros will have little impact.
Embodiments described herein provide a dual-level deduplication system. In certain cases, an enterprise can utilize different types of storage systems to form a complete data storage environment. In one arrangement, the enterprise can utilize both a block based storage system and a file based storage hardware, such as a VNX™ or VNXe™ system (produced by EMC Corporation, Hopkinton, Mass.). In such an arrangement, typically the file based storage hardware operates as a front-end to the block based storage system such that the file based storage hardware and the block based storage system form a unified storage system.
Turning now to
The file system driver 506 may run on in any file system that utilizes the deduplicated block storage system 504 as a background storage. The file system 502 includes a number of files 508. Each of the files 508 in the file system has two parts: the file data (file contents) 510 and the metadata 512 describing the file structure of the respective file.
Each original file is chunked into multiple pieces (portions) 514 using a variable length chunking algorithm, which may be byte aligned. It is understood that the variable length chunking algorithm can be alternatively bit aligned. At the end of each chunk the system will put trailing zeros until the end of the block (i.e., pad with block until the size of the chunk is a multiple of the block size of the deduplicated block storage), e.g., in XtremIO the deduplicated block storage is 16 KB. The metadata 512 for each file will include the list of locations where the file was chunked.
The chunk size for files can be larger than the chunks for the block deduplication to allow smaller metadata files. Since the chunking algorithm is deterministic, the system can verify if a location is a place for chunking without the metadata indicating this. However, the metadata is needed to be able to access the file randomly.
The modified file system driver 506 will hide the metadata file and allow access only to the data files, leveraging the metadata files to allow random access. The block storage will have correct alignment of the data in the blocks, and the effect of the trailing zeros will be minimal on the space as compression, such as run-length encoding (RLE) and Lempel Ziv (LZ) will reduce the space used by the relevant blocks.
An example technique for two data files 602 and 608 is shown in
The deduplication ratio will be zero, but if chunked correctly and zeros are added in the proper place, the deduplication will be more effective.
The first file 602 will result in chunked blocks 606 having regular block alignment (i.e., no change from the blocks 604). The second file 608 will result in chunked blocks 612. The block storage will not deduplicate between ABCD ABCZ and D000 but the rest of the data will be deduplicated, and the three zeros added after the ‘D’ will be effectively compressed.
Turning now to
In block 706, the process 700 chunks the blocks. At the end of each block the process 700 adds trailing zeros until the size of the chunk is a multiple of a block size of the deduplication block storage. This step may be implemented by the file system driver via a variable length chunking algorithm. The resulting chunked files may be stored in the file system.
Processing may be implemented in hardware, software, or a combination of the two. Processing may be implemented in computer programs executed on programmable computers/machines that each includes a processor, a storage medium or other article of manufacture that is readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and one or more output devices. Program code may be applied to data entered using an input device to perform processing and to generate output information.
The system can perform processing, at least in part, via a computer program product, (e.g., in a machine-readable storage device), for execution by, or to control the operation of, data processing apparatus (e.g., a programmable processor, a computer, or multiple computers). Each such program may be implemented in a high level procedural or object-oriented programming language to communicate with a computer system. However, the programs may be implemented in assembly or machine language. The language may be a compiled or an interpreted language and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program may be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites and interconnected by a communication network. A computer program may be stored on a storage medium or device (e.g., CD-ROM, hard disk, or magnetic diskette) that is readable by a general or special purpose programmable computer for configuring and operating the computer when the storage medium or device is read by the computer. Processing may also be implemented as a machine-readable storage medium, configured with a computer program, where upon execution, instructions in the computer program cause the computer to operate.
Processing may be performed by one or more programmable processors executing one or more computer programs to perform the functions of the system. All or part of the system may be implemented as, special purpose logic circuitry (e.g., an FPGA (field programmable gate array) and/or an ASIC (application-specific integrated circuit)).
Having described exemplary embodiments of the invention, it will now become apparent to one of ordinary skill in the art that other embodiments incorporating their concepts may also be used. The embodiments contained herein should not be limited to disclosed embodiments but rather should be limited only by the spirit and scope of the appended claims. All publications and references cited herein are expressly incorporated herein by reference in their entirety.
Elements of different embodiments described herein may be combined to form other embodiments not specifically set forth above. Various elements, which are described in the context of a single embodiment, may also be provided separately or in any suitable subcombination. Other embodiments not specifically described herein are also within the scope of the following claims.
