N/A
This invention is generally related to data storage, and more particularly to replicated data storage systems.
Networked computing resources are widely used and relied upon by enterprises. Referring to
In accordance with an aspect of the invention, an apparatus comprises: first and second physical data storage devices upon which are stored first and second copies of a data set, wherein the first copy of the data set is maintained in an uncompressed state and the second copy of the data set is maintained in a compressed state, and wherein a first member of the data set is stored uncompressed on the first data storage device and stored compressed on the second data storage device.
In accordance with another aspect of the invention, a method comprises: storing first and second copies of a data set, including maintaining the first copy of the data set in an uncompressed state and maintaining the second copy of the data set in a compressed state, and storing a first member of the data set uncompressed on a first data storage device and storing the first member of the data set compressed on a second data storage device.
In accordance with another aspect of the invention, a computer program stored on a non-transitory computer readable medium comprises: logic which stores first and second copies of a data set, including logic which maintains the first copy of the data set in an uncompressed state and maintains the second copy of the data set in a compressed state, and logic which stores a first member of the data set uncompressed on a first data storage device and stores the first member of the data set compressed on a second data storage device.
An advantage associated with at least one embodiment of the invention is RAID 1 reliability with reduced storage capacity requirements because the compressed copy requires less storage capacity, e.g., 1/2 nB where the uncompressed copy is nB and the data compresses at 50%. More particularly, storage space consumption is similar to RAID 5-3+1. Furthermore, the existence of the uncompressed copy obviates the need for decompressing data for every READ operation. Compression striping in which the compressed and uncompressed data is distributed across two or more logical volumes or storage devices can be used to reduce data access response time. Further, delayed compression can be used to reduce the time required to perform WRITE ops. Compression may also be performed based on resource availability or storage tier hierarchy. These and other advantages of the invention will be more apparent from the detailed description and the drawing.
Various aspects of the invention may be implemented partially or completely in software using computer program code. The computer program code is stored on non-transitory computer-readable memory and utilized by processing hardware to implement instructions corresponding to certain steps. The program code may be provided as a computer program product or be integrated into network storage equipment.
Referring to
Referring to
A variation on the tier based compression described above is to store the copies on different tiers. For example, one copy of a RAID 1 half compressed data group could be stored at Tier 1 while the other copy is stored on Tier 2. It should be noted however that none of the embodiments are limited to any particular storage media, and cache, RAM and other types of memory and storage could be utilized.
The determination of whether to store a compressed or uncompressed version of the data can be decided independently for each tier and each data extent or sub-extent based on the cost and availability of storage on that tier when compared with the cost and availability of CPU cycles to compress the WRITEs and uncompress future expected READs; as well as the expected future READ and WRITE access for the data from this tier. This decision can be static, i.e. decided on a fixed basis that is change relatively infrequently; or dynamic, i.e. decided at a more frequent interval or real-time. For example, the system may determine that data that is rarely accessed but which needs quick availability for those relatively rare situations when it is required would be stored compressed on both a fast storage tier and a slow storage tier. Examples of such data is data that is needed when a client machine boots up, or data that is needed for a few minutes at the beginning of every business day.
The system may choose to change the compression of one or the other of the copies at a later time based on changes in the availability of compute and storage resources or changes in the anticipated READ and WRITE access patterns for the data.
The system may choose to not compress while there is plenty of unused space on the storage medium and then compress one or both of the copies when there is less storage space available. Conversely, the system may choose to uncompress data when additional storage space becomes available for the desired storage medium.
When the storage system needs to move data internally, the storage system can move the data uncompressed. This is especially true if the source copy is compressed and the target copy is intended to be compressed. Since the content of the data is not being interpreted by the storage system, the storage system can potentially move the data more efficiently by keeping the data compressed during the move without uncompressing the data. Similarly, a plurality of storage systems that work together in a federated way can transfer compressed copies of the data between them without an intermediate uncompressed copy being made. For example, the storage systems in such a federation may be working with each other to achieve data replication for increased availability, increased performance, load-balancing, backup, migration or other purposes.
A variation on this scheme is to have more than one compression algorithm available. Compression algorithms often have different tradeoffs between cpu-usage and compression ratios. The system could choose between no-compression or using a particular compression scheme based on the criteria given above.
The techniques described above are not limited to use with RAID and can be implemented without complying with a RAID level or used in general for replicated storage systems including but not limited to Hadoop and similar file systems. In systems where data has n replicas, some subset of those replicas could be compressed, e.g., depending on whether the scheduler/resource manager determines that storage or compute is the rarer commodity at a particular node, and how often a particular copy of the data is predicted to be decompressed.
While the invention is described through the above exemplary embodiments, it will be understood by those of ordinary skill in the art that modification to and variation of the illustrated embodiments may be made without departing from the inventive concepts herein disclosed. Moreover, while the embodiments are described in connection with various illustrative structures, one skilled in the art will recognize that the system may be embodied using a variety of specific structures. Accordingly, the invention should not be viewed as limited except by the scope and spirit of the appended claims.
Number | Name | Date | Kind |
---|---|---|---|
5694619 | Konno | Dec 1997 | A |
5720025 | Wilkes et al. | Feb 1998 | A |
6442659 | Blumenau | Aug 2002 | B1 |
6879266 | Dye et al. | Apr 2005 | B1 |
7103740 | Colgrove et al. | Sep 2006 | B1 |
7424592 | Karr | Sep 2008 | B1 |
7434091 | Karr | Oct 2008 | B1 |
7500070 | Schilling et al. | Mar 2009 | B2 |
7512862 | Taylor | Mar 2009 | B1 |
7631151 | Prahlad et al. | Dec 2009 | B2 |
20030041211 | Merkey | Feb 2003 | A1 |
20050076263 | Tomita | Apr 2005 | A1 |
20050108484 | Park | May 2005 | A1 |
20050168934 | Wendel | Aug 2005 | A1 |
20060101130 | Adams | May 2006 | A1 |
20070136541 | Herz | Jun 2007 | A1 |
20080022058 | Nadathur et al. | Jan 2008 | A1 |
20100077168 | Arakawa | Mar 2010 | A1 |
20100115198 | Jess | May 2010 | A1 |
20110239041 | S et al. | Sep 2011 | A1 |
20120131293 | Benhase et al. | May 2012 | A1 |
Entry |
---|
K. Gopinath, Nitin Muppalaneni, N. Suresh Kumar, and Pankaj Risbood. 2000. A 3-tier RAID storage system with RAID1, RAID5 and compressed RAID5 for Linux. In Proceedings of the annual conference on USENIX Annual Technical Conference (ATEC '00). USENIX Association, Berkeley, CA, USA, 30-30. |
Paul Massiglia; “The RAID book: A Storage System Technology Handbook”; 1997; pp. 139-140. |
K. Gopinath, Nitin Muppalaneni, N. Suresh Kumar, and Pankaj Risbood. 2000. A 3-tier RAID storage system with RAID1, RAID5 and compressed RAID5 for Linux. In Proceedings of the annual conference on USENIX Annual Technical Conference (ATEC '00). USENIX Association, Berkeley, CA, USA, 30-30. (Year: 2000). |
David A. Patterson, Garth Gibson, and Randy H. Katz. A case for redundant arrays of inexpensive disks (RAID). In Proceedings of the 1988 ACM SIGMOD international conference on Management of data (SIGMOD '88), Haran Boral and Per-Ake Larson (Eds.). (Year: 1988). |