1. Field of the Invention
The present invention relates to texture compression techniques.
2. Background of Invention
Compression and decompression intended to minimize the memory size needed to store 2D textures is a promising field of application for these techniques in the 3D graphic domain. This possible field of use is becoming more and more significant as the dimensions and number of these textures tend to increase in real applications. The level of detail tends to increase as required by some applications, such as 3D games, and, without the help of such techniques, memory size and bandwidth for access would tend to require increasing performance levels hardly sustainable in mobile, ultra low power, handheld systems. More to the point, these techniques are becoming increasingly important in wireless phone architectures with 3D games processing capabilities.
For example, assuming a texture dimension of 512×512 pixels 16 bit/color each and a depth of 3, the amount of memory needed is 1.5 M bytes. Assuming 20-30 frames per second, the memory bandwidth is 30 to 45 Mbytes/s.
Additional background information on this topic can be gathered from “Real-Time Rendering” by Tomas Akenine-Möller and Eric Haines, A. K. Peters Ltd, 2nd edition, ISBN 1568811829.
A well-known solution in this scenario was developed by the company S3; the related algorithm is designated S3TC (where TC stands for Texture Compression).
This has become a widely used de-facto standard and is included in the Microsoft DirectX libraries with adhoc API support.
Compression is performed off-line at compile time and the textures are stored in the main memory. Decompression processes act to compress textures accessing the memory run-time. This means that only decompression is implemented in hardware form while compression is not.
Important parameters for the decompression engine are: steps needed to decompress textures and possible parallel operation; low latency between data-access-from-memory and data-out-from the decompression engine.
In order to better understand operation of the S3TC algorithm one may refer to an image in RGB format, where each color component R (Red) or G (Green) or B (Blue) is a sub-image composed by N pixels in the horizontal dimension and M pixels in vertical dimension. If each color component is coded with P bits, the number of bits per image is N*M*3*P.
For example, assuming N=M=256 and P=8, then the resulting size is 1,572,864 bits. If each sub-image R or G or B is decomposed in not-overlapped blocks of Q pixels in the horizontal dimension and S pixel in the vertical dimension, the number of blocks per sub-image is (N*M)/(Q*S) while per image is [3(NM/(Q*S(WM)] and the number of bits per block is [3*(Q*S)]*P. If, for example Q=S=4 and P=8, then the resulting size of each block is 384 bits. If the number of bits per channel is R=5, G=6, B=5 then the resulting size of each block per image is (4*4)*(5+6+5)=256 bits. The S3TC algorithm is able to compress such an amount of data by 6 times when R=8, G=8, B=8 and 4 times when R=5, G=6, B=5. 64 bits compose the resulting compressed block always sent to decompression stage. This number is the results of the coding steps described below assuming Q=S=4.
To sum up, operation of the S3TC algorithm may be regarded as comprised of the following steps:
i) Decompose the R G B image in non overlapped Q=4*S=4 blocks of R G B colors
ii) Consider the following block composed by 16 pixels each one composed by R, G and B color components:
Pij=Rij U Gij U Bij (this denotes the pixel at the ij position the R G B image, and U is the union operator)
iii) Decompose the block above in three sub-blocks called sub-block R, sub-block G and sub-block B as shown hereinbelow, each block including only one color component:
as shown in
Specifically,
iv) Sort in ascending order each sub-block color
v) Detect the black color, which is a pixel made of R=0 and G=0 and B=0
vi) If the black color is not detected, then set a color palette made by
vii) Otherwise, if black color is detected then set a color palette made by
viii) If black color is not detected, define the look-up color palette as
If black color is detected define the color palette as
ix) Associate the following 2 bits code (in boldface, under the palette) to each column of the above palette
x) For each Pij=Rij U Gij U Bij (where i ranges from 1 to Q=4 and j ranges from 1 to S=4) compute the Euclidean distance Dist between it and each look-up color as defined above in vi.a,b,c,d or vii.a,b,c,d depending if black color has been detected or not. Note that the difference is within a homologue color component (between R or G or B).
Dist1=√(|Rij−MinR|2+|Gij−MinG|2+|Bij−MinB|2)
Dist2=√(|Rij−Int1R|2+|Gij−Int1G|2+|Bij−Int1B|2)
Dist3=√(|Rij−Int2R|2+|Gij−Int2G|2+|Bij−Int2B|2)
Dist4=√(|Rij−MaxR|2+|Gij−MaxG|2+|Bij−MaxB|2)
xi) For each Pij=Rij U Gij U Bij find the minimum distance among Dist1, Dist2, Dist3 and Dist4. For example let this be Dist1.
xii) Send to a decoder process the code associated to the color enclosed in the look-up table that has the minimum distance. If it is Dist1 then the code is 00.
xiii) The decoder receives for each Q*S block as shown in
xiv) If Min is received before Max by the decoder, then black has been detected by the encoder otherwise not
xv) As shown in
xvi) As shown in
As stated before, the compression ratio is 6:1 or 4:1. This is because if colors are in R=8 G=8 B=8 format then 384 bits are coded with 64 (384/64=6) and if colors are in R=5 G=6 B=5 format then 256 bits are coded with 64 (256/64=4).
As shown in
However satisfactory the prior art solution considered in the foregoing may be, the need is felt for alternative texture compression/decompression techniques. The aim of the present invention is thus to provide such an alternative technique.
According to the present invention such an object is achieved by means of a method having the features set forth in the claims that follow. The invention also encompasses the decoding process as well as corresponding apparatus in the form of either a dedicated processor or a suitably programmed general-purpose computer (such as a DSP). In that respect the invention also relates to a computer program product directly loadable into the memory of a digital computer and including software code portions for performing the method of the invention when the product is run on a computer.
In brief, the presently preferred embodiment of the invention differs, in one of its aspects, from the S3TC algorithm in the way the reference colors are selected to construct the look-up table. The way of choosing these colors is made adaptive and consists in creating groups of colors for each color component R,G,B and select at first a group from which a representative color for this group is derived. Preferably, each group is composed by any number of colors between 3 up to 15 members. For each of them the median color is chosen as the representative color of the group to which it belongs. For sake of clarity, the median of a set of numbers put in ascending order is the number located in the middle position of them.
For example if the set is (1, 3, 5, 6, 20) then the median is the 3rd value (from right) and is equal to 5.
For each group, an error is computed as the sum of the absolute differences (SAD) between each group member and the representative (the median value of the group) color.
Still preferably, at least two different criteria are used to select the group first and then extract from this group a representative color.
The former is to select the group that minimizes the error as defined before, assuming each group comprised of the lower colors sorted in ascending order. The same applies for the groups comprised of the higher colors.
The latter accrues the error computed separately for the two groups in all possible combinations and then provides for finding the minimum of the composite error.
Groups that include only the minimum color or the maximum color are not considered during the processing which are, instead, the reference colors for S3TC.
The arrangement disclosed herein detects black colors. Also the encoding steps, the bitstream composition and the decoding steps are different if compared to S3TC.
The invention will now be described, by way of example only, with reference to the annexed figures of drawing, wherein:
The presently preferred embodiment of the invention differs, in one of its aspects, from the S3TC algorithm in the way the reference colors are selected to construct the look-up table. The way of choosing these colors is made adaptive and consists in creating groups of colors for each color component R,G,B and select at first a group from which a representative color for this group is derived. Preferably, each group is composed by any number of colors between 3 up to 15 members. For each of them the median color is chosen as the representative color of the group to which it belongs. For sake of clarity, the median of a set of numbers put in ascending order is the number located in the middle position of them.
For example if the set is (1, 3, 5, 6, 20) then the median is the 3rd value (from right) and is equal to 5.
For each group, an error is computed as the sum of the absolute differences (SAD) between each group member and the representative (the median value of the group) color.
Still preferably, at least two different criteria are used to select the group first and then extract from this group a representative color.
The former is to select the group that minimizes the error as defined before, assuming each group comprised of the lower colors sorted in ascending order. The same applies for the groups comprised of the higher colors.
The latter accrues the error computed separately for the two groups in all possible combinations and then provides for finding the minimum of the composite error.
Groups that include only the minimum color or the maximum color are not considered during the processing which are, instead, the reference colors for S3TC.
The arrangement disclosed herein detects black colors. Also the encoding steps, the bitstream composition and the decoding steps are different if compared to S3TC.
An embodiment of the invention will now be described by using the approach previously adopted for describing the S3TC arrangement and assuming Q=S=4.
i) Decompose the R G B image in non overlapped Q=4 S=4 blocks of R G B colors
ii) Consider the following 4×4 block composed of 16 pixels each one composed by R, G and B components:
Pij=Rij U Gij U Bij (this again denotes the pixel at the ij position in the R G B image, where U is the union operator)
iii) Decompose the block above in three sub-blocks called sub-block R, sub-block G and sub-block B each block including only a color component:
iv) Sort in ascending order each sub-block color R, G, B as shown in
v) Define two sets, each set including some groups of color for each R, G, B component independently. The left-hand portion of
vi) For each group, compute the error as the sum of absolute differences (SAD) between its median color and each color composing the group. Referring to the right hand portion of
vii) Two sets of errors are computed, Ei and ej. Selection of the yellow group and red group (and then depending on which one is selected, the median is taken as the representative color) can occur in two ways:
viii) The color representatives as defined in step vii) will be used to set the encoding step.
If the black color is detected, step vi) is modified in such a way that each group of color does not include the black.
The basic scheme described in the foregoing lends itself to a numbers of variants.
A first variant has only two groups of colors of 3 and 5 elements as shown in
Depending on the criteria a) and b) assumed in the previous section vii two additional variants can be defined.
In particular, referring to
In the second variant:
A further additional variant takes always as min_medianreference1 equal to the second element and as max_median_reference—2 equal to the 15th, while another additional variant takes always as min_median reference 1 the 3rd element and max_median as reference 2 the 14th as shown in
At the end of above described variants, each one produces as a result two reference colors named:
1) min_medianR U min_medianG U min_medianB
2) max_medianR U max_medianG U max_medianB
where U is the union operator grouping them as a whole pixel.
Next, the proposed method computes a value called length as follows.
If the black colour (which is a pixel made of R=0 and G=0 and B=0) has not been detected:
Length—R=(max_medianR−min_medianR)/6
Length—G=(max_medianG−min_medianG)/6
Length—B=(max_medianB−min_medianB)/6
Length=√(|Length—R|2+|Length—G|2+|Length—B|2)
where max_medianR,G,B and min_medianR,G,B are the representative colors for each selected group belonging to the red and yellow sets.
This is the maximum quantization error the method can compute when Pij colors are quantized during the encoding step, here described.
If the black color is not detected for each Pij=Rij U Gij U Bij (where i range is from 1 to Q=4 and j range is from 1 to S=4) compute the Euclidean distance
Dist—ij=√(|Rij−min_medianR|2+|Gij−min_medianG|2+|Bij−min_medianB|2)
Now the encoder quantizes each color as follows:
When a block is encoded, the decoder receives a 2 bits code for each Pij as above defined, plus min_medianR U min_medianG U min_medianR plus length_R, length_G, length_B
Conversely, if the encoder detects the black color, then
Length—R=(max_medianR−min_medianR)/4
Length—G=(max_medianG−min_medianG)/4
Length—B=(max_medianB−min_medianB)/4
Length=√(|Length—R|2+|Length—G|2+|Length—B|2)
for each Pij=Rij U Gij U Bij (where i range is from 1 to Q=4 and j range is from 1 to S=4) quantize them as follows:
compute Dist ij=√(|Rij−min_medianR|2+|Gij−min_medianG|2+|Bij−min_medianB|2)
When a block is encoded the decoder receives 2 bits code for each Pij as above defined, plus min_medianR U min_medianG U min_medianR after length_R, length_B, length_B
If decoder receives min_medianR U min_medianG U min_medianR before length_R, length_B, length_B this means that the black color is not detected so the output colors will be
if the code is 00
if the code is 01
if the code is 10
if the code is 11
If the decoder receives Min_medianR U min_medianG U min_medianR after length_R, length_B, length_B it means that black color is detected so the output colors will be
if the code is 00
if the code is 01
if the code is 10
if the code is 11
The various arrangements described in the foregoing have been applied to the following standard images by using two formats: RGB 565 and RGB 888, where 5, 6 or 8 is the number of bits per color channel.
1. 256×256 (horizontal×vertical size dimension)
2. 512×512 (horizontal×vertical size dimension)
3. 512×1024 (horizontal×vertical size dimension)
4. 640×480 (horizontal×vertical size dimension)
5. 1024×768 (horizontal×vertical size dimension)
These pictures are a representative set on which texture compression is typicaly applied.
All the pictures are in true-color format or 888, while the 565 format is obtained from the 888 format by truncating the 323 lowest bits of the 888 pictures. Alternative truncating methods can be used to go from 888 to 565 such as rounding to nearest integer, Floyd-Steinberg dithering etc. These do not imply any changes in the arrangement disclosed herein.
To evaluate the performance of each arrangement, visual assessments and objective measures can be performed. In particular two parameters are taken as reference measures:
mean square error (MSE), and
peak signal/noise ratio (PSNR) for each RGB channel.
Input images IS in the 888 format (called Source888) are converted at 200 into the 565 format (called Source565), then compressed at 201 and further decompressed at 202 to the 565 format. These are back converted at 203 into the 888 format to generate a first set of output images OS′ (also called Decoded888).
The Source-565 images from block 200 are back converted into 888 at 204 to generate a second set of output images OS″ to be used as a reference (called Source565to888).
A first set of PSNR values (called PSNR 888) are computed between the Source 888 IS and the Decoded888 OS′ images. A second set of PSNR (called PSNR 565) values are computed between the Source565to888 OS″ and the Decoded888 OS′ images.
In particular, 565 images are back reported to 888 by simple zero bit stuffing of the 323 least significant positions.
How the Source888 IS images are converted to the 565 format and back to the 888 foamt corresponds to techniques that are well known to the experts in this area and do not need to be described in detail here:
MSE=(Σ|Pij−Paij|2)/(w*h)
where:
The results show that all the variants of the solution disclosed herein perform significantly better than S3TC in most tests.
Of course, the underlying principle of the invention remaining the same, the details and embodiments may vary, also significantly, with respect to what has been described and shown by way of example only, without departing from the scope of the invention as defined by the annexed claims.
Number | Date | Country | Kind |
---|---|---|---|
03002205 | Jan 2003 | EP | regional |
Number | Name | Date | Kind |
---|---|---|---|
6269181 | Acharya | Jul 2001 | B1 |
6819793 | Reshetov et al. | Nov 2004 | B1 |
20040081356 | Shimizu et al. | Apr 2004 | A1 |
Number | Date | Country |
---|---|---|
WO 9918537 | Apr 1999 | WO |
WO 9922519 | May 1999 | WO |
Number | Date | Country | |
---|---|---|---|
20040156542 A1 | Aug 2004 | US |