The present invention generally relates to a method of processing image and/or audio information, an image and/or audio processing apparatus, and a computer program for processing image and/or audio information.
The present invention more particularly relates to a method of processing image and/or audio information, an image and/or audio processing apparatus, and a computer program for processing image and/or audio information, in which the degrading of the encoded image and/or audio information caused by the truncation of bit planes can be evaluated without decoding the encoded image and/or audio information.
The Joint Photographic Experts Group (JPEG) and JPEG 2000 are well known international standards of a method of compressing image data.
In this specification, the “image information” includes the image data 11 and any data derived from the image data 11 such as the transformed image data, the quantized image data, and the encoded image data encoded with the entropy encoding.
The image processing apparatus that compresses the image information and the image processing apparatus that decompresses the image information are often combined as an image processing system.
The transforming unit 13 will be described by reference to
The quantizing unit 14 will be described by reference to
The encoding unit 15 will be described by reference to
The code blocks 53 are further divided into bit planes 54 as showed in
In the above description, the image data 11 are assumed to represent a monochrome image. In the case in which the image data 11 represent a color image, the image data (component) of each color can be input to the image processing apparatus 12 as showed in
As described above, in the case of JPEG 2000, the quantized wavelet coefficient data are divided into bit planes, and encoded by the bit plane. If some bit planes are cut off, the image data are further compressed. For example, the image data can be compressed by cutting off (truncating) the lower side of the encoded bit planes.
If a compression ratio is given as a target, data are cut off until the compression ratio reaches the target. If data are cut off, image quality is degraded. Accordingly, it is necessary to determine, when the data are cut off to a certain extent, how much the image quality degrades.
According to a method of determining the extent of the degrading showed in “Example and Guideline” (EG) of JPEG 2000, the bit planes are truncated one by one from the lowest bit plane to the highest bit plane, and the distortion is obtained after each truncation of a bit plane. For example, the lowest bit plane is truncated, and the distortion is obtained. Then, the second lowest bit plane is additionally truncated, and the distortion is obtained. This procedure is continued until the highest bit plane is truncated and the distortion is obtained. The distortion is obtained by decoding the encoded data after each truncation and comparing the decoded data with the original image data. The distortion is defined as mean squared error (MSE).
Since the encoded data after each truncation are decoded as described above, it takes long time to obtain the distortion. Otherwise, hardware that accelerates the above procedure is additionally required.
Accordingly, it is a general object of the present invention to provide a novel and useful method of processing image data in which one or more of the above problems are eliminated. A more specific object of the present invention is to provide a method of evaluating a distortion caused by truncation of bit planes without decoding encoded data.
To achieve one of the above problems, a method of processing image information, according to an aspect of the present invention, includes the step of encoding said image information by a bit plane; the step of generating index parameter indexing degradation of said image information caused by truncation of one or more bit planes based on said image information; and the step of compressing said image information by truncating the bit planes; wherein the bit planes to be truncated are determined based on said index parameter.
When truncating the bit planes of the encoded image information, one can evaluate the degradation of the image information caused by the truncation based on the index parameter generated in advance without decoding the image information.
Other objects, features and advantages of the present invention will become more apparent from the following detailed description when read in conjunction with the accompanying drawings.
A detailed description of the preferred embodiments will be described by reference to the drawings.
The truncation of layers (bit planes) will be discussed below.
In the case of
As described above, the number of MSBs of each layer is a parameter that indexes the degrading of the image quality. It is easy to obtain the number of MSBs from the wavelet coefficient data. Accordingly, one can evaluate the degrading of an image due to the truncation easily and quickly based on the number of MSBs.
Furthermore, when the bit planes of the encoded data are truncated, the degrading of an image due to the truncation of bit planes can be evaluated without decoding the encoded data encoded with the entropy code.
For example, if one obtains the number of MSBs of each bit plane in advance based on the wavelet coefficient data before encoded with the entropy code, and attaches the obtained number of MSBs to the wavelet coefficient data after the entropy encoding or stores the obtained number of MSBs in a storing unit, the one can evaluate, without entropy decoding, the degrading of the image due to truncations using the obtained number of MSBs that is attached or stored.
If one predicts the image distortion of the image due to the truncations of bit planes or a slope parameter (the ratio of the distortion of the image to the amount of reduced image data by the truncation), and attaches the predicted distortion or slope parameter, instead of the obtained number of MSBs, to the wavelet coefficient data after the entropy encoding or stores the predicted value in the storing unit, the one can evaluate, without entropy decoding, the degrading of the image due to the truncations using the predicted distortion or slope parameter that is attached or stored.
In the following description, the parameters that indicate the degrading of the image due to the truncation of bit planes such as the number of MSBs of each bit plane, the distortion of the image, and the slope parameter will be called “index parameter”.
According to the first embodiment, the encoding unit 15 provides the index generating unit 71 with the wavelet coefficient data 51 before entropy encoding (referred to as “wavelet coefficient data 77 before entropy encoding”) as showed in
According to the first embodiment, the counting unit 72 provides “Na” to the distortion predicting unit 73 and the slope parameter predicting unit 74 provided in the index generating unit 71 as showed in
In the case where bit planes 1 through “n” are truncated, the distortion value of the image of this case may be defined as the sum of “Na” multiplied by the level of each bit plane. For example, in the case of
According to the first embodiment of the present invention, the encoding unit 15 provides the compressing unit 76 with the wavelet coefficient data 51 after entropy encoding (referred to as “wavelet coefficient data 78 after entropy encoding”) as showed in
For example, one may set a predetermined threshold for each bit plane 54, and compare the index value generated by the index generating unit 71 with the predetermined threshold. The index value of the bit plane 1 is compared with corresponding threshold; the index value of the bit plane 2 is compared with corresponding threshold; and so on. If the index value of the bit plane “n” exceeds corresponding threshold for the first time, the bit planes 1 through “n-1” are determined to be truncated. If the index value of the bit plane 1 exceeds corresponding threshold, that is, n=1, no bit plane is truncated. The thresholds may be the same, or may be different.
As described above, the compressing unit 76 can determine, without entropy decoding, the bit planes to be truncated that does not degrade too much using the index values generated by the index generating unit 71.
According to the embodiment, the encoding unit 15 provides the index adding unit 75 with the wavelet coefficient data 78 after entropy encoding, and the index generating unit 71 provides the index adding unit 75 with the index values generated in advance as showed in
After attaching the index values to the wavelet coefficient data 78, the index adding unit 75 provides the compressing unit 76 with the wavelet coefficient data 78. The compressing unit 76 can determine the bit planes to be truncated based on the index values attached to the wavelet coefficient data 78 and truncate them.
Accordingly, the compressing unit 76 can determine, without entropy decoding, the bit planes 54 to be truncated using the index values attached to the wavelet coefficient data 78 generated by the index generating unit in advance,.and can truncate the determined bit planes.
The wavelet coefficient data 78 after entropy encoding according to JPEG 2000 will be described below.
In the case of the image processing apparatus according to the embodiment showed in
In general and in this specification, the term “encoded data” includes not only the encoded data 16 output by the image processing apparatus but also the wavelet coefficient data 78 after entropy encoding.
The encoded data of
A tile part header 172A follows the main header 171. The tile part header includes a “start of tile” (SOT) 175A indicating the start of the tile part header 172S, a “tile (A) header maker segment” (T(A)) 176A indicating the content of the tile part header 172A, and a “start of data” (SOD) 177A indicating the start of data.
A bit stream 178A follows the tile part header 172A.
A plurality of tile part headers 172B, 172C, each followed by a bit stream 178B, 178C, . . . , respectively follows the bit stream 178A, if applicable. The “end of codestream” (EOC) 179 indicating the end of the codestream follows the last bit stream.
As described above, the index adding unit 75 attaches the index values generated by the index generating unit 71 to the encoded data of JPEG 2000. According to the current format of the encoded data of JPEG 2000, one can insert a comment marker in which the one can store comment text in the encoded data of JPEG 2000. The index values can be stored in the comment marker. The comment marker may be provided in the main header 171 or the tile part header 172. Otherwise, the comment marker may be provided in a header dedicated for the comment marker.
In the above description, the bit plane of (quantized) wavelet coefficient data is considered. The present invention, however, is also applicable to the bit plane of other image information.
In the above description, the number of MSBs of each bit plane and so forth are used as an index parameter. The present invention, however, is not limited to those index parameters described above, and can use any parameter that indexes the degrading of the image due to the truncation of bit planes.
For example, the present invention can use the distortion and/or the slope parameter caused by the truncation of the bit planes using the methods described in the EG of JPEG 2000 as the index parameters. One can provide a distortion obtaining unit and/or a slope parameter obtaining unit instead of the counting unit 72 in the image processing apparatus according to the embodiment.
In the above description, it is assumed that the image data are compressed with JPEG 2000. The present invention, however, is not limited to JPEG 2000, and is applicable to any other image compression method in which the image information is encoded by the bit plane, and the encoded image information are compressed by truncating the bit planes.
Additionally, the present invention is not limited to the case where the image information is encoded by the bit plane, and the encoded image information is compressed by truncating the bit planes. The present invention is also applicable to the case where the image information is encoded part by part, and the each encoded part of the image information is compressed by reducing the encoded part The present invention is also applicable to audio information such as voice data instead of the image information such as the image data.
An MSB embodies a half of the amount of information contained in a data item. If the second most significant bit (second MSB), that is, the bit subsequent to the MSB is “1” (effective bit), the MSB and the second MSB holds ½ through ¾, in total, of the amount of information contained in the data item. If the second MSB is otherwise “0”, the MSB and the second MSB hold ¾ through 1 of the amount of information contained in the data item in total. Accordingly, the distortion of an image can be evaluated more accurately based on not only the number (Na) of MSBs but also the number (Nb) of MSBs followed by second MSB that is 1. For example, one may assume that an MSB followed by a second MSB that is 0 is 1.5 times effective on the degrading of the image than an MSB followed by a second MSB that is 1, in other words, the MSB followed by a second MSB that is 0 corresponds to 1.5 MSBs followed by a second MSB that is 1.
When predicting the distortion of an image and the slope parameter, the distortion predicting unit 73 and the slope parameter predicting unit 74 may not necessarily handle all code blocks equally. The distortion predicting unit 73 and the slope parameter predicting unit 74 may weight each code block based on the component and the sub band so that the degrading of the image can be accurately evaluated.
If the encoding unit 15 can compress with entropy encoding the wavelet coefficient data 51 at a compression rate more than desired one, the compressing unit 76 does not need to truncate the bit planes. The compressing unit 76 does not need in this case to attach the index value to the wavelet coefficient data 51.
The index adding unit 75 attaches the index value to the wavelet coefficient data 51, but can be selectively set, in response to an instruction from an exterior, not to attach the index value to the wavelet coefficient data 51 so as not to lower the compression rate by attaching the index value unnecessarily.
When a plurality of code blocks in a sub band is truncated at different truncation level, the truncation sometimes causes visible distortion between the code blocks. To solve this problem, the truncation is generally executed by the sub band instead of code block. When the truncation is executed by the sub band, one can obtain the number of MSBs by the sub band instead of by the code block so as to reduce required calculation.
The preferred embodiments of the present invention are described above. The present invention is not limited to these embodiments, but various variations and modifications may be made without departing from the scope of the present invention.
This patent application is based on Japanese Laid-open Patent Application No. 2002-128682 filed on Apr. 30, 2002, the entire contents of which are hereby incorporated by reference.
In the case of JPEG 2000, for example, image information is encoded by the bit plane, and is compressed by truncating the bit planes of the encoded image information. When truncating the encoded bit planes, one can evaluate, using the method of processing image information according to the present invention, the degrading of the image information caused by the truncation without decoding the encoded image information.
Number | Date | Country | Kind |
---|---|---|---|
2002-128682 | Apr 2002 | JP | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/JP03/05459 | 4/28/2003 | WO |