The present invention relates to a technology for improving JPEG 2000 system, which is in recent years widely used as a compression technology for moving image data that are frequently transmitted and received via a data communication line, such as the Internet, and particularly relates to a moving image data compressing method, for effectively reducing the processing load in coding and decoding for image frames that constitute moving image data.
In recent years, the communication speed on a data communication line is becoming exponentially increasing. Also between individually owned information terminals, such as personal computers (hereinafter, referred to as a PC/PCs), transmitting and receiving of large volume visual information including moving image data, audio data, etc has come to be performed everyday. However, in case of practically distributing huge moving image data, using a limited network resource, it is necessary to reduce the data amount itself due to limits, such as the processing capacity of individual terminals, the line speed of a network, and the like. In such a situation, technologies for compression of moving image data and audio data to be distributed have been actively researched, and brought into practical use.
In particular, as a technology for compressing moving image data, H.264 (MPEG-4 AVC, hereinafter merely referred to as the MPEG system, including such next generation image compression technologies), which has been brought into practical use, is prevailing. Further, a Motion-JPEG 2000 system (hereinafter, merely referred to as the JPEG 2000 system) in which each image frame is compressed, according to JPEG 2000 standard known as a still image compressing method, as disclosed by Patent Document 1, has also become rapidly prevalent.
A moving image compressing technology, such as the MPEG system, can change the number of frames in a certain time at an information terminal on the transmission side, however, upon transmission, during when processing is possible, of frames in response to a request by an information terminal on the receiving side, for example, a delay is caused for the time from changing the number of frames to transmitting the frames, which makes real time processing impossible. On the other hand, a moving image compressing technology by the JPEG 2000 system is a moving image data compressing technology for moving image data of image frames that are disposed along a time axis without being compressed in the time axis direction and are individually compressed and decompressed. This technology uses the difference information between adjacent image frames. Thus, this technology is definitely distinguished from the above-described MPEG system and the like that performs compression also in the time axis direction.
The JPEG 2000 system, as described above, has a low compression efficiency for the same bit rate, as compared with the MPEG system, because compression is not performed in the time axis direction, however, the JPEG 2000 system has features, such as to enable individual editing of an arbitrary image frame, by separating the image frame from other image frames. Further, the load during coding and decoding at each information terminal is small, enabling real time encoding with comparatively simple equipment. Accordingly, a video capture cards for individual use or the like is employed as an output format.
The present inventors have examined conventional moving image compressing technologies, particularly the JPEG 2000 system, and as a result, have discovered the following problems.
That is, in sequentially compressing image frames that constitute moving image data as in the conventional JPEG 2000 system, according to the JPEG 2000 standard, it is necessary to compress all the moving image data for each image frame, which causes a problem of a low compression efficiency. Further, in a moving image compressing process by the JPEG 2000 system, EBCOT, which is an entropy coding process, is performed. As EBCOT requires an extremely large computation amount, it applies a significant processing load to certain information processing equipment, such as a personal computer.
The present invention has been developed to eliminate the problems described above. It is an object of the present invention to provide a moving image compressing method that improves the compression efficiency of moving image data and is effective for reduction in the processing load, due to the JPEG 2000 system, by performing a movement detection process in a moving image compressing process by the JPEG 2000 system, using data after wavelet transform, prior to EBCOT with a large computation amount.
A moving image data compressing method according to the present invention relates to technology and processing of compression by the JPEG 2000 system, which is a technology for compression of moving image data constituted by image frames that are disposed along a time axis without being compressed in the time axis direction and are individually compressed and decompressed according to the JPEG 2000 standard. In the moving image data compressing method according to the present invention, computation performed for each image frame basically includes processes of color space transform (including DC level shift and component transform), wavelet transform, scalar quantization, and EBCOT.
Particularly, in the moving image data compressing method according to the present invention, a movement detection process is performed by comparison between wavelet transform data of image frames adjacent to each other along the time axis, prior to EBCOT, which is a processing algorithm for entropy coding and requires a large computation amount.
Specifically, in a moving image data compressing method according to the present invention, as a compressing process of image frame to be processed among image frames that constitute moving image data, a movement detection process of image elements and data supplement are performed prior to EBCOT.
The movement detection process that is performed prior to EBCOT detects the presence or absence of movement of image elements with one or more code blocks, each of which is the minimum process unit of EBCOT, as one detection unit, and comparing data after wavelet transform of an image frame to be processed, with data after wavelet transform of an image frame compressed immediately before.
Based on a result (detection result) of the detection movement process, EBCOT or data supplement is performed. That is, EBCOT is performed on respective code blocks which constitute a detection unit having moved. On the other hand, EBCOT is skipped for respective code blocks which constitute a detection unit having not moved, and instead, data after EBCOT process of an image frame compressed immediately before or index data (skip data) that suggests skip operation is assigned.
In the moving image data compressing method according to the present invention, the movement detection process performed prior to EBCOT is executed at a timing before or after quantization performed prior to EBCOT. In other words, the movement detection process is performed after wavelet transform and before quantization, or after quantization and before EBCOT.
Further, in the moving image data compressing method according to the present invention, in order to improve the efficiency (shortening the processing time) of the movement detection process, it is preferable that a detection result of the movement detection process is shared among a plurality of sub-bands obtained by spatially dividing, through wavelet transform, an image frame to be processed into a plurality of kinds of frequency bands. Such information sharing is possible because position information is saved among the plurality of sub-bands obtained through the wavelet transform. That is, regarding the movement detection process on the respective plurality of sub-bands, first, the movement detection process is performed on the sub-band with the lowest frequency component for each detection unit, and a result of the movement detection process on the sub-band with the lowest frequency component is also applied to the other sub-bands. Specifically, a detection result identical to that of each of the code blocks, which constitute the detection unit, is applied to the code block that represents an image area identical to that of each of the code blocks that constitute the detection unit, in each of the other sub-bands. By having a detection result of the movement detection process shared among the respective sub-bands, it is possible to significantly shorten the processing time required by the movement detection process.
The present invention will be more fully understood from the detailed description given hereinbelow and the accompanying drawings, which are given by way of illustration only and are not to be considered as limiting the present invention.
Further scope of applicability of the present invention will become apparent from the detailed description given hereinafter. However, it should be understood that the detailed description and specific examples, while indicating preferred embodiments of the invention, are given by way of illustration only, since various changes and modifications within the scope of the invention will be apparent to those skilled in the art from this detailed description.
By the moving image data compressing method according to the present invention, in a moving image compressing process by the JPEG 2000 system, presence or absence of movement of image elements can be detected (a movement detection process) by comparison of wavelet transform data between image frames adjacent to each other prior to EBCOT with a large amount of computation. Based on a detection result of the movement detection process, it is determined whether or not to skip code blocks, which are a minimum processing unit, in EBCOT. Thus, it is possible to improve the compression efficiency and accordingly reduce the processing load.
Further, a detection result of the movement detection process is shared among plural sub-bands obtained by spatially dividing, through wavelet transform, an image frame to be processed into plural kinds of frequency bands, and it is thereby possible to improve the efficiency of the movement detection process.
In the following, embodiments of a moving image data compressing method according to the present invention will be explained in detail, referring to
Transmitting and receiving of moving image data between a plurality of information processing devices can be realized by a communication system configured, for example, as shown in
First, a movement detection process by the JPEG 2000 system, which is a basic technology for the moving image data compressing method according to the present invention, will be described.
Moving image data to be an object of the JPEG 2000 system are, as shown in
In case that respective image frames are color images, respective components fn-R, fn-G, and fn-B (herein, an RGB primary color system) of original image frames are, in general, divided into rectangular regions 10, as shown in
After color space transform S110 including DC level shift and component transform, as described above, is performed, as a compression process (including coding) according to the JPEG 2000 system, discrete wavelet transform S120, scalar quantization S130, and coding by EBCOT: S140 are sequentially performed, as shown in
First, as shown in
Herein, the wavelet transform S120 refers to discrete wavelet transform (DWT) in the JPEG 2000 standard, and DWT based on the lifting structure of a two-channel filter bank is adopted. There are two kinds of DWT based on the lifting structure, namely, integer type DWT, which is invertible transform, and real type DWT, which is non-invertible transform. The real type DWT is used for lossy (non-invertible) coding, and the integer type DWT is used for lossless (invertible) coding.
Subsequently, the scalar quantization S130 performs scalar quantization of the DWT coefficient for each sub-band. However, the process is omitted in case of using the integer type DWT. The step size of quantization for the scalar quantization is represented as follows.
Herein, Δb represents the quantization step of the sub-band b, and Rb represents the dynamic range of the sub-band b. Eb and μb are respectively expressed by 5 bits and 11 bits, and transmitted to a decoder for the inverse quantization S230. The above-described quantization step size has a high image quality priority. That is, a small step size is set for a tile 10 for which a high image quality is desired, and a small step size is set for a tile 10 to which a low image quality is applicable. Setting the quantization step size to 1 is virtually equivalent to not performing scalar quantization.
Then, EBCOT (Embedded Block Coding with Optimized Truncation): S140 is an algorithm that takes the role of processing corresponding to entropy coding, and is formed by two processes which are coefficient bit modeling and arithmetic coding. EBCOT: S140 is performed on minimum coding processing units called code blocks 20. Code blocks 20 are defined by a rectangular region in a DWT region, and have the same size commonly to all sub-bands. Further, among the above-described three processes, the processes excluding the arithmetic coding are independently performed for each unit of the code block size.
In the coefficient bit modeling, coefficients in code blocks are subjected to bit plane decomposition, and the contexts of coefficient bits in respective bit planes are determined. In the determination of the contexts, context assignment maps based on a prepared statistic model are prepared. The context assignment maps are different depending on the band. According to the context of the coefficient bit, a single bit plane is divided and arranged into three coding passes (sub-bit planes).
Then, the arithmetic coding codes the respective coding passes, using an MQ coder, which is a binary arithmetic coder. Context is necessary for the MQ coder, and the context obtained by the coefficient bit modeling is used.
When EBCOT: S140 is completed, the rate control S150 is subsequently preformed. The rate control S150 is arranged by layer dividing and code truncation. In the layer dividing and the code truncation, coded data arrays generated for the respective code blocks are divided onto plural SNR layers in a given coding rate, depending on the degree of contribution to improvement of SNR in a reproduced image frame. The uppermost layer affects the image quality the most, and by receiving the respective layers in the order from the uppermost to the lowermost, the image quality of the reproduced image frame can be improved in stages. Positions that enable layer dividing are limited to the respective ends of the coding passes, and these ends are called truncation points. Rate control in the JPEG 2000 standard can be attained by cutting off data that exceeds the given coding rate with a unit of a truncation point, from the data re-arrayed in the order of higher contribution to the image quality.
As described above, a portion of coded data is cut off by the rate control S150 for matching with a target coding amount, and after adding a header and through packet generation S160, moving image coded data subjected to a compression process by the JPEG 2000 system is generated.
As shown in
A first embodiment of a moving image data compression method according to the invention, based on a moving image compression process by the above-described JPEG 2000 system will be described below in detail, referring to
In the moving image data compressing method in the first embodiment, prior to EBCOT: S140, a movement detection process S310 (described as ‘movement detection’ in
First, in the movement detection process S310, one or more code blocks 20 is/are set as a detection unit. Specifically, data after wavelet transform of an image frame fn to be processed is compared with data 100 after wavelet transform of an image frame fn-1(previous frame) having been subjected to compression processing immediately before.
As a result of the above-described movement detection process S310, if the data compared with each other are different, the subsequent coding process EBCOT: S140 is performed on the respective code blocks 20 which constitute a detection unit. On the other hand, if the data compared with each other agree with each other (or substantially agree with each other), EBCOT: S140 is skipped for the respective code blocks 20 which constitute a detection unit, and the process is proceeded to the branch point B after data supplement S320 is performed (the process after completion of EBCOT: S140).
In the data supplement S320 in the first embodiment, data 200 of the immediately previous image frame after being subjected to arithmetic computation is assigned to the respective code blocks 20 which constitute the detection unit on which movement of pixel elements has not been detected.
In the first embodiment, data 200 after arithmetic computation on the immediately previous image frame has been used as data for data supplement, however, index data (skip data) suggesting that EBCOT: S140 has been skipped in the compression process of the image frame fn may be used. In a second embodiment, arrangement is made such that actual data supplement can be performed in the decompression process of coded data, referring to skip data that has been assigned to the objective code blocks 20 in the compression process of the image frame fn.
Also in the moving image data compressing method in the second embodiment, prior to EBCOT: S140 similarly to the first embodiment, a movement detection process S410 is performed before or after the scalar quantization S130. That is, as shown in
In the movement detection process S410, one or more code blocks 20 is/are set as one detection unit. Specifically, data after wavelet transform of an image frame fn to be processed is compared for each detection unit, with data 300 after wavelet transform of an image frame fn-1(previous frame) having been subjected to compression processing immediately before.
As a result of the movement detection process S410, if the data compared with each other are different, the subsequent coding process EBCOT: S140 is performed on the respective code blocks 20 which constitute the detection unit. On the other hand, if the data compared with each other agree with each other (or substantially agree with each other), EBCOT: S140 is skipped for the respective code blocks 20 which constitute the detection unit, and the process is proceeded to the branch point C after data supplement S420 is performed (the process after completion of EBCOT: S150). In the data supplement S420 in the second embodiment, skip data, as index data suggesting that EBCOT: S140 has been skipped, is assigned to the respective code blocks 20 which constitute the detection unit on which movement of pixel elements has not been detected.
On the other hand, as shown in
The timing of actually executing the data supplement process S440 can be, for example, after the inverse quantization S230 (branch point E), as described above, however, the data supplement S440 may be performed at another timing as long as the stored data of the immediately previous image frame is in an appropriate stage for reproduction. Further, the above-described compression process and the decompression process may be performed on different PCs or on the same PC.
(Example of Application of Movement Detection Process)
Although, as described above, the moving image data compressing method in the first or the second embodiment, the movement detection process S310, S410 is performed prior to EBCOT: S140, a moving image data compressing method according to the present invention also enables a more efficient movement detection process then.
That is, for efficiency (reduction in the processing time) of the above-described movement detection process S310, S410, a detection result of the movement detection process is shared between plural sub-bands obtained by spatial dividing of an image frame fn to be processed into plural kinds of frequency bands. Such sharing of information is possible because position information is stored among plural sub-bands obtained by the wavelet transform S120.
Specifically, as shown in
Regarding the method for specifying the same image area, in each of other sub-bands at the same decomposition level (Assuming that the sub-band LL2 is the reference, the sub-bands at the same decomposition level are HL2, LH2, and HH2.), the area at the same position and with the same size as those of the comparison object is the same image area. Further, in each of sub-bands at a decomposition level higher by one step (Assuming that the sub-band HH1 is the reference, the sub-bands at the decomposition level higher by one step are LL2, HL2, LH2, and HH2.), the position and the size become ½ times in the area. In each of sub-bands at a decomposition level lower by one step (Assuming that the sub-band LL2 is the reference, the sub-bands at the decomposition level lower by one step are HL1, LH1, and HH1), the position and the size become two times in the area.
By having a detection result of a detection process shared between respective sub-bands in such a manner, the processing time required for the movement detection process can be greatly shortened. Further, the reference for determining whether data is the same in the comparison process depends on the required video image quality, and there may be a case that the data is considered to be the same if similarity to a certain extent or more exists.
From the invention thus described, it will be obvious that the embodiments of the invention may be varied in many ways. Such variations are not to be regarded as a departure from the spirit and scope of the invention, and all such modifications as would be obvious to one skilled in the art are intended for inclusion within the scope of the following claims.
10 . . . tile; 20 . . . code block; fn (n=1, 2, . . . ) . . . image frame; fn-R, fn-G, fn-B . . . component; 110 . . . network; and 120, 10a-120c . . . PC.
Number | Date | Country | Kind |
---|---|---|---|
2008-307608 | Dec 2008 | JP | national |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/JP2009/069938 | 11/26/2009 | WO | 00 | 8/24/2011 |