BACKGROUND OF THE INVENTION
1. Field of Invention
The present invention relates to method and apparatus for reducing data rate of image pixels, and particularly relates to compression and decompression of image frame of the displaying video stream with fast random accessing any line of pixels.
2. Description of Related Art
The digital image and motion video have been adopted in an increasing number of applications, which include digital camera, scanner/printer/fax machine, video telephony, videoconferencing, 3G mobile phone, VCD (Video CD), DVD, digital TV . . . etc. The success of development of the digital image and video compression standards including JPEG, a still image compression standard set by ITU and the MPEG, the motion video compression standard set by the ISO have played key role for the digital image and video applications. Both JPEG and MPEG are lossy algorithms which when being decompressed, the image data are different from the original ones. And the bit rate of each image frame is variable depending on the complexity of the image pattern.
In the television, DVD, STB, Set-Top-Box, 3G mobile phone or other portable devices, a decompressed video stream of continuously image will be temporarily saved into a storage device for other manipulation including scaling and deinterlacing before being sent to a display device. In these kind of applications, a fixed bit rate of compressed image is expected for convenience of allocating fixed density of the storage device or/and allocating fixed bandwidth of accessing the storage device during a pre-scheduled time slot. Fixed bit rate of a compressed image frame provides necessary performance of random accessing pixels of any area with a fixed density of storage device which results in the saving of cost and power consumption. Since these image frames temporarily saved in the storage device will be used as references in the further image frames, keeping good quality is required for ensuring the good image quality for a video stream.
Therefore, it is beneficial to reduce the bit rate of an image frame of a video stream with fixed bit rate of an image while still keeping good image quality for further image frames' reference. The capability of being quickly random accessed and decompressed pixels of any area of an image enhances the performance of other manipulation including scaling, deinterlacing and displaying.
SUMMARY OF THE INVENTION
The present invention is related to a method and apparatus of the image frame compression of the video stream, which plays an important role in image data reduction for the display device controller. The present invention significantly reduces required storage device density, on-chip image buffer and accessing bandwidth of the image within the storage device.
The present invention of the image frame compression takes the advantage of only little difference between adjacent frames and compresses the current frame by referring to the bit rate information of each area of the previous frame.
According to an embodiment of this invention, the bit rate information of each area within a frame is saved into a temporary storage device and when compressing the current image, the bit rate information of the corresponding area of previous image is referred.
According to an embodiment of this invention, the bit rate information of each area within a frame is saved into a temporary storage device and when compressing the current image, the bit rate information of the corresponding area of previous image is referred.
According to another embodiment of the present invention, each area of compressed pixels has different bit rate.
According to an embodiment of the present invention, a predetermined length of code is assigned to represent the bit rate of each area of an image, and the codes are inserted into the predetermined location of an image stream.
According to an embodiment of the present invention, during random accessing any area, the starting location of the selected area within storage device is calculated firstly, and then, the pixels are decompressed in pipelining.
According to another embodiment of the present invention, a threshold value is predetermined to decide whether the bit rate allocation of an area of the current image should refer to the bit rate of the bit rate information of the previous image.
It is to be understood that both the foregoing general description and the following detailed description are by examples, and are intended to provide further explanation of the invention as claimed.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 illustrates the prior art of image frame compression with constant bit rate of each line.
FIG. 2 illustrates an image frame compression with variable bit rate of each line/area and a predetermined frame bit rate.
FIG. 3 illustrates the video stream showing little difference from frame to frame.
FIG. 4 shows the procedure of compressing an image frame with reference of bit rate information of each area of previous image frame of video stream.
FIG. 5 lists a table of an example of bit rate information
FIG. 6 shows how a compressed image stream is organized with bit rate information inserted.
FIG. 7 illustrates how the starting location of each area (or line) can be calculated and pixels being quickly decompressed.
FIG. 8 depicts the procedure of deciding whether an image frame can be compressed by referring to bit rate information of the previous frame.
DESCRIPTION OF THE PREFERRED EMBODIMENTS
Some display device require high density of the storage device to temporarily saving the decompressed image(s) generated from a MEPG/JEPG decoder. Most of the storage devices are used to save the image frame for quality enhancement manipulation including image scaling and deinterlacing. Especially the later, for achieving high quality, at least three previous frames of image are saved for image compensation to reconstructing the current image frame. Quickly accessing pixels of any area within image frame becomes very challenging task which requires high I/O bandwidth.
This invention of the image frame compression of the video stream reduces the bit rate of the image frame by assigning a fixed bit rate of an image frame with the capability of quickly random accessing and decompressing pixels of any area, for example, any line.
FIG. 1 illustrates a prior art of image compression. An image frame 11 is comprised of a certain numbers of lines of pixels 17, 18. In this prior art image compression, each line of pixels 13 is compressed with fixed bit rate 14, for instance, 2.0X compression rate and comes out of a fixed length of bits of each line 15, 16 and together forms the compressed image with fixed bit rate 12. The advantage of this prior art image compression with fixed line compression rate is that this scheme allows quickly accessing and decompressing any line of pixels. The disadvantage of this prior art of image frame compression is lack of flexibility of allowing more bits for lines with more complex pattern which might be difficult in compressing pixels to a predetermined bit rate.
FIG. 2 shows a basic of another image compression method of this invention of image frame compression. Those lines with shorter bit rate 22, 23, are line having simple patterns which saves more bits to be shared with the line with more bit rate 24, 25 of more complex patterns. Together, this variable length of bits of each line of pixels forms a fixed bit rate of an image frame.
This invention of image frame compression takes another feature of little change between adjacent image frames as shown in FIG. 3. The continuous image frames 31, 32, 33 have high possibility of the same background 34, 35, 36 of views and even the motion parts have only partial area having movements 37, 38, 39 in a short period of time of adjacent frames. In a short period of time of a continuous image frames, there are high similarity of each area of an image frame from frame to frame.
In prior art image frame compression shown in FIG. 1, the difficulty in compressing an image frame is the unpredictable bit rate allocation, hence and can only assign fixed number to each area/line. This invention of image frame compression takes advantage of little change between adjacent frames and allocate bit rate from area to area according to the previous image frame. FIG. 4 illustrates the procedure of image frame compression area by area with reference of bit rate of previous frame. After compression, each “Area” (for example a “Line”) bit rate is calculated and rounded to a predetermined length 43 before being saved into a bit rate register 41 which is comprised of area bit rate registers 44, 45, 46 with fixed bit rate for each area.
FIG. 5 shows a list of an example of coding of the bit rate of an area, said for example, a line with base of a fixed bit rate of each area 51, for example the 1st bit represents 4.0X (if “0”) or 3.6X (if “1”) compression rate. The 2 md to 5th bits 52 of the code represents a known bit rate of a line plus or minus the value of the code times 8 bits or said a “Byte” 53. For instance, the code of “0 0000” represent bit rate of 4.0X compression of a line plus 0 Byte, “0 0001” represent bit rate of 4.0X compression of a line plus 1 Byte, “0 0010” represent bit rate of 4.0X compression of a line plus 2 Byte, “1 0001” represent bit rate of 4.0X compression of a line minus 1 Byte, . . . If a line is comprised of 1920 pixels 54 with 4:2:2 YUV format which represents 16 bits per pixels, the code of “0 0000” represent bit rate of 4.0X compression of a line plus 0 Byte equals to 7680 bits, and the code of “0 0001” represents 7688 bits . . . the code of “0 0001” represents 7724 bits. For simplicity and easy accessing, the “Byte”, “Word” or “Double Word” boundary is selected to represent the even number of bit rate of an area of an image frame. Should the bit rate of an area does not fit exactly the Byte, Word or Double Word boundary, some dummy codes are inserted or the code beyond the meaningful bits are ignored in decompressing.
With the code of bit rate of each area/line, random accessing any area or line becomes easy and costs only a short of time to access by decoding and calculating the starting location of each area/line. FIG. 6 depicts the bit stream organization of this invention of the image frame compression 61 which comprises the compressed information image frame 62 followed by code 63, 64, 65 of bit rate indication of each area. The code of bit rate of each area is copied from the register saving the bit rate information of each area.
FIG. 7 illustrates the mechanism of quickly accessing any area or line of a compressed image frame. Codes 71, 72, 73 of bit rate information are inserted to a predetermined location of a compressed image frame which is most likely in the end of a compressed image. For fast accessing, each area/line of an image frame uses a decoder 74, 75 to decode and calculate the total bit number 76 of each area/line. The calculated bit number of each line can be added 77, 78 to become an accumulative starting location of any area. Since the chain and accumulative delay of decoding and calculating might be longer than the time of one clock cycle, a predetermined number of continuous areas of pixels is clustered as a group of areas which reduces one clock cycle time to decode and calculate the starting location of any selected area. For example, if an image frame has 1920 lines, for a code of 5 bits with the 2nd to 5th bits representing the bit rate of each line and the 1st bit as a compression rate of 4.0X or 3.6X, the total of delay time of the cascaded gate can be longer than 3-4 clock cycles. If every 512 lines are clustered into a group, there will be a total of 4 clocks cycles needed to decode any line of a compressed image frame with each cycle time decoding starting location of about 480 lines. The starting location of any the 480 lines can be hierarchically decoded by 3 levels with each having 8 inputs of bit rate values. All 5 bits of code representing the bit rate of each area or each line can be decoded and calculated together.
The 1st bit of the bit rate code can be fixed to indicate two compression rate, for example a 4.0X and another 3.6X. The accumulative addition of the line bit rate can be calculated separately from the other 4 bits of bit rate variance. In the end of the last stage of addition, the result of the line lengths 79 are added with the bit rate variance of each line which can be calculated by “Addition” and “Shifting” instead of long chain of addition.
Even the bit rate information of previous image frame helps in bit rate allocation of the current frame, those pixels in the area with no correlation from corresponding area of the previous image should not be compressed with referring to previous frame. FIG. 8 depicts the flow chart of the decision making on whether or not the bit rate allocation of the present area refers the bit rate information of the corresponding area in previous image frame. Pixels of a new area are examined and calculated the bit rate 81 of applying a lossless compression algorithm. The bit rate number is then compared to that of the corresponding area 82 of the previous image. The “Corresponding Area” might be a shifted area with the displacements of X- and Y-axis according to the motion vector 86 of the present area of the current image frame. If the difference is under the predetermined threshold value, then, compressing the current area of pixels by referring to the bit rate information of the corresponding area of the previous image frame 84, of the difference value is over the threshold, then, compressing the present area independent on the previous image frame 85. Some areas will be compression with reference of previous image, some will be independent on the previous image and a MUX 87 selects which result to be the output of the compressed area. For simplicity, the majority of the motion vectors of area within an image frame can be used to represent the “Frame Motion Vector” since most likely, the movement within a frame are the same. In practical implementation of this invention of the image frame compression of a video stream, the line by line compression either from top to down or from bottom to up is most common mechanism and the X-axis is the most concerned motion displacement for the corresponding area (or line) and the corresponding area (or line) is calculated accordingly. For obtaining more accurate bit rate allocation, both X- and Y-axis are referred.
It will be apparent to those skills in the art that various modifications and variations can be made to the structure of the present invention without departing from the scope or the spirit of the invention. In the view of the foregoing, it is intended that the present invention cover modifications and variations of this invention provided they fall within the scope of the following claims and their equivalents.
Patent Application
20070242749
