Patent Grant
8526730
The application claims benefit of Japanese Application No. JP-A-2010-93036. The disclosure of the prior application is hereby incorporated by reference in its entirety.
This disclosure relates to image processing apparatuses and methods of processing color image data that successively receive color image data of successive frames, correct the received color image data in accordance with differences of the image between frames, and output corrected color image data.
Liquid crystal display panels have a characteristic that ON-OFF response time of liquid crystal cells is longer than a frame period of moving images. In order to shorten the response time of liquid crystal cells of the display panel, an overdrive technique is widely utilized. That is, color image data of the previous frame is stored in a frame memory and compared with color image data of the current frame, and corrections are made in accordance with changes of the image between the frames.
Color image data of the current frame (RGB input) is stored in the frame memory 34. The OD amount calculation block 36 calculates amounts of overdrive based on R-, G-, and B-element values of respective pixels of a previous frame read from the frame memory 34 and R-, G-, and B-element values of corresponding pixels of the current frame. Corrected color image data (RGB output) is generated by adding, by using the adder 38, R-, G-, and B-element values of respective pixels of the current frame and the calculated overdrive amounts and output.
The frame memory 34 that stores color image data of the previous frame need to have a large memory capacity in order to store R-, G-, and B-element values of respective pixels. In order to reduce a required capacity of the frame memory, Japanese Laid-open Patent JP 6-237396 (Patent document 1) proposes to perform a high-rate compression of input image signal and to store the compressed image information in the frame memory. However, rate of reduction of the memory capacity in the technique proposed by Patent Document 1 is limited by the limitation of compression rate.
On the other hand, US Patent Publication US 2005-008078 (Patent document 2) proposes to supply Y-element values alone from the frame memory and to perform LAO (level-adaptive overdrive) process only to the Y-element values. According to Patent document 2 (3rd embodiment and FIG. 5), a human eye recognizes a significant improvement of display characteristic by performing LAO process only to Y-element (luminance element). Accordingly, amount of task of the LAO process block can be decreased.
The technique proposed by Patent document 2 only requires storing Y-element values in the frame memory. Accordingly, it enables to reduce the capacity of the frame memory by ⅓ compared with a case that all of R-, G-, and B-element values are stored. However, when the overdrive is performed by only using Y-element values as the previous frame color image data, display quality may be degraded due to, for example, color blurring at boundaries between objects with different colors.
It would be advantageous to provide image processing apparatuses and methods of processing color image data that can reduce capacity of frame memory without degrading display quality.
This disclosure provides image processing apparatuses and methods of processing color image data that restore R-, G-, and B-element values of respective pixels of previous one of successive frames based on Y-element values of the respective pixels of the previous one of the successive frames and color image data of a current one of the successive frames. The apparatuses and methods further compare the restored R-, G-, and B-element values of the respective pixels of the previous one of the successive frames and R-, G-, and B-element values of corresponding pixels of the current one of the successive frames to generate corrected color image data.
Various exemplary embodiments of this disclosure provide image processing apparatuses that receive color image data of successive frames and output corrected color image data. The apparatuses may include a restoration block and a correction block. The restoration block restores R-, G-, and B-element values of respective pixels of previous one of the successive frames based on the Y-element values of the respective pixels of the previous one of the successive frames, which may be read from a frame memory, and the color image data of a current one of the successive frames, which is next to the previous one of the successive frames. The correction block compares the R-, G-, and B-element values of the respective pixels of the previous one of the successive frames that the restoration block restored and R-, G-, and B-element values of corresponding pixels of the current one of the successive frames and generates the corrected color image data.
According to some exemplary embodiments, the restoration block may restore the R-, G-, and B-element values of the respective pixels of the previous one of the successive frames based on the R-, G-, and B-element values of the corresponding pixels of the current one of the successive frames and the Y-element values of the respective pixels of the previous one of the successive frames.
According to some exemplary embodiments, the restoration block may include a UV element value generation circuit that generates U- and V-element values of the corresponding pixels of the current one of the successive frames based on the R-, G-, and B-element values of the corresponding pixels of the current one of the successive frames, and the restoration block may restore the R-, G-, and B-element values of the respective pixels of the previous one of the successive frames based on the U- and V-element values of the corresponding pixels of the current one of the successive frames that the UV element value generation circuit generated and the Y-element values of the respective pixels of the previous one of the successive frames.
According to some exemplary embodiments, the restoration block may include a Y element value generation circuit that generates Y-element values of the corresponding pixels of the current one of the successive frames based on the R-, G-, and B-element values of the corresponding pixels of the current one of the successive frames, and the restoration block may restore the R-, G-, and B-element values of the respective pixels of the previous one of the successive frames based on the Y-element values of the corresponding pixels of the current one of the successive frames that the Y element value generation circuit generated, the Y-element values of the respective pixels of the previous one of the successive frames, and the R-, G-, and B-element values of the corresponding pixels of the current one of the successive frames.
According to some exemplary embodiments, the apparatuses may further include a compression block that compresses received color image data into a first compressed image data that includes one of i) R-, G-, and B-element values and ii) Y-, U-, and V-element values and a second compressed image data that only includes Y-element values, and selects one of the first and second compressed image data to be stored in a frame memory. The compression block may further include an evaluation circuit that performs an evaluation of at least one of the received color image data and the first compressed image data and performs a selection of one of the first and second compressed image data based on a result of the evaluation, and a detection circuit that detects a start of each of the frames in the received color image data and permits the evaluation circuit to update the selection only during a predetermined first period in each of the frames.
Various exemplary embodiments of this disclosure may also provide methods of processing color image data that include receiving color image data of successive frames, storing Y-element values of respective pixels of a previous one of the successive frames in a frame memory, and restoring R-, G-, and B-element values of the respective pixels of the previous one of the successive frames based on the Y-element values of the respective pixels of the previous one of the successive frames read from the frame memory and the color image data of a current one of the successive frames. The methods may further include comparing the restored R-, G-, and B-element values of the respective pixels of the previous one of the successive frames and R-, G-, and B-element values of corresponding pixels of the current one of the successive frames to generate a corrected color image data, and outputting the corrected color image data.
The file of this patent contains at least one drawing executed in color. Copies of this patent with color drawing(s) will be provided by the Patent and Trademark Office upon request and payment of the necessary fee.
Various exemplary embodiments of this disclosure will be described in detail with reference to the following figures, wherein like numerals reference like elements, and wherein:
The Y element value generation circuit 12 generates values of Y elements (Y-element values) from values of R, G, and B elements (R-, G-, and B-element values) of respective pixels in a current frame included in the color image data (RGB input). Any methods for generating Y-element values may be used. In this exemplary embodiment, Y-element values are calculated using an equation of Y=0.299R+0.587G+0.114B, where R, G, and B represents values of R, G, and B elements.
The frame memory 14 is a semiconductor memory that stores Y-element values of pixels constituting a frame received from the Y element value generation circuit 12. Stored Y-element values of respective pixels constituting a frame are read from the frame memory at timings of one-frame period later than the input of the R-, G-, and B-element values of respective pixels. As a result, Y-element values of pixels of a previous frame are read from the frame memory 14. In other words, the frame memory 14 stores Y-element values of pixels of a previous frame when the values are read.
The OD amount calculation block 16 restores R-, G-, and B-element values of respective pixels of the previous frame based on Y-element values of respective pixels of the previous frame read from the frame memory 14 and R-, G-, and B-element values of corresponding pixels of the current frame. That is, the OD amount calculation block 16 includes a restoration block that restores R-, G-, and B-element values of pixels of the previous frame. Then, the OD amount calculation block calculates overdrive amounts for respective pixels of the current frame based on the restored R-, G-, and B-element values of respective pixels of the previous frame and R-, G-, and B-element values of corresponding pixels of the current frame.
Finally, the adder 18 adds the overdrive amounts for respective pixels of the current frame that the OD amount calculation block 16 calculated and R-, G-, and B-element values of corresponding pixels of the current frame to generate corrected color image data (RGB output) for overdriving a liquid crystal display.
That is, a combination of the OD amount calculation block 16 and the adder 18 constitutes an exemplary correction block, which compares values of RGB elements of pixels constituting the previous frame and values of RGB elements of corresponding pixels constituting the current frame and generates corrected image data. The correction block may have various different constructions. For example, it is not necessary to generate the corrected color image data by generating amounts of overdrive in the OD amount calculation block 16 and adding the generated amounts to R-, G-, and B-element values of the current frame. That is, the OD amount calculation block 16 may generate the corrected color image data to which the overdrive amounts are added.
Next, exemplary embodiments of the OD amount calculation block 16 will be explained.
The UV element value generation circuit 20 generates U- and V-element values of pixels of the current frame from R-, G-, and B-element values of pixels of the current frame (current frame RGB). Here, U- and V-element values may be generated from R-, G-, and B-element values using various techniques. For example, U- and V-element values may be generated from R-, G-, and B-element values using equations similar to the equation for Y-element values explained above.
The RGB element value restoration circuit 22a restores R-, G-, and B-element values of respective pixels of the previous frame based on i) the Y-element values of respective pixels of the previous frame read from the frame memory 14 (previous frame Y) and ii) the U- and V-element values of corresponding pixels of the current frame generated by the UV element value generation circuit 20. That is, the UV element value generation circuit 20 and the RGB element value restoration circuit 22a constitute an exemplary restoration block that restores R-, G-, and B-element values of the previous frame.
The look-up tables 24R, 24G, and 24B compare R-, G-, and B-element values, respectively, of respective pixels of the previous frame that the RGB element value restoration circuit 22a restored and R-, G-, and B-element values of corresponding pixels of the current frame. Moreover, the look-up tables 24R, 24G, and 24B generates OD amounts for R, G, and B elements (OD amount for R, OD amount for G, and OD amount for B), respectively, corresponding to R, G, and B elements of respective pixels of the current frame. Here, the look-up-tables 24R, 24G, and 24B constitute an exemplary correction block according to this disclosure that outputs corrected image data.
The OD amount calculation block 16a shown in
Here, when differences between U- and V-element values of respective pixels of the previous frame and U- and V-element values of corresponding pixels of the current frame are negligibly small, the RGB element value restoration circuit can restore exact R-, G-, and B-element values of the previous frame. Accordingly, it is possible to generate exact OD amounts for the current frame based on the restored R-, G-, and B-element values of respective pixels of the previous frame and R-, G-, and B-element values of corresponding pixels of the current frame.
When differences between U- and V-element values of respective pixels of the previous frame and U- and V-element values of corresponding pixels of the current frame are significantly large, the R-, G-, and B-element values of pixels of the previous frame that the RGB element value restoration circuit 22a restored include errors. Accordingly, the OD amounts generated based on the restored R-, G-, and B-element values of respective pixels of the previous frame and R-, G-, and B-element values of corresponding pixels of the current frame include errors.
Nonetheless, errors included in the OD amounts generated based on restored R-, G-, and B-element values of respective pixels of the previous frame and R-, G-, and B-element values of corresponding pixels of the current frame are smaller compared with errors included in OD amounts generated solely based on Y-element values of the previous and current frames. As a result, display quality can be improved. That is, for example, color blurring at boundaries between objects with different colors can be suppressed.
Next, another exemplary OD amount calculation block 16 will be explained.
The Y element value generation circuit 26 generates Y-element values of pixels of the current frame from R-, G-, and B-element values of pixels of the current frame. The Y element value generation circuit 26 may utilize various techniques of generating Y-element values. For example, the Y element value generation circuit 26 may use the calculation equation described above.
The RGB element value restoration circuit 22b restores R-, G-, and B-element values of pixels of the previous frame based on Y-element values of pixels of the previous frame read from the frame memory 14, Y-element values of pixels of the current frame that the Y element value generation circuit 26 generated, and R-, G-, and B-element values of pixels of the current frame. Here, the Y element value generation circuit 26 and the RGB element value restoration circuit 22b constitute an exemplary restoration block according to this disclosure.
The look-up tables 24R, 24G, and 24B may have the same constructions as those shown in
The OD amount calculation circuit 16b shown in
Next, an equivalency between the process in the RGB element value restoration circuit 22a shown in
Assuming that A′ to F′ are appropriate coefficients to convert from Y-, U-, and V-element values to R-, G-, and B-element values, the process in the RGB element value restoration circuit 22a may be expressed by following equations.
R (previous)=Y (previous)+A′×U+B′×V
G (previous)=Y (previous)+C′×U+D′×V
B (previous)=Y (previous)+E′×U+F′×V (1)
Here, U and V represent U- and V-element values, respectively, of corresponding pixels of the current frame.
On the other hand, assuming that differences between U- and V-element values of successive frames are negligibly small, R-, G-, and B-element values of respective pixels of the current frame may be expressed by following equations.
R (current)=Y (current)+A′×U+B′×V
G (current)=Y (current)+C′×U+D′×V
B (current)=Y (current)+E′×U+F′×V (1)
The equations above may be transformed by moving “Y(current)” to the left side as follows.
R (current)−Y (current)=+A′×U+B′×V
G (current)−Y (current)=+C′×U+D′×V
B (current)−Y (current)=+E′×U+F′×V (3)
Accordingly, the equations (1) of the RGB element value restoration circuit 22a shown above may be transformed to following equations.
R (previous)=Y (previous)+R (current)−Y (current)
G (previous)=Y (previous)+G (current)−Y (current)
B (previous)=Y (previous)+B (current)−Y (current) (4)
That is, R-, G-, and B-element values of respective pixels of the previous frame may be expressed by Y-element values of respective pixels of the previous frame, Y-element values of corresponding pixels of the current frame, and R-, G-, and B-element values of corresponding pixels of the current frame. The process in the RGB element value restoration circuit 22b is performed according to these equations.
The analysis explained above shows that the RGB element value restoration circuits 22a and 22b perform equivalent processes.
The RGB element value restoration circuit 22a requires U- and V-element values, or values of two of Y, U, and V elements, that the UV element value generation circuit 20 generated from R-, G-, and B-element values of pixels of the current frame. On the other hand, the RGB element value restoration circuit 22b requires Y-element values, or values of only one of Y, U, and V elements, that the Y element value generation circuit 26 generated from R-, G-, and B-element values of pixels of the current frame. Further, the process in the RGB element value restoration circuit 22b represented by the equations (4) do not include multiplications. Accordingly, the process of the OD amount calculation block 16b is easier, and may be implemented with a smaller circuit, than the process of the OD amount calculation block 16a.
When RGB format color image data is input, as shown in
On the other hand, when YUV format color image data is input, Y-element values of pixels of the input color image data may be stored in the frame memory. And R-, G-, and B-element values of pixels of the previous frame may be restored by using Y-element values of pixels of the previous frame read from the frame memory and the color image data of pixels of the current frame. In this case, the RGB element value restoration circuit 22a shown in
Next, operation of the image processing apparatus 10 will be explained. During a period that the image processing apparatus 10 receives color image data of a frame, the image processing apparatus generates, by using the Y element value generation circuit 12, Y-element values of respective pixels from R-, G-, and B-element values in color image data (RGB input) of the current frame. Further, the image processing apparatus 10 stores generated Y-element values of the current frame in the frame memory 14.
Next, during a period that the image processing apparatus receives color image data of a next frame, by using the OD amount calculation block 16, the image processing apparatus calculates overdrive amounts for respective pixels of the current frame base on Y-element values of pixels of the previous frame read from the frame memory 14 and R-, G-, and B-element values of corresponding pixels of the current frame.
Finally, by using the adder 18, the image processing apparatus 10 adds calculated overdrive amounts for respective pixels of the current frame and R-, G-, and B-element values of corresponding pixels of the current frame, and generates corrected color image data (RGB output) for overdriving liquid crystal displays.
In the image processing apparatus 10, the frame memory 14 only stores Y-element values of pixels of each frame. Accordingly, compared with a case that all of R-, G-, and B-element values are stored in the frame memory 14, the capacity of the frame memory can be decreased to about ⅓. It is also possible to further decrease the capacity of the frame memory by, for example, quantizing or compressing the Y-element values that the Y element value generation circuit 12 generated.
Also in such cases, the OD amount calculation block 16 restores R-, G-, and B-element values of pixels of the previous frame based on R-, G-, and B-element values of respective pixels of the current frame and Y-element values of corresponding pixels of the previous frame. Thereby, color blurring at boundaries between objects with different colors can be suppressed.
Depending on characteristics of the color image data, a rate of compression may be increased. As a result, compressed image data including values of entire color elements may be stored in the frame memory 14 having a limited capacity. Accordingly, it is possible to construct an image processing apparatus to 1) store compressed image data including values of entire color elements in the frame memory 14 when it is possible, and 2) store compressed image data that only includes Y-element values in the frame memory when it is impossible to store compressed image data including values of entire color elements.
For example, values of entire color elements of simple images such as an image that a single object moves with a constant velocity before a homogeneous background may be compressed with a high compression rate and can be stored in the frame memory 14. In such simple images, human eyes can easily recognize effects of overdrive. Accordingly, it might be impossible to realize a sufficient display quality by calculating OD amounts by using Y-element values of the previous frame alone.
It is possible to construct an image processing apparatus to store values of entire color elements in the frame memory 14 and utilize them in the overdrive process when an image quality realized by the overdrive process by using Y-element values alone is insufficient. By reading compressed image data including values of entire RGB elements of the previous frame and comparing them with values of RGB elements of the current frame, a highly accurate overdrive process is enabled. Thereby, higher display quality can be realized.
On the other hand, for example, when a spatial frequency of the image is high, or when a large number of objects are randomly arranged in entire portions of the frame, it is difficult to increase the compression rate. In such cases, it is necessary to compress Y-element values alone to store in the frame memory having a limited capacity. However, in such cases, human eyes cannot easily recognize effects of overdrive. Accordingly, a sufficient display quality can be realized by storing Y-element values alone in the frame memory 14 and performing the overdrive.
The exemplary compression block 40 shown in
In this case, a de-compression block that expands the compressed image data of the previous frame read from the frame memory 14 may be provided between the frame memory 14 and the OD amount calculation block 16. The de-compressed image data may be compared with image data of the present frame.
The YUV element value generation circuit 42 generates Y-, U-, and V-element values from R-, G-, and B-element values of the input image data. The calculation equation described above may be used to generate the Y-element values. U- and V-element values may be generated by using calculation equations of, for example, U=0.500R−0.419G−0.081B and V=−0.169R−0.332G+0.500B. When the input image data is represented by Y-, U-, and V-element values, on the other hand, the YUV element value generation circuit 42 is not required.
Next, the Y-, U-, and V-element values generated by the YUV element value generation circuit 42 are quantized by the quantization circuit 44 to generate quantized Y-, U-, and V-element values. The quantized Y-, U-, and V-element values are input to the first and second compression circuits 46a and 46b. The first compression circuit 46a compresses Y-, U-, and V-element values and generates compressed image data including all of Y-, U-, and V-element values. The second compression circuit 46b only compresses Y-element values and generates compressed image data including Y-element values alone.
The first and second compression circuits 46a and 46b compress input image data by, for example, grouping a plurality of pixels and performing a variable-length coding. As a result, sizes of the compressed data, or the compression rate, varies depending on characteristics of the input image data. For example, the compression rate may depend on spatial frequency of the input image data. More specifically, when the spatial frequency is low, the compression rate becomes high and the size of compressed image data decreases.
The selector 50 selects one of the first compressed image data including all of Y-, U-, and V-element values that the first compression circuit 46a generated and the second compressed image data only including Y-element values that the second compression circuit 46b generated, and output selected one of the image data as the compressed image data. The image evaluation circuit 48 evaluates input image data or the compressed image data, and generates and outputs a selection signal based on the result of the evaluation to the selector 50.
The image evaluation circuit 48 may perform the evaluation by, for example, measuring a data size of the compressed image data and generate the selection signal. Specifically, for example, the image evaluation circuit 48 may measure a size of the first compressed image data that the first compression circuit 46a generated. When the size of the first compresses image data is not larger than a standard value, the image evaluation circuit 48 determines that the first compressed image data of a frame can be stored in the frame memory 14 and generates a selection signal that selects the first compressed image data. When the size of the first compressed image data is larger than the standard value, on the other hand, the image evaluation circuit 48 generates a selection signal that selects the second compressed image data.
When the image data evaluation and the selection signal generation in the image evaluation circuit 48 is performed, although it is omitted in
It is also possible to generate the selection signal by evaluating RGB or YUV image data before the compression. For example, it is possible to evaluate frequency and amplitude of variation of each element in a certain number of pixels. When the frequency and amplitude of variation are not larger than respective standard values, the image evaluation circuit 48 may determine that a high compression rate can be obtained and that the first compressed image data can be stored in the frame memory 14. In this case, the image evaluation circuit may generate a selection signal that selects the first compressed image data. When the frequency and amplitude of variation are larger than respective standard values, on the other hand, the image evaluation circuit 48 may generate a selection signal that selects the second compressed image data.
It is further possible to generate and store compressed image data including all of R-, G-, and B-element values in the frame memory 14, instead of compressed image data including Y-, U-, and V-element values. In this case, a compression circuit that generates compressed image data including all of R-, G-, and B-element values is provide as the first compression circuit 46a, and input image data including R-, G-, and B-element values is input to this compression circuit.
When the selection of compression circuit changes within a frame, the change of image quality may become noticeable. In order to prevent this phenomenon, it is possible to provide a detection circuit that detects starts of a frame and lines within each frame in the compression block 40. The detection circuit may generate a control signal that permits the image evaluation circuit 48 to update the selection signal based on the result of image evaluation only during a predetermined first period, or during a first few lines, and prohibits the image evaluation circuit to update the selection signal thereafter in each frame. Starts of a frame and lines may be detected by monitoring a level of vertical synchronization signal and a level of data valid signal input with the image data.
The compressed image data including all of Y-, U, and V element values stored in the frame memory 14 may be read from the frame memory and the de-compression block may restore the Y-, U-, and V-element values of the previous frame. The restored Y-, U-, and V-element values of the previous frame may be input to, for example, the RGB element value restoration circuit 22a of the OD amount calculation block 16a shown in
The selector provided in the OD amount calculation block 16a may include means to hold the selection signal supplied to the selector 50 of the compression block 40 shown in
When the OD amount calculation block 16b shown in
When compressed image data including R-, G-, and B-element values is stored in the frame memory 14, the R-, G-, and B-element values of the previous frame read from the frame memory and de-compressed by the de-compression block may be input, instead of R-, G-, and B-element values of the previous frame restored by the RGB element value restoration circuit 22a or 22b shown in
An effect of an exemplary embodiment of this disclosure for an exemplary natural image shown in
In practice, the Y element value generation circuit 12, the restoration block, which may be constituted by the UV element value generation circuit 20 and the RGB element value restoration circuit 22a or the Y element value generation circuit 26 and the RGB element value restoration circuit 22b, and the correction block, which may be constituted by the LUT 24R, 24G and 24B and the adder 18, may be integrated in a signal semiconductor integrated circuit chip. The semiconductor integrated circuit chip can be used as an apparatus to process color image data together with a frame memory that store the Y-element values of respective pixels.
Further, the semiconductor integrated circuit chip and a frame memory chip may be assembled in a signal package to constitute a device that can be used as a complete image processing apparatus. Note that, it is not necessary to integrate the Y element value generation circuit 12 in the semiconductor integrated circuit chip when YUV format color image data is input. Further, the compression block 40 may be integrated in the semiconductor integrated circuit chip instead of the Y element value generation circuit 12.
Various exemplary apparatuses and methods of this disclosure restore R-, G-, and B-element values of pixels of previous frame based on Y-element values of pixels of the previous frame and color image data of the current frame, and generate corrected image data by comparing the restored RGB-element values of pixels of the previous frame and RGB-element values of corresponding pixels of the current frame. Accordingly, it is only required to store Y-element values in a frame memory and a capacity of the frame memory can be reduced. Furthermore, degradation of display quality can be suppressed.
The exemplary embodiment described above utilizes RGB color format as inputting and outputting color image formats. It is also possible to utilize other color formats such as YUV color format. Constructions of the Y element value generation circuit and the OD amount calculation block may be modified as long as their functions are realized.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2010-93036 | Apr 2010 | JP | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 5982424 | Simerly et al. | Nov 1999 | A |
| 8098256 | Zipnick et al. | Jan 2012 | B2 |
| 20050008078 | Kurumisawa | Jan 2005 | A1 |
| Number | Date | Country |
|---|---|---|
| A-6-237396 | Aug 1994 | JP |
| Number | Date | Country | |
|---|---|---|---|
| 20110255778 A1 | Oct 2011 | US |
