1. Field of the Invention
The present invention relates to an image processing technique and, more particularly, to image processing when a display device displays a moving image.
2. Description of the Related Art
Moving image display devices represented by a TV set can be classified into hold-type display devices and impulse-type display devices. A hold-type display device continues displaying a single image in one frame interval ( 1/60 sec when the frame rate is 60 Hz). A liquid crystal display device and organic EL display using TFTs are known as hold-type display devices. On the other hand, an impulse-type display device displays an image only in the scanning interval of one frame interval so the pixel luminances start lowering immediately after the scanning. A CRT (Cathode Ray Tube) and FED (Field-Emission-type Display) are known as impulse-type display devices.
A hold-type display device is known to have a problem that a viewer readily perceives blurs of a moving object displayed on the screen (motion blurring). To cope with the blurs, the hold-type display device raises the driving frequency of its display to shorten the hold time. For example, Japanese Patent Laid-Open No. 2006-184896 discloses a technique (to be referred to as driving distributing hereinafter) which generates two sub frames from one input frame, that is, a sub frame without a high frequency component and a sub frame containing an emphasized high frequency component, and alternately displays two sub frames generated in correspondence with each frame.
On the other hand, an impulse-type display device is more advantageous in moving image visibility than a hold-type display device. However, since the device emits light only instantaneously in each frame interval ( 1/60 sec when the frame rate is 60 Hz), and repeats light emission at the period of 1/60 sec, a problem of flickering may arise. Flickering is more noticeable on a larger screen, and therefore tends to be a serious problem especially in the recent trend shifting toward display devices with wider screens. The impulse-type display device adopts, as a measure against flickering, a technique of raising the driving frequency of its display.
However, the present inventor found by experiments that even when the frame rate is raised by driving distributing, flickering is observed if the frame rate of the moving image before processing is relatively low. More specifically, if the moving image before processing has about 50 frames or less per sec, and a frame image includes a write (bright) image portion and a black (dark) image portion adjacent to each other, relatively strong flickering is observed at the peripheral portion of the white image portion.
The present invention provides a higher-quality display image for a viewer when a display device displays a moving image.
According to one aspect of the present invention, an image processing apparatus which generates and outputs a plurality of sub frame images having different luminance patterns for each of a plurality of frame images included in input moving image data, comprises: a filtering unit configured to perform filter processing for the frame image; a setting unit configured to set a first coefficient A and a second coefficient B so as to satisfy 0<A/(A+B)<1; a first generation unit configured to generate a first sub frame image by subtracting an image obtained by multiplying a frame image processed by the filtering unit by the second coefficient B from an image obtained by multiplying a frame image unprocessed by the filtering unit by the first coefficient A; a second generation unit configured to generate a second sub frame image by adding the image obtained by multiplying the frame image unprocessed by the filtering unit by the first coefficient A to the image obtained by multiplying the frame image processed by the filtering unit by the second coefficient B; and an output control unit configured to selectively output the first sub frame image and the second sub frame image at a predetermined timing.
According to another aspect of the present invention, a method of controlling an image processing apparatus which generates and outputs a plurality of sub frame images having different luminance patterns for each of a plurality of frame images included in input moving image data, the image processing apparatus including a filtering unit configured to perform filter processing for the frame image, the method comprises the steps of: setting a first coefficient A and a second coefficient B so as to satisfy 0<A/(A+B)<1; generating a first sub frame image by subtracting an image obtained by multiplying a frame image processed by the filtering unit by the second coefficient B from an image obtained by multiplying a frame image unprocessed by the filtering unit by the first coefficient A; generating a second sub frame image by adding the image obtained by multiplying the frame image unprocessed by the filtering unit by the first coefficient A to the image obtained by multiplying the frame image processed by the filtering unit by the second coefficient B; and selectively outputting the first sub frame image and the second sub frame image at a predetermined timing.
According to the present invention, it is possible to provide a higher-quality display image for a viewer when a display device displays a moving image.
Further features of the present invention will become apparent from the following description of exemplary embodiments (with reference to the attached drawings).
The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.
Preferred embodiments of the present invention will now be described in detail in accordance with the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the invention, but are merely examples.
As the first embodiment of an image processing apparatus according to the present invention, an image processing apparatus 100 which outputs an image to a panel module serving as a display device will be exemplified below. Note that in the following description, “frame frequency” indicates the number of frames displayed per sec in progressive scanning, or the number of fields displayed per sec in interlaced scanning.
<Technical Premise>
The display characteristics of the hold-type display device and impulse-type display device described above in “BACKGROUND” will be described in more detail. An example of moving image data of 60 frames per sec (60 Hz) will be explained here.
Hold-Type Display Device
As shown on the left view of
Impulse-Type Display Device
As shown on the left view of
<Arrangement of Apparatus>
A frame frequency conversion circuit 101 converts the frame frequency of an input original image to a higher frequency. A minimum value filter 102 is configured to substitute the value of a pixel of interest of the input image with the minimum pixel value out of the peripheral pixels around the pixel of interest, and output the image. A Gaussian filter 103 performs filter processing using, for example, a Gaussian function for the input image. A delay circuit 104 outputs the image output from the frame frequency conversion circuit 101 to a multiplication circuit 105 to be described later at a timing adjusted in consideration of the delay of processing from the minimum value filter 102 to the Gaussian filter 103.
The multiplication circuit 105 performs multiplication of the image (O image) output from the delay circuit 104. A multiplication circuit 106 performs multiplication of the image (F image) output from the Gaussian filter 103. A subtraction circuit 107 subtracts the image output from the multiplication circuit 106 from the image output from the multiplication circuit 105, and outputs a “first sub frame” (M image). An addition circuit 108 adds the image output from the multiplication circuit 106 to the image output from the multiplication circuit 105, and outputs a “second sub frame” (S image). A selector circuit 109 (output control unit) selectively sequentially outputs the first sub frame and second sub frame at a predetermined timing. Note that the output image from the selector circuit 109 is output to, for example, the panel module. Note that the F image is formed from the low frequency component of the original frame image because it is obtained by processing the original frame image by the Gaussian filter 103, as is apparent.
<Operation of Apparatus>
Flickering Threshold Frequency
[Reference 1] Moses, R. A. and Hart, W. M. (ed), Adleris Physiology of the eye, Clinical Application. St. Louis: The C.V. Mosby Company, 1987.
Driving Distributing
That is, when the first sub frame (waveform 401) and the second sub frame (waveform 402) are alternately displayed, the waveform 403 is perceived as brightness. However, flickering occurs at the peripheral portion of the object image, as indicated by the waveform 404.
This will be explained in more detail with reference to
As described with reference to
Consider decreasing the luminance difference between sub frames using the multiplication circuits 105 and 106, subtraction circuit 107, and addition circuit 108 shown in
The waveform 603 that is the same as the waveform 602 can be regarded as a component that is displayed twice in one frame interval ( 1/50 sec). On the other hand, the luminance component of the waveform 604 can be regarded as a component that is displayed once in one frame interval ( 1/50 sec). However, as for the luminance ratio of the waveforms 603 and 604, the luminance ratio is not only low at the central portion of the luminance waveform but also not so high even at the peripheral portion, unlike the case in
As is apparent from
Note that the multiplication coefficients of the multiplication circuits 105 and 106 need to be set in association with the frequency of the input image. Letting A be the multiplication coefficient (first coefficient A) of the multiplication circuit 105, and B be the multiplication coefficient (second coefficient B) of the multiplication circuit 106,
0<A/(A+B)<1
holds. However, when A/(A+B)≈1, double blurring of a moving image is intense. When A/(A+B)≈0, flickering is storing at the peripheral portion of a white (bright) image. The present inventor inspected actual images and found that
0.2<A/(A+B)<0.8
is effective. More preferably, a higher-quality image in which both double blurring and flickering are suppressed is obtained within the range of
0.4<A/(A+B)<0.6
Example of Application to Movie (24p) Content Display
An example has been described above in which two sub frames (sub frame images) are generated from each of a plurality of frame images included in moving image data of 50 frames per sec (50 Hz), and a moving image of 100 frames per sec (100 Hz) is output. However, the present invention is also applicable to any other input frame rate and any other output frame rate.
An example will be described below in which a video having a frame frequency of 24 Hz used in many movie contents is displayed. Referring to
In general, a director makes a picture with unique blurring in 48-Hz display at a movie theater in mind, and checks the video actually displayed on a screen at 48 Hz. However, since the luminance of a commercially available FPD is higher than in a movie theater, the viewer senses flickering in 48-Hz display as in a movie theater. To prevent this, the commercially available FPD displays a content at a frequency three times (72 Hz) or five times (120 Hz). However, simple 72- or 120-Hz display yields blurring different from that in 48-Hz display at a movie theater. The display cannot reproduce the same blurring as in a movie theater, and thus cannot exactly reflect the director's intention.
In this state, the waveform 705 that is the same as the waveform 702 can be regarded as a component that is displayed four times in one frame interval ( 1/24 sec). On the other hand, the luminance component of the waveform 706 can be regarded as a component that is displayed only twice in one frame interval ( 1/24 sec). With the waveform 706 of the 48-Hz component, motion blurring unique to the 48-Hz image is obtained, unlike an image simply displayed at 96 Hz. This motion blurring has the same frequency as in 48-Hz display in screening at a movie theater where each of 24 frames is displayed twice, and is very close to that in screening at a movie theater. However, the waveform 706 of the 48-Hz component increases the flickering intensity at the peripheral portion, as indicated by a waveform 707 representing a flickering region.
An example will be explained below in which a “first sub frame” (M image) is generated by setting the multiplication coefficient of the multiplication circuit 105 to 1.0 and that of the multiplication circuit 106 to 0.4, and a “second sub frame” (S image) is generated by setting the multiplication coefficient of the multiplication circuit 105 to 0.3 and that of the multiplication circuit 106 to 0.4.
In this state, the waveform 805 that is the same as the waveform 802 can be regarded as a component that is displayed four times in one frame interval ( 1/24 sec). On the other hand, the luminance component of the waveform 806 can be regarded as a component that is displayed only twice in one frame interval ( 1/24 sec). With the waveform 806 of the 48-Hz component, motion blurring unique to the 48-Hz image is obtained, unlike an image simply displayed at 96 Hz. This motion blurring has the same frequency as in 48-Hz display in screening at a movie theater where each of 24 frames is displayed twice, and is very close to that in screening at a movie theater. Focusing on the contrast ratio at 48 Hz, the contrast ratio at the central portion is about 0.1, that is, the same as in
More specifically, as is apparent from
Note that in the above description, two kinds of multiplication coefficients for two kinds of sub frames are used for four sub frames generated in one frame interval. However, all multiplication coefficients may be set to be different to generate four kinds of sub frames. For example, the coefficient of the multiplication circuit 105 may be set to be 1.0, 0.5, 0.9, and 0.6, and the coefficient of the multiplication circuit 106 may be set to be 0.3, 0.3, 0.2, and 0.2 for the first, second, third, and fourth sub frames, respectively. Defining four sub frames as one set to display the same frame every 24 Hz makes it possible to impart motion blurring unique to 24-Hz display and obtain display much closer to that at a movie theater.
As described above, according to the first embodiment, it is possible to display a moving image while suppressing both double blurring and flickering. This enables to display a higher-quality moving image for the user. Note that it is possible to not only generate the above-described 50-Hz (50i) moving image and 24-Hz (24p) moving image but also generate, from a moving image having an arbitrary frame rate, a moving image of a frame rate M (M is an even number) times the frame rate of the moving image.
Note that flickering in the above-described driving distributed processing is observable in both the hold-type display device and the impulse-type display device. Hence, the above-described image processing apparatus can obtain the same effect for both the hold-type display device and the impulse-type display device.
Note that although simple “luminance” distributed processing has been described above, the processing may be performed for the luminance (Y) component of an image expressed by YCbCr components or for the pixel value of each of the RGB colors (the luminance value of each color) of an RGB image.
Consider decreasing the luminance difference between sub frames using multiplication circuits 105 and 106, subtraction circuit 107, and addition circuit 108 shown in
According to the same viewpoint as in the first embodiment, the waveform 2203 can be regarded as a component that is displayed twice in one frame interval ( 1/50 sec). On the other hand, the luminance component of the waveform 2204 can be regarded as a component that is displayed once in one frame interval ( 1/50 sec). However, the luminance ratio is low not only at the central portion of the image but also at the peripheral portion. More specifically, focusing on the contrast ratio at 50 Hz, the contrast ratio at the central portion is about 0.1, and that at the peripheral portion is about 0.3.
As is apparent from
As described above, according to the second embodiment, it is possible to display a moving image while suppressing both double blurring and flickering. This enables to display a higher-quality moving image for the user. As can be seen, a high-quality image can be obtained.
(Modification)
Note that the present inventor also found that when the first sub frame (waveform 401) and the second sub frame (waveform 402) shown in
Aspects of the present invention can also be realized by a computer of a system or apparatus (or devices such as a CPU or MPU) that reads out and executes a program recorded on a memory device to perform the functions of the above-described embodiment(s), and by a method, the steps of which are performed by a computer of a system or apparatus by, for example, reading out and executing a program recorded on a memory device to perform the functions of the above-described embodiment(s). For this purpose, the program is provided to the computer for example via a network or from a recording medium of various types serving as the memory device (for example, computer-readable medium).
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Application No. 2009-251358, filed Oct. 30, 2009 which is hereby incorporated by reference herein in its entirety.
Number | Date | Country | Kind |
---|---|---|---|
2009-251358 | Oct 2009 | JP | national |