Patent Application
20110007213
1. Field of the Invention
The present invention relates to an image processing technique of appropriately controlling a flicker in a moving image.
2. Description of the Related Art
Many movie contents are produced at a frame frequency of 24 Hz. However, if such a movie content is displayed directly at a period of 24 Hz, the viewers perceive blink with the same period as 24 Hz, that is, a 24-Hz flicker very strongly, and accordingly suffer video quality degradation and fatigue.
To prevent this, conventionally, light is projected to a single frame on a film twice at equal intervals so that a single image is displayed twice at an equal interval. Since the two times of display are done at an equal interval, video display at a period of 48 Hz is obtained.
This operation is implemented by a so-called Geneva mechanism. The Geneva mechanism includes film feed that temporarily stops in each frame, and a disk shutter for +2 times lighting in synchronism with the film feed (for example, U.S. Pat. Nos. 6,183,087 and 6,513,932). The 24-Hz flicker is not observed in this mechanism.
Instead, a 48-Hz flicker is newly perceived. The 48-Hz flicker that replaces the 24-Hz flicker largely improves the viewers' fatigue and suffering. However, it is a fact that the 48-Hz flicker still remains, and most people recognize it.
Electronic displays that mainly aim to display TV broadcasting have gone through evolution on condition that the frame frequency of display matches that of TV broadcasting. More specifically, the frame frequency is 60 Hz in regions where NTSC is adopted, and 50 Hz in regions of PAL/SECAM.
Since the frame frequency of many movie contents is 24 Hz, TV broadcasting of movie contents dose not succeed because of the difference in frame frequency. To solve this problem, for example, in the NTSC regions, a video having a period of 24 Hz is replaced with a video of 60-Hz period by a 3-2 pull-down method. The video of 60-Hz period is broadcast and directly displayed on an electronic display of 60-Hz period.
A DVD or Video CD also supplies a video that has undergone the 3-2 pull-down processing, and an electronic display displays it as an image of 60-Hz period.
Some recent DVDs or blu-ray disks are made assuming the 24-Hz period to directly express the atmosphere of 24-Hz movies. According to this method, instead of providing data having undergone the 3-2 pull-down processing as video information, the original video of 24-Hz period is provided and subjected to the 3-2 pull-down processing at the time of playback, thereby attaining a higher saving efficiency as recording media. This method is suitable for playing back a 24-Hz content without converting it (in fact, the content is played back at a period of 48 Hz as in a theater).
The present invention proposes a technique of displaying a 24-Hz content on an electronic display at a period of 48 Hz in an atmosphere as close as possible to a theater where a movie projector is used.
For example, there are conventionally the following methods of this type.
(1) A hold-type liquid crystal display or the like directly displays a 24-Hz movie content at a frame rate of 24 Hz.
(2) For example, an impulse-type display device simply doubles the frame rate of 24 Hz to display a content at 48 Hz.
(3) The frame rate is simply multiplied by four to display a content at 96 Hz.
These will be referred to as conventional display methods (1), (2), and (3) hereinafter.
However, a flicker or a double image that is usually supposed to be a detrimental effect has already become one of expression techniques of so-called film movies for a long time, and the detrimental effect itself has been accepted by viewers as atmosphere of movies. Some videos are created even in consideration of such expression.
Against this backdrop, there is a demand for a method of appropriately reproducing a flicker or a double image.
However, the above-described conventional display methods (1), (2), and (3) are not necessarily capable of display in the same atmosphere as in screening film movies in theaters. “Same atmosphere” means whether the above-described 48-Hz flicker can appropriately be reproduced, or whether a double image in visual tracking can be reproduced as in a movie.
To reproduce appropriate flickers as in a movie in a general viewing environment (for example, in a living room) where, for example, the screen size, angle of view, and ambient brightness are different from a theater, it is necessary to adjust and change at least the flicker level by image processing.
In the conventional display method (1), neither the 24-Hz flicker nor the 48-Hz flicker occurs fundamentally, or the flickers occur in a very small amount. Hence, the atmosphere of a theater cannot appropriately be reproduced.
The conventional display method (2) is the same as the projector in a theater in simply doubling the frame rate to display a content at 48 Hz. However, although the display duty is about 50% in screening in a theater, as described above, it is often, for example, 10% or less in an impulse-type display device. In this case, for example, when the amplitude is adjusted to equalize the average luminance, the amplitude of the 48-Hz fundamental component increases to 1.5 times or more that at a display duty of 50%. The 48-Hz flicker also increases to 1.5 times or more. That is, the conventional display method (2) cannot reproduce the atmosphere of a theater because the 48-Hz flicker is too large.
The conventional display method (3) raises the frame rate to 96 Hz so that the frequency component of the flicker falls outside the range detectable by human eye. This is generally welcomed but inappropriate from the viewpoint of reproducing the flicker expression in a theater. In addition, since the frame rate is simply multiplied by four, visual tracking yields not a double image as in a theater but a quadruple image. It is also inappropriate in a sense of reproducing the atmosphere of screening in a theater.
The present invention provides an image display apparatus capable of, even in an environment different from a theater, reproducing a video of the same atmosphere as that of a film movie screened in the theater, and a method of controlling the same.
According to one aspect of the present invention, an image display apparatus includes a multiplier which multiplies input image data by a predetermined ratio to obtain first image data, a subtracter which subtracts the first image data obtained by the multiplier from the input image data to obtain second image data, buffer memories each of which temporarily stores a corresponding one of the first image data and the second image data, a display control unit which reads and displays, in alternate order, the first image data and the second image data stored in the buffer memories, and an adjustment unit which adjusts the predetermined ratio so as to set a flicker of an image displayed by the display control unit to a predetermined level.
Further features of the present invention will become apparent from the following description of exemplary embodiments (with reference to the attached drawings).
<Arrangement of Image Display Apparatus>
Reference numeral 14 denotes a CPU. An ambient light sensor 19 detects the brightness of the environment where the display unit 13 is installed. A distance sensor 20 detects the distance between the user and the display unit 13. A user interface (user IF) 18 is a remote controller or the like. A status memory 15 stores setting information the user sets via the user IF 18, and the detected values from the ambient light sensor 19 and the distance sensor 20. An initial status memory 16 stores the initial statuses of the ambient light level and distance, codes for calculation processing, and the like. A graphic memory 17 controls the image to be displayed as OSD.
The user can perform an operation of changing parameters via the user IF 18 while viewing the state of the image displayed on the display unit 13.
The CPU 14 decides parameters for controlling a flicker based on, for example, the ambient light level obtained by the ambient light sensor 19 and viewing distance information obtained by the distance sensor 20, thereby controlling the flicker control circuit 100. The control can also be done based on user's intension obtained via the user IF 18, screen size information stored in the initial status memory 16, or the like.
<Conventional Film Movie Display Method>
Before a description of image processing according to this embodiment, display conditions upon conventionally screening a film movie content in a theater will be explained.
Many movie contents are produced at a frame frequency of 24 Hz.
However, if such a movie content is displayed directly at a period of 24 Hz, the viewers perceive blink at a period of 24 Hz, that is, a 24-Hz flicker very strongly, and accordingly suffer video quality degradation and fatigue.
To prevent this, conventionally, light is projected to a single frame on a film twice at an equal interval so that a single image is displayed twice at an equal interval. As a result, display at a period of 48 Hz is obtained.
This operation is implemented by intermittent film feed of a Geneva mechanism and a disk shutter designed to open and close in two periods in synchronism the film feed (U.S. Pat. Nos. 6,183,087 and 6,513,932).
The 24-Hz flicker is not observed in this mechanism. Instead, a 48-Hz flicker is newly perceived. The 48-Hz flicker that replaces the 24-Hz flicker largely improves the viewers' fatigue and suffering.
In the Geneva mechanism, first, the rotation force of a motor is transmitted to a star wheel via a cam. The star wheel converts the rotary motion into an intermittent rotary motion. The intermittent rotary motion is transmitted to a sprocket, thereby intermittently feeding a movie film.
In synchronism with the intermittent film feed, a disk that shields projection light releases the shutter for the projection light. The film is fed while the shutter is shielding the projection light. Conversely, when the film stands still, the shutter opens so as to project an image printed in each frame of the film. The shutter opens twice per intermittent film feed at an equal interval as a whole, thereby implementing displaying a single image twice at 48 Hz, that is, simple 2× rate display.
The shutter is formed from a disk as shown in
When such a disk is used as a shutter, projection light is projected twice at an equal interval of 1/24 sec, that is, 48 Hz. The duty ratio of the light projection time is about 50%.
In such display with lighting at the period of 48 Hz, a 48-Hz flicker is observed. Since the display is simple 2× rate display, a double image is observed by visual tracking of a moving part.
However, the 48-Hz flicker or a double image observed by visual tracking of a moving part, which is usually supposed to be a detrimental effect, has already become one of expression techniques of so-called film movies for a long time. The detrimental effect itself has been accepted by viewers as atmosphere of movies as long as it falls within the acceptable limits. Some film movie contents are created assuming such expression.
In consideration of this background, the present invention proposes a technique of intentionally reproducing flicker expression of double frequency (48 Hz) or a double image expressed in a conventional theater appropriately on an electronic display in a home or digital theater.
<Flicker Control Method>
A flicker control method according to this embodiment will be described below with reference to
Referring to
Dt0=Tp0*2/T24 (0-0)
where T24 means 1/24 sec.
Referring to
Dt1=Tp1*2/T24 (0-1)
Referring to
Dts=Tps*2/T24 (0-3)
In this embodiment,
Tps=Tp1=Tp2 (0-4)
Dts=Dt1 (0-5)
Dt=Tp*2/T24 (0-6)
The present invention proposes a method of reproducing the same flicker expression as in the display method shown in
In the display method in a conventional theater shown in
When the amplitude of the 48-Hz fundamental component in
|Fnd—0|=A0*4/π (1-1)
The 48-Hz fundamental component in each case will be represented by Fnd**.
In a “spatial frequency separation method”, input image data A is separated into low-frequency image data L and high-frequency image data H. Based on these data, a high-frequency enhanced image SH=A+H=H+L/2 and a high-frequency suppressed image SL=L/2 are created. These images are displayed in subframes in alternate order to directly reproduce the original image and simultaneously improve flicker suppression and motion blur (hold type) or double blur (impulse type).
To completely suppress the flicker by the “spatial frequency separation method”, the low-frequency component contained in the high-frequency enhanced image SH and that contained in the high-frequency suppressed image SL are set to the same level. When the ratio of the two low-frequency component levels is intentionally changed, an intended flicker can be realized without any other image quality change.
As shown in
To obtain |Fnd_s|, a waveform obtained by subtracting L2 from L of each pulse in
In the present invention, the flicker is controlled by controlling the relationship between L1 and L2 (more specifically L1−L2), thereby reproducing the same flicker as in the display method shown in
The flicker amount control range is maximum in
<Arrangement and Operation of Flicker Control Circuit>
The upper side of
The flicker control circuit 100 of this embodiment is externally controlled by changing the value of the gain α of the multiplier 102. Referring to
SW in
With this operation, the flicker control circuit 100 outputs S1S2
S3
S4 in this order.
This arrangement allows to execute calculation processing represented by
L=LPF(A) (2-1)
SH=A−α*L (2-2)
SL=αL (2-3)
A=H+L (2-4)
Hence,
SH=H+(1−α)L (2-5)
The difference ΔL between the low-frequency component of SH and that of SL is given by
ΔL=(1−2*α)*L (2-6)
For example, when α=0.5, SH=H+0.5 L, SL=0.5 L, and ΔL=0. That is, the low-frequency components of SH and SL have the same value, and no flicker exists. When α=0, SH=H+L, SL=0, ΔL=L. At this time, the flicker is maximum.
<Relationship Between Display Duty and Flicker Value>
A detailed method of determining the flicker controlled variable will be described next.
First, as shown in
|Fnd_(A,Dt)|=A*4/π*SIN(π*Dt) (3-1)
(Dt=Tp*2/T24) (3-2)
holds.
The condition in
|Fnd—0(A=A0,Dt=0.5)|=A0*4/π*SIN(π*0.5)=A0*4/π (3-3)
The present invention assumes that even when the display duties are different, the luminance levels are set to the same value (more specifically, when the duty is low, the instantaneous value of the light intensity becomes large, and when the duty is high, the instantaneous value becomes small). More specifically, the amplitude A is almost in inverse proportion to the duty. For example, to set the amplitude A when the duty is always Dt to the same luminance as in
A=A0*(Dt0/Dt)=A0*(0.5/Dt) (3-4)
That is, under this condition, equation (3-1) can be rewritten as
|Fnd(Dt)|=A0*(0.5/Dt)*4/π*SIN(π*Dt) (3-5)
That is, when the luminance is set at a predetermined level, |Fnd| is uniquely determined by Dt.
When a relative value given by
|FND_relativ(Dt)|=|Fnd(Dt)|/|Fnd—0(A=A0,Dt=0.5)| (3-6)
is defined for the case in
|FND_relativ(Dt)|=(0.5/Dt)*SIN(π*Dt) (3-7)
Equation (3-7) expresses the 48-Hz flicker relative intensity when the amplitude of light intensity is in inverse proportion to the duty as the function of Dt.
Calculation is performed based on this equation. When the impulse-type display device as shown in
Dt1=(0.5 msec)*2/( 1/24 sec)=0.048 (3-8)
This is applied to equation (3-7) to obtain
|FND—1_relativ(Dt=Dt1)|=1.565 (3-9)
As is apparent, the flicker amount in
<Flicker Controlled Variable Determination 1 (Basic Controlled Variable)>
The display waveform shown in
In this embodiment, the flicker value that increases by 1.565 times in display at the duty shown in
The waveform of this embodiment, that is, |Fnd_s| in
The original level in
Since the flicker is mainly determined by the low-frequency component, the difference between the low-frequency component levels may be used as the difference between the image level of the first subframe and that of the second subframe. More specifically, |ΔL| is used in place of A in
ΔL=(1−α)L−αL=(1−2α)L (3-11)
In the present invention, α is controlled for the display waveform shown in
From equation (3-1), we obtain
|Fnd—s|=|ΔL|*4/π*SIN(π*Dts) (3-12)
From equation (3-11), we obtain
|Fnd—s|=(1−2α)L*4/π*SIN(π*Dts) (3-13)
When
|FND—s_relative|=|Fnd—s|/|Fnd—0(A=A0,Dt=0.5)| (3-14)
is defined, like equation (3-6), |FND_s_relative|=1 in this embodiment, and
Equation (3-4) is applicable not only to A but also to L. Since L=A0*(0.5/Dts), we obtain
|FND—s_relative|=(1−2α)*(0.5/Dts)*SIN(π*Dts)
Since Dts=Dt1, this is rewritten as
|FND—s_relative|=(1−2α)*(0.5/Dt1)*SIN(π*Dt1) (3-15)
As described above, since
(0.5/Dt1)*SIN(π*Dt1)=1.565 when Dt1=0.048,
|FND—s_relative|=(1−2α)*1.565 (3-16)
When α which ensures FND_s_relative=1 is defined as α0,
|FND—s_relative|=(1−2α0)*1.565=1
Hence,
α0=0.1805 (3-17)
L1:L2=1−α0:α=0.8195:0.1805 (3-18)
Hence, in the display method shown in
This will be generalized again. When L1:L2=1−α:α,
FND
—
s_relative=|Fnd—s|/|Fnd—0(Dt0)|=1
That is, a combination of α and Dt which satisfies
(1−2α)*(0.5/Dt)*SIN(π*Dt)/SIN(π*Dt0)=1 (3-19)
can ensure FND_s_relative=1
<Flicker Controlled Variable Determination 2 (Controlled Variable Correction)>
In this embodiment, the flicker control circuit designed as shown in
As for the difference in ambient brightness, a theater screens a movie in an almost pitch dark room. On the other hand, a kind of light usually exists in, for example, a living room. That is, it is generally brighter than a theater. Hence, even when a content is displayed with an equivalent flicker, the flicker expression is insufficient. Display needs to be performed with a slightly stronger flicker. In this embodiment, the brightness of viewing environment is detected using the ambient light sensor 19 so that the controlled variable of flicker can be controlled based on it.
As for the difference in the angle of view, the angle of view recommended for a theater is 45° in many cases. Hence, a seat where the angle of view is 45° is supposed to be the best in a theater, and most contents are produced based on it. On the other hand, for an electronic display such as a 1920*1080 full HD, 3H viewing environment is recommended. At this time, the angle of view is about 35°. Hence, the angle of view is smaller in viewing at this distance, and the degree of flicker to be perceived becomes lower. From this viewpoint as well, display needs to be done with a slightly stronger flicker. In this embodiment, the distance between the user and the display unit 13 is detected using the distance sensor 20 so that the controlled variable of flicker can be controlled based on it.
The third correction factor is the set level of the display luminance of the electronic display. Generally, when the set level is high, the flicker is corrected to a relatively small amount. When the set level is low, the flicker is set to be relatively large.
The fourth correction factor is the individual difference. The present invention provides matching the flicker expression on an electronic display with that in screening in a theater, and the individual difference is basically small. However, a person who is sensitive to a flicker by nature would desire a relatively small flicker. A person who is not so sensitive would more realistically feel expression similar to that in screening in a theater by increasing the flicker. In this embodiment, the user can designate a flicker value representing the flicker intensity via the user IF 18. The flicker control circuit 100 can control the controlled variable of flicker in accordance with the designation.
To do this, in place of the value α0 which is originally supposed to be set, the set value α is newly defined in consideration of the above-described four factors. At this time, α0, α, and the factors have a relationship given by
1−α=(1−α0)*(1+k1*Env/Ldisp+k2*Ldisp+k3*AngleCinema/AngleDisp+k4*Prsnl) (4)
where Env is the ambient light level, Ldisp is the distance luminance set level, AngleDisp is a parameter representing the angle of view of the electronic display, AngleCinema is a parameter representing the angle of view in a theater, Prsnl is a parameter representing the intensity of individual taste, and k1, k2, k3, and k4 are positive proportional coefficients.
When the correction represented by equation (4) is applied to it, the value α becomes larger or smaller than α0. More specifically, the position determined by the actual value α in
Accordingly, the value of Fnd_s_relativ also changes upward or downward. That is, this allows to adjust the value |Fnd_s_relativ| on the electronic display to a value with which the same flicker expression as in screening in a theater can be obtained in consideration of the influence of the environmental factors.
Note that in this embodiment, the effect does not change even when the positions of even-numbered signals and odd-numbered signals are exchanged. That is, the same effect can be obtained.
In the second embodiment, a case of a hold-type display device will be described.
The overall arrangement of an image display apparatus, the arrangement of a flicker control circuit, and the operation timing of the flicker control circuit according to this embodiment are the same as in the first embodiment (
Note that this embodiment will be explained by exemplifying a case in which DtH<DtL. The effect of the present invention can be obtained even in normal 2× rate driving, that is, DtH=DtL. When DtH<DtL, the maximum flicker value increases, and the flicker adjustment range widens. Hence, the effect of the present invention becomes clearer.
In this embodiment, the ratio of the low-frequency component of the first subframe to that of the second subframe is controlled using the “spatial frequency separation method”, thereby reproducing a 48-Hz flicker of intended level, as in the first embodiment.
In this embodiment, the flicker is controlled by controlling the relationship between L1 and L2 in
<Flicker Adjustment Range>
The flicker adjustment range according to this embodiment is determined by the ratio of DtH and DtL. An adjustment range when the display duty of the first subframe is DtH is obtained below.
The flicker is maximized at a predetermined duty, that is, when α=1, as in the first embodiment. At this time, the light intensity is wholly distributed to the first subframe at the duty DtH.
The 48-Hz fundamental component at this time is obtained by substituting Dt=DtH into equation (3-7). Hence, we obtain
|Fnd—4_relativ_Max|=(0.5/DtH)*SIN(π*DtH) (5-1)
When, for example, DtH=0.3 is set as a realistic value,
|Fnd—4_relativ_Max|=(0.5/0.3)*SIN(π*0.3)=1.19 (5-2)
This is the maximum relative flicker value.
As described in the first embodiment, the relative flicker value can be adjusted by the value α. As shown in
|Fnd—4_relativ|=|Fnd—4_relativ_Max|=1.19
Similarly, as shown in
|Fnd—4_relativ|=0
This enables to control |Fnd—4_relativ| from 0 to 1.19 by the value α. More specifically, a flicker level within the range of 0 to 1.19 times that in the case shown in
Note that in this embodiment, the effect does not change even when the positions of even-numbered signals and odd-numbered signals are exchanged. That is, the same effect can be obtained.
The overall arrangement of an image display apparatus according to the third embodiment is the same as in the first and second embodiments (
In this embodiment, no processing concerning the spatial frequency like the above-described “spatial frequency separation method” is performed. S1, S2, S3, and S4 in
The upper side of
A multiplier 201 multiplies input image data As by a predetermined ratio α, thereby obtaining first image data. The first image data output from the multiplier 201 is S_even. A subtracter 203 subtracts S_even from the input signal As to generate S_odd that is second image data.
Hence, S_odd and S_even are given by
S
—
odd=As*(1−α) (6-1)
S_even=As*α (6-2)
In this embodiment, data corresponding to |ΔL| in the first embodiment is
|S_odd−S_even|
From equations (6-1) and (6-2),
|S_odd−S_even|=(1−2α)*As (6-3)
holds.
Hence, letting Dt5 be the duty determined by one pulse width Tps according to this embodiment (that is, Dt5=Tps*2/T24), the 48-Hz fundamental component of the display method shown in
|Fnd—s|=(1−2α)As*4/π*SIN(π*Dt5) (6-4)
like equation (3-13).
Hence, letting Dt5 be the duty of the first and second subframes according to this embodiment, the relative value of the 48-Hz fundamental component in the display method shown in
Since
As=A0*Dt0/Dts=A0*0.5/Dts (6-6)
the relationship to α is represented by the same result as equation (3-15), that is,
|FND—s_relative|=(1−2α)*(0.5/Dts)*SIN(π*Dts) (6-7)
That is, in this embodiment, the same control as in the first embodiment can be done using α and Dts. In the third embodiment, however, the manner the double image appears in visual tracking changes from that of the first embodiment in accordance with the level of the flicker adjustment value. More specifically, when the flicker is maximum (that is, α=0), the double image is the same as in the first embodiment or as in the display method shown in
In this embodiment, from the viewpoint of reproducing screening in a theater, the performance is poor because the double image is spoiled. However, there are advantages in terms of cost because, for example, no low-pass filter is necessary, and the circuit scale becomes smaller.
Note that in this embodiment, the effect does not change even when the positions of even-numbered signals and odd-numbered signals are exchanged. That is, the same effect can be obtained.
Aspects of the present invention can also be realized by a computer of a system or apparatus (or devices such as a CPU or MPU) that reads out and executes a program recorded on a memory device to perform the functions of the above-described embodiment(s), and by a method, the steps of which are performed by a computer of a system or apparatus by, for example, reading out and executing a program recorded on a memory device to perform the functions of the above-described embodiment(s). For this purpose, the program is provided to the computer for example via a network or from a recording medium of various types serving as the memory device (for example, computer-readable medium).
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Application No. 2009-163005, filed Jul. 9, 2009, which is hereby incorporated by reference herein in its entirety.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2009-163005 | Jul 2009 | JP | national |
