Patent Application
20130241947
The technique disclosed in the present specification relates to a display device obtained by combining display panels and lenses like e.g. a head-mounted display, an image processing device, an image processing method, and a computer program, and particularly to a display device, an image processing device, an image processing method, and a computer program that correct image distortion attributed to distortion involved in the lens by signal processing.
A display device mounted on the head to view images, i.e. the head-mounted display (HMD), is widely known. The head-mounted display has an optical unit for each of the left and right eyes and is so configured as to foe used in combination with a headphone to allow control of senses of vision and hearing. If it is so configured that the vision of the external world is completely blocked when it is mounted on the head, the feeling of virtual reality in viewing images increases. Furthermore, it is also possible for the head-mounted display to display different images for the left and right eyes, and a 3D (three-dimensional) image can be presented if images having a parallax are displayed for the left and right eyes.
As display sections for the left and right eyes in the head-mounted display, e.g. a high-resolution display panel formed of a liquid crystal or an organic electro-luminescence (EL) element can be used. Furthermore, if the image from the image display element is projected in an enlarged manner by an eyepiece optical system to set a wide angle of view and multiple channels are reproduced by a headphone, it will foe possible to reproduce a feeling of presence as if the user viewed the image at a movie theater.
It is known that the optical lens has distortion. For example, when a wide angle of view is ensured in a head-mounted display, there is a fear that complicated distortion and color deviation occur when a displayed image is viewed attributed to distortion of the lens used in the eyepiece optical system and thus the quality deteriorates.
Furthermore, if the number of lenses configuring the eyepiece optical system is increased to ensure a wide angle of view, the weight of the head-mounted display increases and therefore the burden of the user who wears it becomes larger. If the number of lenses is decreased for weight reduction, the distortion occurring in the respective lenses becomes larger and the lens system to correct the distortion becomes absent. As a result, it becomes difficult to ensure a wide angle of view.
A method of correcting the distortion occurring in the eyepiece optical system by signal processing is known. Specifically, if the eyepiece optical system has a distortion shown in
The distortion involved in the lens has a characteristic of slightly changing depending on the wavelength of light. Specifically, the distortion involved in the eyepiece optical system is as shown in
For example, proposals have been made about a method in which image deterioration due to a chromatic aberration of the optical system is also corrected by individually performing distortion correction for image signals of the respective colors of RGB (refer to e.g. Japanese Patent Laid-opens Ho. Hei 9-61750, No. Hei 9-113823, No. 2001-186442, No. 2004-233869, and No. 2006-258802).
However, there is a fear that the imbalance of RGB occurs as an adverse effect caused when correction processing is executed independently for each of the color components of RGB. The imbalance of RGB is observed as a pseudo color with a hue at e.g. a thin white line and a white bright spot in the image.
There is a need for the technique disclosed in the present specification to provide an excellent display device, image processing device, image processing method, and computer program that can suitably correct image distortion attributed to distortion involved in the lens by signal processing when an image is displayed based on the combination of a display panel and a lens.
There is another need for the technique disclosed in the present specification to provide an excellent display device, image processing device, image processing method, and computer program that can suitably correct image distortion attributed to distortion involved in the lens by signal processing for each color component.
There is another need for the technique disclosed in the present specification to provide an excellent display device, image processing device, image processing method, and computer program that can suitably correct image distortion attributed to distortion involved in the lens by signal processing with suppression of an adverse effect due to signal processing independent for each color component.
According to an embodiment of the present technique, there is provided a display device including an image corrector configured to execute correction processing of an input image independently for each color component, a display section configured to display an output image of the image corrector, and an eyepiece optical section configured to project a displayed image of the display section in such a manner that a predetermined angle of view is obtained. The image corrector executes, about each color component, correction processing of distortion generated by the eyepiece optical section after executing de-gamma processing of an input image for which gamma processing has been executed, and executes re-gamma processing to output a resulting image.
According to another embodiment of the present technique, there is provided an image processing device including, for each color component, a de-gamma processor configured to execute de-gamma processing of an input image signal for which gamma processing has been executed, an image corrector configured to execute correction processing of distortion generated in projection by a predetermined eyepiece optical section for a linear input image resulting from the de-gamma processing, and a gamma processor configured to execute re-gamma processing of a linear image resulting from correction and output a resulting image.
According to a further embodiment of the present technique, there is provided an image processing method including, for each color component, executing de-gamma processing of an input image signal for which gamma processing has been executed, executing correction processing of distortion generated in projection by a predetermined eyepiece optical section for a linear input image resulting from the de-gamma processing, and executing re-gamma processing of a linear image resulting from correction and outputting a resulting image.
According to a still further embodiment of the present technique, there is provided a computer program that is described in a computer-readable format and is to cause a computer to function as an entity including, for each color component of an input image, a de-gamma processor configured to execute de-gamma processing of an input image signal for which gamma processing has been executed, an image corrector configured to execute correction processing of distortion generated in projection by a predetermined eyepiece optical section for a linear input image resulting from the de-gamma processing, and a gamma processor configured to execute re-gamma processing of a linear image resulting from correction and output a resulting image.
The computer program according to the embodiment of the present technique is defined as a computer program described in a computer-readable format so that predetermined processing may be realized on a computer. In other words, by installing the computer program according to the embodiment of the present technique in a computer, cooperative operation is exerted on the computer and the same operation and effects as those of the image processing device according to the embodiment of the present technique can be achieved.
According to the technique disclosed in the present specification, it is possible to provide an excellent display device, image processing device, image processing method, and computer program that can suitably correct image distortion attributed to distortion involved in the lens by signal processing with suppression of an adverse effect due to signal processing independent for each color component.
According to the technique disclosed in the present specification, in a display device obtained by combining a display panel end a lens, particularly the occurrence of color unevenness and the degradation of fineness as an adverse effect by signal processing independent for each color component can tae prevented and it becomes possible to display images with higher image quality.
Further other desires, features, and advantages of the technique disclosed in the present specification will become apparent from more detailed description based on an embodiment, to foe described later and the accompanying drawings.
An embodiment of the technique disclosed in the present specification will be described in detail below with reference to the drawings.
The front end box 40 is equivalent to an HDMI repeater that executes e.g. signal processing for an HDMI-input AV signal output from the Blu-ray disc reproduction device 20 and HDMI-outputs the resulting signal. Furthermore, the front end box 40 serves also as a two-output switcher that switches the output destination of the Blu-ray disc reproduction device 20 to either the head-mounted unit 10 or the high-definition display 30. Although the front end box 40 has two outputs in the example shown in the diagram, it may have three or more outputs. However, the front end box 40 makes the output destination of the AV signal for exclusive and places the highest priority on the output to the head-mounted unit 10.
The HDMI (high-definition multimedia interface) is an interface standard that is mainly used for the purpose of transmitting audio and video and aimed at digital home appliances. The HDMI is based on the digital visual interface (DVI) and uses the transition minimised differential signaling (TMDS) as a physical layer. This system conforms to e.g. HDMI 1.4.
A connection by an HDMI cable is made between the Blu-ray disc reproduction device 20 and the front end box 40 and between the front end box 40 and the high-definition display 30. Although it is also possible to make a connection by an HDMI cable also between the front end box 40 and the head-mounted unit 10, the AV signal may be serially transferred by using a cable based on another specification. However, the AV signal and power are supplied by one cable connecting the front end box 40 and the head-mounted unit 10, and the head-mounted unit 10 can also obtain driving power via this cable.
The head-mounted unit 10 includes independent display sections for the left eye and the right eye. Each display section uses a display panel formed of e.g. an organic EL element. Furthermore, the left and right respective display sections are equipped with a low-distortion, high-resolution eyepiece optical system with a wide viewing angle. If the image from the image display element is projected in an enlarged manner by the eyepiece optical system to set a wide angle of view and multiple channels are reproduced by a headphone, a feeling of presence as if the user viewed the image at a movie theater can be reproduced.
There is a fear that distortion is generated in a viewed image of the display panel attributed to distortion of the lens used in the eyepiece optical system. The distortion of the viewed image can be corrected by an optical system. However, in this method, a lens for distortion correction is added and therefore there is a fear that the weight of the head-mounted unit 10 increases and the burden of the user who wears it increases. So, in the present embodiment, a method of correcting the distortion generated in the eyepiece optical system by signal processing is employed.
The “signal processing” here is equivalent to processing to give the presented image a distortion in the opposite direction to that of the distortion generated in the projected image of the eyepiece optical system.
An image is input from an image source like the Blu-ray disc reproduction device 20 to an HDMI receiver 201. A distort ion is generated about the respective pixels of this input image due to passage through an eyepiece optical system 204. An image corrector 202 gives a distortion in the opposite direction to the respective pixels of the presented image to thereby perform motion compensation (MC), i.e. compensate for the displacement of the respective pixels generated due to the distortion, to generate a display image to which the preliminary opposite-distortion is applied. The distortion in the opposite direction, given to the pixels, will be referred to as the motion vector (MV) hereinafter. The start point of the motion vector is a pixel position on the input image and the end point thereof is the pixel position corresponding to this start point on the display image.
A display section 203 displays, on a display panel, the input image resulting from the correction with the distortion in the opposite direction by the image corrector 202. This displayed image is projected onto the retina of the eye of the viewer via the eyepiece optical system 204. Although a distortion is generated when the displayed image passes through the eyepiece optical system 204, a normal virtual image including no distortion is formed on the retina because the distortion in the opposite direction to that of this distortion has been given to the displayed image.
The image corrector 202 may be provided in either the head-mounted unit 10 or the front end box 40. Given that an image distortion based on the distortion parameter possessed by the lens configuring the eyepiece optical system 204 in the head-mounted unit 10 is corrected, providing the image corrector 202 in the head-mounted unit 10 allows the front end box 40 to output an image signal without being conscious of which head-mounted unit 10 is the output destination of the image signal.
The distortion involved in the lens configuring the eyepiece optical system 204 has a characteristic of slightly changing depending on the wavelength of light. Therefore, the image corrector 202 should execute the correction processing about the input image independently for each of the color components of RGB. However, there is a fear that the imbalance of RGB occurs as an adverse effect caused when the correction processing is executed independently for each of the color components of RGB.
In the following, a consideration will be made about the adverse effect caused when the correction processing is executed for the input image independently for each of the color components of RGB.
The following description is based on the assumption that the respective distortion correction blocks 301, 302, and 303 perform correction only in the horizontal direction for simplification of explanation. Furthermore, linear interpolation is employed as the interpolation method in the correction in the following description. Of course, the following description similarly holds even when the respective distortion correction blocks 301, 302, and 303 execute two-dimensional interpolation processing in the horizontal and vertical directions or multi-tap interpolation processing such as cubic interpolation.
An input image signal din and a reference signal ref(k) are input to the distortion correction block 301. The input image signal din is written into an image memory 401. The reference signal ref(k) represents a pixel position mk of the input image signal din to which an output image signal dout(k) of the k-th pixel position refers. However, the pixel position mk of the input image signal din to which the output image signal dout(k) refers is not necessarily an integer. Thus, the integer part of the reference signal ref(k) is represented as and the decimal part is represented as sk. The output image signal dout(k) is equivalent to the end point of a motion vector MV and ref(k) is equivalent to the start point of the motion vector. That is, the pixel position mk is the position resulting from distortion in the opposite direction to that of the distortion generated in the eyepiece optical system 204 regarding the k-th pixel of the output image.
In accordance with the value of the integer part mk of the reference signal ref(k), values din(mk) and din(mk+1-th) of the input image signal of adjacent mk-th and mk+1-th pixel positions are output from the image memory 401.
An interpolator 402 performs linear interpolation of the values din(mk) and din(mk+1) of the input image signal of adjacent two pixels, read out from the image memory 401, based on the value of the decimal part sk of the reference signal ref(k) as shown by the following expression (1) to obtain the output image signal dout(k) of the k-th pixel position.
dout(k)=(1−sk)×din(mk)+sk×din(mk+1) (1)
A more detailed consideration will foe made below about the behavior in this distortion correction block 301 when a bright spot of one pixel exists in the input image din.
In general, the distortion generated in an image by the lens gently changes in the screen. Therefore, in the vicinity of the k-th output image dout(k), the reference signal ref(k) can be approximated as shown by the following expression (2).
ref(k+Δk)=mk+skΔk (2)
The point to which attention should be paid here is that the input image signal din has been subjected to gamma processing. In general, the image signal is subjected to bit reduction by gamma processing using a gamma curve like that shown in
In the examples shown in
Based on the above, a consideration will be made below about the case in which a 100% white bright spot of one pixel exists in the input image. The reference signals of RGB are different from each other because of the chromatic aberration involved in the eyepiece optical system 204. For example, if the decimal part of the reference signal ref(k) at a certain output pixel position k is various, specifically R: sk=0.2, G: sk=0.5, and B: sk=0.8, as shown in
It will be effective to damp the input image signal by using a low-pass filter so that the image signal having a sharp change like a bright spot of one pixel may be prevented from being input to the image corrector 202. However, this scheme has a problem that fineness possessed by the original video is lost.
So, in the present embodiment, distortion correction is performed, after the input image signal is subjected to de-gamma processing to be temporarily converted to a linear image, and thereafter gamma processing is executed again to output the resulting image.
The input image signal din and the reference signal ref(k) are input to the distortion correction block 301.
The input image signal din is subjected to de-gamma processing by a de-gamma processor 1501 disposed at the input stage and a linear input image signal din′ as its output is written into an image memory 1502.
The reference signal ref(k) represents the pixel position of the input image signal din to which the output image signal dout(k) of the k-th pixel position refers. The integer part mk of ref(k) is input to the image memory 1502 and the decimal part sk is input to an interpolator 1503.
In accordance with the value of the integer part mk of the reference signal ref(k), values din′ (mk) and din′ (mk+1) of the linear input image signal of adjacent mk-th and mk+1-th pixel positions are output from the image memory 1502.
The interpolator 1503 performs linear interpolation of the values din′ and din′ (mk+1) of the linear input image signal of adjacent two pixels, read out from the image memory 1502, based on the value of the decimal part sk of the reference signal ref(k) as shown by the following expression (3) to obtain a corrected image signal dout′ (k) of the k-th pixel position.
dout′(k)=(1=sk)×din′(mk)+sk×din′(mk+1) (3)
A gamma processor 1504 disposed at the output stage executes re-gamma processing of the linear corrected image signal dout′ (k) and outputs an output image signal dout(k).
A more detailed consideration will be made below about the behavior in the distortion correction block 301 shown in
Comparison of the respective diagrams of
Furthermore, the signal value of the output image signal dout(k) resulting from the re-gamma processing of the corrected image signal dout′ (k) also changes depending on the value of the decimal part sk of the reference signal ref(k). When sk=0.2, 0.5, and 0.8, the total of the signal value is 49+90=139%, 70+70=140%, and 90+49=139%, respectively. Moreover, when the respective output image signals dout(k) are converted to the luminance, it turns out that the total of the luminance is 20+80=50+50=80+20=100% in each case and the 100% bright spot is distributed into plural pixels.
Based on the above, a consideration will be made below about the case in which a 100% white bright spot of one pixel exists in the input image. The reference signals of RGB are different from each other because of the chromatic aberration of the eyepiece optical system 204. For example, if the decimal part of the reference signal ref(k) at a certain output pixel position k is various, specifically R: sk=0.2, G: sk=0.5, and B: sk=0.8, as shown in
As above, by performing image correction by using the distortion correction black 301 shown in
It is also possible for the technique disclosed in the present specification to employ the following configurations.
(1) A display device including: an image corrector configured to execute correction processing of an input image independently for each color component; a display section configured to display an output image of the image corrector; and an eyepiece optical section configured to project a displayed image of the display section in such a manner that a predetermined angle of view is obtained, wherein the image corrector executes, about each color component, correction processing of distortion generated by the eyepiece optical section after executing de-gamma processing of an input image for which gamma processing has been executed, and executes re-gamma processing to output a resulting image.
(2) The display device according to the above-described (1), wherein the image corrector interpolates a pixel of the output image by a plurality of corresponding pixels on a linear input image resulting from the de-gamma processing.
(3) An image processing device including, for each color component: a de-gamma processor configured to execute de-gamma processing of an input image signal for which gamma processing has been executed; an image corrector configured to execute correction processing of distortion generated in projection by a predetermined eyepiece optical section for a linear input image resulting from the de-gamma processing; and a gamma processor configured to execute re-gamma processing of a linear image resulting from correction and output a resulting image.
(4) An image processing method including, for each color component: executing de-gamma processing of an input image signal for which gamma processing has been executed; executing correction processing of distortion generated in projection by a predetermined eyepiece optical section for a linear input image resulting from the de-gamma processing; and executing re-gamma processing of a linear image resulting from correction and outputting a resulting image.
(5) A computer program that is described in a computer-readable format and is to cause a computer to function as an entity including, for each color component of an input image: a de-gamma processor configured to execute de-gamma processing of an input image signal for which gamma processing has been executed; an image corrector configured to execute correction processing of distortion generated in projection by a predetermined eyepiece optical section for a linear input image resulting from the de-gamma processing; and a gamma processor configured to execute re-gamma processing of a linear image resulting from correction and output a resulting image.
The technique disclosed in the present specification is explained in detail above with reference to a specific embodiment. However, it is obvious that those skilled in the art can make modifications and alternatives of the embodiment without departing from the gist of the technique disclosed in the present specification.
Although the embodiment in which the technique disclosed in the present specification is applied to a head-mounted display is mainly described in the present specification, the gist of the technique disclosed in the present specification is not limited to the configuration of a specific head-mounted display. The technique disclosed in the present specification can be similarly applied also to various types of display system that presents images to the user based on the combination of a display panel and a lens.
In short, the technique disclosed in the present, specification is explained above based on a form of exemplification and the described contents of the present specification should not be interpreted in a limited manner. To determine the gist of the technique disclosed in the present specification, the scope of claims should be taken into consideration.
The present disclosure contains subject matter related to that disclosed in Japanese Priority Patent Application JP 2012-05893 filed in the Japan Patent Office on Mar. 15, 2012, the entire content of which is hereby incorporated by reference.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2012-058937 | Mar 2012 | JP | national |
