The present invention relates to an image display device, an image display system, and an image display method.
In the related art, a stereoscope called a synopter is known as an optical device that displays the same video to left and right eyes using a monocular stereopsis principle. The synopter divides light received in the same position by combining half mirrors and supplies the divided light to two eyes. According to the synopter, it is known that retinal images of two eyes are identical and stereoscopic depth is given to a non-stereo image (for example, see Non-Patent Literature 1).
Non-Patent Literature 1: Jan J Koenderink et al., “On so-called paradoxical monocular stereoscopy,” Perception, Pion Publication (UK), 1994, volume 23, pp. 583-594
However, if a right-eye image and a left-eye image are generated from an original two-dimensional (2D) image using the synopter principle and displayed on a display, a display region of the display is physically limited. Thus, there is a problem in that an image of an end part of each image is lost or an invalid region where there is no image at the end part occurs.
The loss region or the invalid region as described above occurs in one of the right-eye image and the left-eye image and does not occur in the other. Thus, when a viewer visually recognizes both the right-eye image and the left-eye image converted from 2D into three-dimensional (3D) using a monocular stereopsis principle, there is a problem in that a screen flickering phenomenon called a binocular vision field conflict is caused and display quality is degraded.
The present invention has been made in view of the above-described problems, and an object of the invention is to provide a novel and improved image display device, image display system, and image display method that can prevent display quality from being degraded due to a loss region of a screen end part or an invalid region when an image converted from 2D into 3D is displayed using a monocular stereopsis principle.
According to an aspect of the present invention in order to achieve the above-mentioned object, there is provided an image display device including: a mask addition unit for adding a mask to an input image of a 2D image on the basis of a parameter for converting the 2D image into a 3D image by a monocular stereopsis principle; a conversion unit for converting the input image to which the mask is added by the mask addition unit into a right-eye image and a left-eye image by the monocular stereopsis principle; and a display unit for displaying the right-eye image and the left-eye image.
The image display device may include a mask amount calculation unit for calculating a range of the mask, wherein, if an invalid region occurs in one of the right-eye image and the left-eye image when the mask is not added, the mask amount calculation unit calculates a range in which a region corresponding to the invalid region for the other of the right-eye image and the left-eye image is not visibly recognized as the mask range.
The image display device may include a mask amount calculation unit for calculating a range of the mask, wherein, if a loss region occurs in one of the right-eye image and the left-eye image when the mask is not added, the mask amount calculation unit calculates a range in which a region corresponding to the loss region for the other of the right-eye image and the left-eye image is not visibly recognized as the mask range.
The conversion unit may perform conversion by a parallel shift type.
The conversion unit may perform conversion by a tilt-shift plane attachment type.
According to another aspect of the present invention in order to achieve the above-mentioned object, there is provided an image display observation system including: an image display device including a mask addition unit for adding a mask to an input image of a 2D image on the basis of a parameter for converting the 2D image into a 3D image by a monocular stereopsis principle, a conversion unit for converting the input image to which the mask is added by the mask addition unit into a right-eye image and a left-eye image by the monocular stereopsis principle, and a display unit for displaying the right-eye image and the left-eye image; and stereoscopic video observation glasses, having shutters for right and left eyes, for opening and closing the shutters for the right and left eyes according to switching of the right-eye image and the left-eye image in the display unit.
According to another aspect of the present invention in order to achieve the above-mentioned object, there is provided an image display method including the steps of: adding a mask to an input image of a 2D image on the basis of a parameter for converting the 2D image into a 3D image by a monocular stereopsis principle; converting the input image to which the mask is added by the mask addition unit into a right-eye image and a left-eye image by the monocular stereopsis principle; and displaying the right-eye image and the left-eye image.
According to the present invention, display quality can be prevented from being degraded due to a loss region of a screen end part or an invalid region when an image converted from 2D into 3D is displayed using a monocular stereopsis principle.
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the appended drawings. Note that, in this specification and the drawings, elements that have substantially the same function and structure are denoted with the same reference signs, and repeated explanation is omitted.
Description will be given in the following order.
1. Assumed Technology
2. Overview of Mask Processing according to This Embodiment
3. Configuration Example of Image Display Device according to This Embodiment
4. Example of 2D-3D Conversion of 2D-3D Conversion Unit
5. Configuration Example of Stereoscopic Image Display Observation System
[1. Assumed Technology]
There are various techniques of converting a 2D image into a 3D image. The most common technique is a method of obtaining depth information (a depth map) of an object included in a 2D input image in any method and adding parallax generated from the depth information for each object or region of the 2D input image.
One method unlike the above-described method allows a viewer to feel a stereoscopic effect by performing deformation processing for a 2D image without using the depth map or the like. For example, there is a method of performing projection conversion for an image towards a direction of each of left and right eyes on the display surface so that the same video appears in retinas of the left and right eyes by applying a synopter principle (a tilt-shift plane attachment type), or the like. Also, there is a method of simply adding uniform parallax for the entire 2D image or each line and adding a gradient of depth while moving an image before or after a display screen.
The former method using the depth information is aimed at implementing “binocular stereopsis” by generating different parallax for each object. On the other hand, the latter method using the synopter principle is a type of deriving a “monocular stereopsis” ability of the viewer from a 2D image viewing state by removing information indicating that “display is performed on the display surface” from the user.
Because only the right-eye image R is visually recognized in the right eye of the user and only the left-eye image L is visually recognized in the left eye in the techniques of
Incidentally, because different parallax for each object of the image is not added if a 3D image is generated in a type using the “monocular stereopsis” principle, an anteroposterior relationship of a position of a depth direction for each object is absent. Thus, a portion viewed by only one eye called occlusion does not occur in any portion on the screen. In other words, if a 3D image is stereoscopically viewed after conversion from 2D into 3D is performed in a type of performing conversion from 2D into 3D using the “monocular stereopsis” principle, a pair of images are always present on the left and right. If this condition is not satisfied, inconsistency occurs in viewing and the viewer feels a sense of strangeness. Because the inconsistency is generated by the presence of an image viewed by only one eye, in spite of the absence of the anteroposterior relationship of the depth direction, this is referred to as binocular vision field conflict.
However, because a parallel shift of an image, enlargement/reduction, projection conversion processing, or the like is performed in processing of the type of performing conversion from 2D into 3D using the “monocular stereopsis” principle, parts of image ends of the right-eye image R and the left-eye image L after the conversion from 2D into 3D may not be displayed on the display screen due to a limitation of a display screen size. In contrast, a region (an invalid image region) outside a valid image may be displayed on the display screen. In this case, because a portion in which a pair of images are absent on the left and right occurs if the display screen is in binocular stereopsis, the binocular vision field conflict occurs in the portion and viewing difficulties such as flickering of the screen or the like may occur. Furthermore, this phenomenon may not be eliminated in processing (for example, over-scan processing, mask processing, or the like) after conversion processing from 2D into 3D.
These phenomena will be described on the basis of
At this time, an invalid image is displayed because a range of a left width WR2 on the display screen exceeds a valid range of the right-eye image R if the right-eye image R is displayed at a uniform distance w on the left and right by designating a point OR as the center on the virtual input image surface. Because no image is displayed in a range of a right width WR1 of the display screen if left and right widths of the display screen are not sufficient, an image of a right end of the right-eye image R is lost.
Likewise, an invalid image is also displayed in the left-eye image L because the range of the right width WL1 on the display screen exceeds a valid range of the left-eye image L if the left-eye image L is displayed at a uniform distance w on the left and right by designating a point OL as the center on the virtual input image surface. Because no image is displayed in the range of the left width WL2 of the display screen if left and right widths of the display screen are not sufficient, an image of a left end of the left-eye image L is lost.
While the invalid image of the width WR2 and the loss of the width WR1 are visually recognized in the right eye if left and right images are alternately displayed in the above-described state, they are not visually recognized in the left eye. While the invalid image of the width WL1 and the loss of the width WL2 are visually recognized in the left eye, they are not visually recognized in the right eye. Thus, an image viewed by only one eye is present and hence the above-described binocular vision field conflict occurs.
The same phenomenon also occurs in the tilt-shift plane attachment type shown in
[2. Overview of Mask Processing According to This Embodiment]
Thus, in this embodiment, mask processing for causing the right-eye image R and the left-eye image L to be the same as each other is performed to suppress the binocular vision field conflict caused by a difference between the right-eye image R and the left-eye image L in a technique of converting an image from 2D into 3D using the above-described “monocular stereopsis” principle.
[3. Configuration Example of Image Display Device According to This Embodiment]
As input image data, 2D image data I2D is input to the mask addition unit 102. An adjustment parameter CONT for performing 2D-3D conversion for the input image is input to the 2D-3D conversion parameter calculation unit 104. The adjustment parameter CONT is a parameter such as a viewing distance al, a screen size dw, the number of horizontal pixels of the display (1920 in a full HD size), a setting position cl of the virtual input screen (a maximum pull-in amount of a horopter surface from the display surface), or a spacing el between two eyes.
The 2D-3D conversion parameter calculation unit 104 calculates a 2D-3D conversion parameter PRM from the adjustment parameter CONT, and outputs the 2D-3D conversion parameter PRM to the optimal mask amount calculation unit 106 and the 2D-3D conversion unit 108. The adjustment parameter CONT and the 2D-3D conversion parameter PRM are not limited to one value. Input image information may be used as the adjustment parameter CONT, or the 2D-3D conversion parameter PRM may be changed according to an image region.
The optimal mask amount calculation unit 106 calculates a width of a part that is not displayed on the display surface from the calculated 2D-3D conversion parameter PRM, and calculates a minimal mask amount capable of masking its region as an optimal mask amount MPRM. At this time, the loss of image information by mask processing is minimized by minimizing a necessary mask width.
Generally, if a shape of a valid image region after masking is rectangular, the optimal mask amount MPRM has 4 independent values for the up, down, left, and right of the screen. However, the mask shape is not limited to a rectangle because the purpose of the mask is to replace a valid pixel in which a corresponding pixel is absent on the left and right by an invalid pixel. That is, the shape of the valid image region after masking is not limited to the rectangle, and may be a circle, an oval, or the like.
The mask addition unit 102 performs mask overlap processing for the input image I2D on the basis of a calculation result of the optimal mask amount MPRM calculated by the optimal mask amount calculation unit 106. The mask addition unit 102 outputs a 2D input image M2D to which the mask is added. The 2D input image to which the mask is added corresponds to an image shown in the middle of
The 2D-3D conversion unit 108 performs 2D-3D conversion processing on the basis of the 2D-3D conversion parameter PRM for the 2D input image M2D to which the mask is added, and outputs a left-eye output signal L3D and a right-eye output signal R3D. As described above, the 2D-3D conversion processing is projection processing (coordinate conversion) for a tilt-shift effect in the tilt-shift plane type. In the plane shift type, it is processing of separating the right-eye image R and the left-eye image L on the input image surface by a spacing between two eyes.
Thereby, the 2D-3D conversion unit 108 performs tilt-shift processing for the right-eye image R and the left-eye image L as shown in
[4. Example of 2D-3D Conversion of 2D-3D Conversion Unit]
As an example of conversion of the 2D-3D conversion unit 108, tilt-shift processing will be described. A left-eye image L and a right-eye image R are output from the correspondence of an input image and a display image defined in the following equations. Here, Equation 1 is a coordinate conversion equation of the left-eye image L, and Equation 2 is a coordinate conversion equation of the right-eye image.
In Equations 1 and 2,
y: horizontal pixel position of input image surface
x: horizontal pixel position of display surface
al: viewing distance (cm)
cl: distance (cm) from display surface to input image reference surface
el: spacing (about 6.5 cm) between two eyes
width: number of horizontal pixels of display (1920 in full HD)
dw: display width (cm)
scale: scaling factor for aspect radio adjustment
According to Equation 1, as shown in
In terms of calculation of a mask amount of the left end of the screen, first, a value of y (=yLL) for x=0 is obtained in Equation 1, which is a conversion equation of the left-eye image L (STEP 1). Next, in Equation 2, which is a conversion equation of the right-eye image R, a value of y (=yRL) for x=0 is obtained (STEP 2). Next, y1=MAX(yLL, yRL) is obtained (STEP 3). Next, if y1 is a value greater than 0, a left end of the input image is not displayed (is lost) on the display surface in any one of the L image and the R image after 2D-3D conversion (STEP 4). Accordingly, the mask width of the left end of the input image can be in a width range from y=0 to y=ROUNDUP (y1). ROUNDUP is performed to finally largely calculate a mask width when a value of y1 is converted into an integer.
In terms of calculation of a mask amount of the right end of the screen, first, a value of y (=yLR) for x=1919 is obtained in Equation 1, which is a conversion equation of the left-eye image L (STEP 1). Next, in Equation 2, which is a conversion equation of the right-eye image R, a value of y (=yRR) for x=1919 is obtained (STEP 2). Next, y2=MIN(yLR, yRR) is obtained (STEP 3). Next, if y2 is a value less than 1919, the right end of the input image is not displayed (is lost) on the display surface in any one of the L image and the R image after 2D-3D conversion (STEP 4). Accordingly, the mask width of the right end of the input image can be in a width range from y=ROUNDDOWN(y2) to y=1919. As described above, ROUNDDOWN is performed to finally largely calculate a mask width when a value of y2 is converted into an integer. As described above, it is preferable to convert a coordinate conversion result into an integer so that a mask amount is calculated slightly largely.
As described above, mask processing is performed in the step of the input image before 2D-3D conversion by obtaining a portion incapable of being displayed on the display surface (x=0 to x=1919) in the mask processing as a pixel position of the input image (a value of y). For example, the loss in the left end of the display surface is present in an image after processing for the left eye, but whether it is present in the image after processing for the right eye depends on a result of coordinate conversion calculation. A loss in the left end of the display surface occurs in the image after processing for the left eye in a geometric arrangement as shown in
It is also possible to obtain a mask width in the plane shift type by the same method. Because the right-eye image R and the left-eye image L are shifted by a spacing between two eyes on the input image surface in the plane shift type, a conversion equation showing a relationship between a horizontal pixel position y of an input and a horizontal pixel position x of a display surface can be obtained on the basis of the shift amount and the geometric arrangement of
[5. Configuration Example of Stereoscopic Image Display Observation System]
The image display device 100 is, for example, a time division type of stereoscopic video display device, and a left-eye video L and a right-eye video R output from the 2D-3D conversion unit 108 are alternately displayed on the entire screen of a display unit 110 in a significantly short cycle. The image display device 100 separately provides the left and right eyes with videos in synchronization with display cycles of the left-eye video L and the right-eye video R. For example, the image display device 100 alternately displays a right-eye parallax image (a right-eye image R) and a left-eye parallax image (a left-eye image L) in each field. In the display image viewing glasses 200, a pair of liquid crystal shutters 200a and 200b are provided in portions corresponding to lenses.
The image display device 100 includes an infrared transmission unit, which transmits an infrared signal in synchronization with display switching of the left-eye video L and the right-eye video R, and the viewing glasses 200 include an infrared reception unit. The liquid crystal shutters 200a and 200b alternately perform an opening/closing operation in synchronization with image switching of each field of the image display device 100 on the basis of a received infrared signal. That is, in the field in which the right-eye image R is displayed on the image display device 100, the left-eye liquid crystal shutter 200b is in a closing state and the right-eye liquid crystal shutter is in an open state 200a. In the field in which the left-eye image L is displayed, an operation opposite thereto is performed. As described above, the image display device 100 separately provides the left eye and the right eye with videos in synchronization with display cycles of the left-eye video L and the right-eye video R simultaneously when the left-eye video L and the right-eye video R are alternately displayed on the entire screen in a significantly short cycle.
According to the above-described operation, only the right-eye image R is incident to the right eye of the user viewing the image display device 100 with the viewing glasses 200, and only the left-eye image L is incident to the left eye. Thus, the user can recognize a stereoscopic video converted from 2D into 3D using the above-described monocular stereopsis principle.
According to this embodiment as described above, it is possible to eliminate a binocular vision field conflict caused by a region incapable of being displayed due to a limitation of a display screen size after conversion processing in 2D-3D conversion using the “monocular stereopsis.”
The preferred embodiments of the present invention have been described above with reference to the accompanying drawings, whilst the present invention is not limited to the above examples, of course. A person skilled in the art may find various alternations and modifications within the scope of the appended claims, and it should be understood that they will naturally come under the technical scope of the present invention.
For example, the present invention is widely applicable to an image display device, an image display system, and an image display method that display a right-eye image and a left-eye image.
Number | Date | Country | Kind |
---|---|---|---|
2010-024403 | Feb 2010 | JP | national |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/JP2011/050956 | 1/20/2011 | WO | 00 | 10/4/2011 |