Patent Application
20120062556
This application is based upon and claims the benefit of priority from Japanese Patent Application No. 2010-204762, filed on Sep. 13, 2010; the entire contents of which are incorporated herein by reference.
Embodiments described herein relate generally to a three-dimensional image display technique.
In recent years, at a movie theater, a way of viewing a three-dimensional image has been spread in which the observer wears glasses and views different images (parallax images) with the left and right eyes at corresponding positions. In addition, as a glasses-free three-dimensional image display technique, a technique has been proposed in which a parallax barrier, such as pinholes, slits, or a lens array, is provided on the display surface of a flat panel display (FPD) and the sub-pixels viewed by the observer are changed according to an observation position so as to allow the observer to recognize parallax information corresponding to the observation position of the observer or the positions of the left and right eyes of the observer.
In the above-mentioned techniques, when there is a large parallax, the observer perceives the depth of an image such that objects seem to burst out from the screen, and at other times, objects seem to reach deep behind the screen. When there is no parallax, the observer perceives the image of an object as a planar image.
As a technique for displaying a high-reality three-dimensional image, there is a technique for enabling the observer to view the side of an object by using motion parallax that results from movement of the observation position of the observer.
In the three-dimensional image display technique, it is desirable to display a high-quality three-dimensional image regardless of the observation position of the user.
According to an embodiment, a three-dimensional image display apparatus includes a display unit; a detecting unit configured to detect an observation position of an observer relative to the display unit; a determining unit configured to determine an amount of parallax of an input image signal to be reduced as an angle increases between a normal direction of a surface of the display unit and an observation direction based on the detected observation position or as a distance decreases between the surface of the display unit and the detected observation position; and a generating unit configured to generate a multi-view image to be displayed on the display unit on the basis of the determined amount of parallax.
Various embodiments will be described hereinafter with reference to the accompanying drawings. The following embodiments explain a case where an alignment method for a pattern image formed on a reticle is used in an examination apparatus that detects presence or absence of defect of a pattern image formed on a reticle, as an example; however, the embodiments are not limited to this.
In the following embodiments, for convenience of explanation, a scale of each component is adjusted in order to have a recognizable size in the drawings, and the directions in the drawings, such as the vertical and horizontal directions, are relative and may be different from those based on the gravity direction.
In the following embodiments, an example of a three-dimensional image display apparatus that displays a three-dimensional image using a display method, which is called an integral imaging method (hereinafter, referred to as an II method) which naturally prevents flipping (the flipping of images) with a mass of beams using an array of lenticular lenses extended in the vertical direction, as the glasses-free method will be described. However, the invention is not limited to the II method, but can be applied to a glasses-type three-dimensional image display apparatus other than the glasses-free three-dimensional image display apparatus.
As shown in
The input unit 105 receives a multi-view image or a three-dimensional image signal from an image generating apparatus, such as a camera, or receives a three-dimensional image signal through a decoder of an image reproducing apparatus.
The observation position detecting unit 101 detects the observation position of the observer relative to the display unit 104. In this embodiment, an acceleration sensor that measures the angle of the three-dimensional image display apparatus 10 with respect to the gravity direction is provided and the observation angle of the observer with respect to the display unit 104 is detected as the observation position from the output of the acceleration sensor.
The observation position detecting unit 101 is not limited thereto. The observation position detecting unit 101 may be configured as follows: a head tracking sensor for estimating the direction of the face or head recognized using the image captured by a camera is provided in the three-dimensional image display apparatus 10 and the observation position detecting unit 101 detects, as the observation position, the observation angle of the observer with respect to the front surface of the display unit 104 from the output of the head tracking sensor. In addition, the observation position detecting unit 101 may be configured as follows: a distance sensor for measuring the distance between the observer and the display unit 104 is provided in the three-dimensional image display apparatus 10 and the distance of the observer from the display unit 104 is detected as the observation position from the output of the distance sensor.
The parallax amount determining unit 102 determines the amount of parallax of the parallax information of the three-dimensional image signal received by the input unit 105 to be reduced as an angle increases between a normal direction of a surface of the display unit 104 and an observation direction based on the observation position or as a distance decreases between the surface of the display unit 104 and the observation position, in order to generate an element image array to be displayed on the display unit 104 by the display image generating unit 103. In this embodiment, the parallax amount determining unit 102 determines the amount of parallax to be continuously reduced as the observation angle θ with respect to the front surface of the display unit 104, serving as the observation position, increases. Alternatively, the parallax amount determining unit 102 may determine the amount of parallax serving as the observation position to be reduced as the distance from the front surface of the display unit 104 is reduced. The parallax amount determining unit 102 will be described in detail below.
The element image means a set of parallax images displayed by sub-pixels corresponding to an exit pupil (aperture 116), which will be described below, and the element image array means an element image group displayed on the display unit 104.
The display image generating unit 103 generates the element image array (multi-view image) composed of the element images on the basis of the amount of parallax determined by the parallax amount determining unit 102 and displays the generated element image array on the display unit 104.
The display unit 104 is a device that displays the multi-view image generated by the display image generating unit 103.
The aperture control unit 115 is a beam control element that limits beams and emits the beams in a predetermined direction. As shown in
For example, a lenticular sheet, which is an array plate of lenses having a generating line in a direction vertical to the screen of the display element array 114, is used as the aperture control unit 115. Each of the apertures 116 of the lens segment is arranged so as to correspond to the pixel. The aperture control unit 115 is not limited to the array plate having the lenticular segment and the light transmission region integrated with each other, but it may be an LCD serving as an optical shutter in which the position and shape of the light transmission region can be changed over time.
In the general FPD, one pixel includes R, G, and B sub-pixels. It is assumed that one display element corresponds to one sub-pixel. In the example shown in
For the column direction, in the example shown in
The apertures 116 of the aperture control unit 115 are provided so as to correspond to the position of the element image. In the example shown in
In the above-mentioned structure, for example, a plurality of images acquired from the same object with a plurality of different parallaxes is supplied from an image generating apparatus, such as a camera, or a decoder of an image reproducing apparatus, to the input unit 105. The plurality of images is interleaved with the pixels at the corresponding positions and is supplied as one image data item. The embodiment is not limited thereto, but a plurality of image data may be individually supplied. The display image generating unit 103 distributes the sub-pixels that have different parallaxes and are disposed at positions corresponding to the supplied plurality of images in the order corresponding to the parallaxes. The display image generating unit 103 forms an element image using the distribution and supplies the element image to the display element array 114 of the display unit 104.
The aperture 116 of the aperture control unit 115 is an exit pupil that is provided so as to correspond to the element image 141 (hereinafter, in some cases, the aperture 116 is referred to as an exit pupil). Therefore, the beam emitted from the pixel of the parallax corresponding to the direction from the viewpoint of the observer in the element image 141 selectively reaches the viewpoint of the observer. When the beams emitted from the pixels of different parallaxes reach both eyes of the observer, the observer can observe a three-dimensional image.
The display element array 114 that is provided on the rear side of the lenticular sheet as viewed from the observer displays a parallax image group that is slightly differently viewed depending on the angle, i.e., a multi-view image, using the apertures 116 and the element images having a plurality of pixels arranged therein. The emission direction of the beams of the multi-view image is determined by passing through any one of the apertures 116 of the aperture control unit 115. In this way, a three-dimensional image is reproduced.
In the example shown in
When this embodiment is applied to the glasses-type three-dimensional image display apparatus, as shown in
Next, the relationship between the observation position of the observer relative to the display unit 104 and the display of a three-dimensional image will be described.
VW=(pp×N)×L/g (1)
When the visual distance L from the display surface of the display unit 104 to the observation position is constant, the visual range width VW is constant even when N pixels allocated to the exit pupil are changed. In other words, the visual range angle φ shown in
In this embodiment, the lenticular sheet is used as the aperture control unit 115 serving as a beam controller in the display unit 104. In this case, as shown in
As described above, as the observation angle θ with respect to the display surface of the display unit 104 increases, the appearance width or height of the display surface is reduced (see
Therefore, in this embodiment, when the observation angle θ with respect to the display surface of the display unit 104 as the observation position of the observer increases, it is determined that a restriction in representing the depth or a reduction in image quality occurs, and the representation of the depth of a three-dimensional image is controlled to maintain the display quality of a three-dimensional image. That is, in order to reduce the depth as the observation angle θ detected by the observation position detecting unit 101 increases, the parallax amount determining unit 102 determines the amount of parallax to be continuously reduced as the observation angle θ increases such that the display image generating unit 103 can use an image with a small amount of parallax as image information to generate an element image array, as shown in
Specifically, the parallax amount determining unit 102 determines the interval (the gap between viewpoints), at which a multi-view image is acquired, to be continuously reduced as the observation angle θ increases, thereby reducing the amount of parallax. In this way, it is possible to reduce the depth as the observation angle θ increases.
In the example shown in
As a method of determining the value of the acquisition interval of the multi-view image which is reduced as the observation angle θ increases using the parallax amount determining unit 102, the following method is used.
When the number of parallaxes is N, an element image pitch is P, the width of the aperture 116, that is, a lens pitch is Ps, a sub-pixel pitch [mm] in the horizontal direction is pp, the distance [mm] between the aperture control unit 115 (lenticular sheet) and the pixel is g, the distance [mm] from the display surface of the display unit 104 is L, a visual range optimization distance is L0, an observation distance is Lv, and the gap [mm] between beams at Pl is Pl (L) [mm], pp, Pl(L), g, L, P, Pe, and L0 are calculated by the following Expressions (2) and (3):
pp:Pl(L)=g:L (2)
P:Ps=(L0+g):L0 (3)
At pp=0.05, when Pl (400)=62 (interocular distance) is established at Lv=400, g is set as represented by the following Expression (4):
0.05:62=g:400,g=0.32 (4)
In order to maximize the visual range width (VW (500)) at L0=500, P and Ps are set so as to satisfy the following Expression (5):
P:Ps=1:(500+0.32)/500=1:1.00064 (5)
In this case, the visual range width VW (L0) and the observation angle θ are calculated by the following Expressions (6) and (7):
VW(L0)=P×L0/g (6)
θ={a tan(VW(L0)right end/L0)−a tan(VW(L0)left end/L0)}/2 (7)
where (VW (L0) right end) and (VW (L0) left end) are the coordinates of both ends of the visual range width VW (L0).
As can be known from the above-mentioned expressions, as the observation angle θ increases, the visual range width VW is reduced. When the interocular distance is 62, the visual range angle φ required for observation is calculated by the following Expression (8):
φ=a tan(62/Lv)/2 (8)
As can be seen from Expression (8), the visual range angle φ depends on the interocular distance and the observation distance. From the above-mentioned expressions, for example, in order to reduce the depth in operative association with the value of (θ−φ), the parallax amount determining unit 102 may determine the acquisition interval of the multi-view image to be a small value, or it may be configured such that a three-dimensional image is obtained when θ>φ and a planar image is obtained when θ≦φ.
Next, a three-dimensional image display process of this embodiment having the above-mentioned structure will be described with reference to a flowchart shown in
First, the input unit 105 receives a three-dimensional image signal from an image generating apparatus, such as a camera, or receives the three-dimensional image signal through a decoder of an image reproducing apparatus (Step S11). The observation position detecting unit 101 detects the observation angle θ of the observer with respect to the front surface of the display unit 104 using, for example, the acceleration sensor (Step S12).
Then, the parallax amount determining unit 102 determines the acquisition interval (the gap between viewpoints) at which the multi-view image is acquired on the basis of the observation angle θ detected in Step S12 (Step S13). Specifically, the parallax amount determining unit 102 determines the gap between viewpoints to be reduced as the observation angle θ increases according to the graph shown in
The display image generating unit 103 arranges the parallax images in a parallax image arrangement table at the gap between viewpoints determined in Step S13 to generate an element image array (Step S17) and displays the element image array on the display unit 104 (Step S18).
The parallax image arrangement table is an arrangement table indicating the arrangement of the parallax images in each element image of the multi-view image displayed on the display surface of the display unit 104, that is, the element image array, and is described in detail in Japanese Patent No. 3944188.
As described above, in this embodiment, as the observation angle θ increases, the amount of parallax is continuously reduced. In this way, it is possible to correspond to a variation in the width of appearance, the visual range angle φ, the visual range width VW, and the amount of crosstalk which are continuously changed as shown in
In this embodiment, the three-dimensional image display apparatus receives a three-dimensional image to be displayed from an external image generating apparatus or an image reproducing apparatus and displays the three-dimensional image on the display unit 104 in real time, but the invention is not limited thereto. This embodiment can be applied to a three-dimensional image display apparatus that reads a three-dimensional image stored in a storage medium, such as a hard disk drive (HDD) or a volatile or non-volatile memory, and displays the three-dimensional image on the display unit 104.
In the first embodiment, as the observation angle θ increases, the amount of parallax is continuously reduced. In a second embodiment, when the observation angle θ increases to be more than a threshold value, a display image is rapidly changed to a planar image. Furthermore, in the first embodiment, the three-dimensional image signal is received from, for example, an external image generating apparatus or an image reproducing apparatus and is then displayed on the display unit 104. In the second embodiment, a multi-view image or a single-view image that is stored in an image storage unit 1705 in advance is displayed on the display unit 104.
However, this embodiment may be applied to the three-dimensional image display apparatus that receives a three-dimensional image signal from, for example, an external image generating apparatus or an image reproducing apparatus and displays the three-dimensional image signal on the display unit 104.
As shown in
The image storage unit 1705 is a storage medium, such as an HDD or a memory that stores therein three-dimensional image signals of a multi-view image or a single-view image in advance.
The parallax amount determining unit 1702 determines whether the observation angle θ detected by the observation position detecting unit 101 is more than a predetermined threshold value. When it is determined that the observation angle θ is more than the threshold value, the parallax amount determining unit 1702 determines the image to have a small amount of parallax. Specifically, as shown in
When the threshold value is set in this way, the same three-dimensional image as that on the front side is maintained before both eyes are beyond the visual range or before the amount of crosstalk is more than an allowable value.
In order to prevent the occurrence of an abnormal image when the observation angle θ is more than the threshold value and the image is changed to a planar image, the planar image to be changed may or may not be related to the three-dimensional image display to be displayed. For example, a planar image or an achromatic image, such as a black display (non-display) image that is not related to the three-dimensional image to be displayed, may be selected.
Returning to
Next, a three-dimensional image display process of this embodiment having the above-mentioned structure will be described with reference to a flowchart shown in
First, the observation position detecting unit 101 detects the observation angle θ of the observer with respect to the front surface of the display unit 104 using, for example, an acceleration sensor (Step S22).
Then, the parallax amount determining unit 1702 determines whether the observation angle θ detected in Step S22 is more than the threshold value (Step S23). When the observation angle θ is equal to or less than the threshold value (No in Step S23), the parallax amount determining unit 1702 does not change the amount of parallax. When the observation angle θ is more than the threshold value (Yes in Step S23), the parallax amount determining unit 1702 determines to select a planar image (Step S24).
Then, the display image generating unit 1703 acquires the planar image determined in Step S24 from the image storage unit 1705 and arranges it in the parallax image arrangement table to generate an element image array (Step S27). Then, the display image generating unit 1703 displays the element image array on the display unit 104 (Step S28).
As described above, in this embodiment, when the observation angle θ increases to be more than the threshold value, a display image is rapidly changed to a planar image and the planar image is displayed on the display unit 104. Therefore, when the observation position greatly deviates from the front surface, the depth is reduced without hindering motion parallax from being ideally given. Therefore, it is possible to prevent the occurrence of an abnormal image and thus display a high-quality three-dimensional image. Furthermore, it is possible to separately provide motion parallax even in the planar image only when parallax is removed. Therefore, it is possible to continuously provide a small three-dimensional effect.
The first embodiment and the second embodiment may be combined to display a three-dimensional image. In a first modification, as shown in
A three-dimensional image display process of the first modification will be described with reference to a flowchart shown in
Then, the parallax amount determining unit 102 determines whether the observation angle θ detected in Step S42 is more than the threshold value (Step S43). When the observation angle θ is equal to or less than the threshold value (No in Step S43), the parallax amount determining unit 102 does not change the amount of parallax. When the observation angle θ is more than the threshold value (Yes in Step S43), the parallax amount determining unit 102 determines the interval (the gap between viewpoints) at which the multi-view image is acquired on the basis of the observation angle θ detected in Step S42 (Step S44). Specifically, the parallax amount determining unit 102 determines the gap between viewpoints to be reduced as the observation angle θ increases according to the graph shown in
Then, the display image generating unit 103 arranges parallax images in the parallax image arrangement table at the gap between viewpoints determined in Step S44 to generate an element image array (Step S47), and displays the element image array on the display unit 104 (Step S48).
In a second modification, as shown in
A three-dimensional image display process of the second modification will be described with reference to a flowchart shown in
Then, the parallax amount determining unit 102 determines whether the observation angle θ detected in Step S32 is more than the threshold value (Step S33). When the observation angle θ is equal to or less than the threshold value (No in Step S33), the parallax amount determining unit 102 determines the interval (the gap between viewpoints) at which the multi-view image is acquired on the basis of the observation angle θ detected in Step S32 (Step S35). Specifically, the parallax amount determining unit 102 determines the gap between viewpoints to be reduced as the observation angle θ increases, according to the graph shown in
Then, the display image generating unit 103 arranges parallax images in the parallax image arrangement table at the gap between viewpoints determined in Step S34 to generate an element image array (Step S37), and displays the element image array on the display unit 104 (Step S38).
In the first and second modifications, when the observation position greatly deviates from the front surface, the depth is reduced without hindering motion parallax from being ideally given. Therefore, it is possible to prevent the occurrence of an abnormal image and thus display a high-quality three-dimensional image.
In the above-described embodiments and modifications, when parallax information is given in both the horizontal direction and the vertical direction, it is necessary to apply the three-dimensional image display process to both the horizontal direction and the vertical direction. In this case, a method of reducing depth according to the observation angle θ may be independently performed in the horizontal direction and the vertical direction. In a three-dimensional image display apparatus that provides the parallax information only in the horizontal direction, general tracking may be combined with the vertical direction. In addition, the widening of the visual range by the optimization of N pixels and the switching of the multi-view image according to the embodiments may be performed at the same time.
In the above-described embodiments and modifications, the amount of parallax is reduced on the basis of the observation angle θ with respect to the front surface of the display unit 104 to reduce depth. However, any method may be used as long as it can change the amount of parallax to reduce the depth.
In the three-dimensional image display apparatuses 10 and 1700 according to the embodiments, the structures of the observation position detecting unit 101 and the parallax amount determining unit 102 may be changed in various ways.
For example, in the three-dimensional image display apparatuses 10 and 1700 according to the embodiments, a table indicating the correspondence between the observation angle θ and the observation distance of the observer from the front surface of the display unit 104 shown in
The three-dimensional image display apparatuses 10 and 1700 according to the embodiments may be provided with an imaging unit such as a camera. In this case, the observation position detecting unit 101 may calculate the size of the head of the observer from the image captured by the imaging unit and calculate the observation distance of the observer from the front surface of the display unit 104 from the calculated size of the head. In addition, the parallax amount determining unit 102 may determine the amount of parallax on the basis of the observation angle θ and the observation distance. In this case, the observation distance is considered in addition to the observation angle θ. Therefore, it is possible to accurately calculate the amount of parallax and display a high-quality three-dimensional image.
A three-dimensional image display program executed by the three-dimensional image display apparatus according to the above-described embodiments is stored in advance into, for example, a ROM and then provided.
The three-dimensional image display program executed by the three-dimensional image display apparatus according to the above-described embodiments may be stored as a file of an installable format or an executable format in a computer-readable storage medium, such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk), and then provided.
The three-dimensional image display program executed by the three-dimensional image display apparatus according to the above-described embodiments may be stored in a computer that is connected to a network, such as the Internet, downloaded from the computer through the network, and then provided. In addition, the three-dimensional image display program executed by the three-dimensional image display apparatus according to the above-described embodiments may be provided or distributed through a network such as the Internet.
The three-dimensional image display program executed by the three-dimensional image display apparatus according to the above-described embodiments may have a module structure including the above-mentioned units (the input unit, the observation position detecting unit, the parallax amount determining unit, and the display image generating unit). As the actual hardware, a CPU (processor) reads the three-dimensional image display program from the ROM and executes the three-dimensional image display program. Then, the above-mentioned units are loaded on the main storage device, and the input unit, the observation position detecting unit, the parallax amount determining unit, and the display image generating unit are generated in the main storage device.
While certain embodiments have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the inventions. Indeed, the novel embodiments described herein may be embodied in a variety of other forms; furthermore, various omissions, substitutions and changes in the form of the embodiments described herein may be made without departing from the spirit of the inventions. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the inventions.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2010-204762 | Sep 2010 | JP | national |
