The invention relates to a 3D image display method and device.
The invention also relates to an image display data conversion method wherein 3D image data are converted into novel 3D image data.
The invention further relates to a 3D image signal.
The invention further relates to a computer program product to be loaded by a computer arrangement, comprising instructions to generate output 3D image data on basis of a 3D image data input, the computer arrangement comprising processing means.
A three-dimensional representation of graphics and video is one of the holy grails in the display field. Several methods have been devised which give the viewer the impression that he is looking at a 3-dimensional image. A 3D perception can be created in several manners. A three dimensional impression can be created by using stereo pairs (two different images directed at the two eyes of the viewer). There are several ways to produce stereo images. The images may be time multiplexed on a 2D display, but this requires that the viewers wear glasses with e.g. LCD shutters. When the stereo images are displayed at the same time, the images can be directed to the appropriate eye by using a head mounted display, or by using polarized glasses (the images are then produced with orthogonally polarized light). The glasses worn by the observer effectively route the views to each eye. Shutters in the glasses are synchronized to the frame rate to control the routing. A disadvantage with such as system is that the two images do not produce a “look around” capability, unless the position of the head is tracked and the views are adapted accordingly. Furthermore, glasses have to be worn to produce any effect. This is unpleasant for those observers who are not familiar with wearing glasses and a potential problem for those already wearing glasses, since the extra pair of glasses do not always fit.
Instead of near the viewers eyes, the two stereo images can also be split at the display screen by means of splitting screen such as a lenticular screen or a parallax barrier. E.g. in FIGS. 3 and 4 of U.S. Pat. No. 6,275,254 the principle is shown.
Although to view a 3D image significantly improves the sense of reality of the displayed images it has been reported that after some times viewers experience viewing fatigue and even headaches. This strongly reduces the attractiveness of 3D image devices and 3D imaging in general.
It is an object of the invention to improve 3D imaging methods and 3D game display devices by reducing the mentioned problem.
To this end the method in accordance the invention is characterized in the depth of content of the displayed image is temporarily set at a first depth content level, where after the depth of content is set at a second depth content level, lower than the first depth content level.
For displaying the images the pixel data are provided with data relating to the x and y position, i.e. the position in a plane parallel to the display screen, and depth data relating to a z-position, i.e. data indicating the position of a pixel or object perpendicular to the display plane, i.e. the depth.
The depth data is translated into a difference in image seen by the right and left eye, i.e. the parallax angle which gives the depth of content of the image, i.e. just how “deep” the 3D effect is perceived by the viewer. The image can be displayed such that the difference between the left and right eye images is only slight, in which case the depth of content of the displayed image is only small, i.e. there is only a small 3D optical effect, or the difference could be made larger, in which case there is large depth of content, i.e. a large 3D effect. The depth of content of the image or of any part of the image is given by the relation between the depth data, i.e. the z-values of pixels or object of the image, and the differences between of the images seen by the left and right eye, i.e. the parallax angle. In conventional devices this relation is a constant, i.e. the depth of content is stationary, it is fixed and permanent.
Present 3D display devices have the intrinsic property that only one plane is in focus for the viewer. This means that the viewer receives conflicting depth cues: the image disparity (difference in image for left and right eye) and eye accommodation cue never actually match. When the depth reproduction is relatively small (i.e. there is only small 3D effect) this does not present a problem, but for larger depth effects, viewing fatigue and even headaches result. What is not in focus is blurred and this causes fatigue to the eyes. This seems to present an insurmountable dilemma. One the one hand the stronger the 3D effect, the more attractive the 3D display device becomes in respect of conventional 2D displays, but on the other hand a strong 3D effect leads to viewing fatigue and possibly headaches, which effect could be reduced by reducing the 3D effect, i.e. the perceived depth in the image, but this would reduce the advantage of the 3D display device over a conventional 2D display device.
The invention reduces this dilemma. The depth of content of the displayed image or of a part of the image is temporarily set at a first depth content level, where after the depth of content is set at a second depth content level, lower than the first depth content level.
This allows the depth content of the image to be, on average, at a relatively low level. On average therefore the depth content is low, and a relatively small parallax angle is used, which reduce fatigue and headaches. Temporarily, in the meaning of lasting or intended to be used for a relatively short time only, increasing the depth content to a high level, increases the depth perception of the viewer. If this increased depth perception would only be perceived by the viewer during application of the increased depth the effect would be low. However, the increased depth perception remains at the relatively high level for some time after the actual depth content of the image has been reduced to a low level. In a sense, the eyes and brain of a viewer are tricked. Presenting an image with a high depth content, i.e. a large 3D effect, in a sense locks the perception of the viewer into believing he/she sees a picture with a high 3D effect. Even after reduction of the depth content, he/she still perceives the image as having a high depth content, even though in reality he/she is seeing an image with only a small 3D effect. Thus the effect of temporarily showing an image or a part of an image at a high depth content lingers on to well after application. In fact, experiments have shown that viewers do not see any or hardly any difference between displaying an image which has 100% of the time a large depth content, i.e. show a clear 3D effect, and images which intermittently have a large depth content, for instance x1 second large depth content followed by x2 seconds low or no depth content, wherein x1 and x2 are in the order of 0.1 to 10 seconds, where preferably x2 is equal to or larger than x1. Apparently even though at as substantial percentage of the time a “flat image” is shown, such images are not perceived to be different by the viewer from real 3D images in so far as 3D effect are concerned. In fact, intermittently high-low depth content provides for an overall somewhat sharper image, which is perceived favorably by the viewer.
Temporarily setting the depth content at a first, high level, is in embodiments done independent of the displayed image. In embodiments the depth of content is cyclically varied with a period in the order of 0.1 to 10 seconds, for instance x second large depth content followed by y seconds low or no depth content, wherein x and y are in the order of 0.1 to 10 seconds, where preferably y is equal to or larger than x. This is a simple scheme.
The invention removes viewer fatigue by reducing the overall “3D load with conflicting 3D cues” on the viewer.
In an embodiment of the invention, the depth content is set at the first level in dependence of the displayed image, upon introduction of a cue comprised in the image data. A cue may be the introduction of a novel element into the movie or image, or change in scene. The cue may be deduced from the content of the image, or may be a label in the bit stream. Comprised in includes any metadata associated with the image data as well as any image date per se. For instance, in a 3D video engine for computer games, debt information for all objects are known and novel object are labeled as such. The introduction of novel objects is then the cue which sets the depth of content of the image or the part of the image comprising the novel object to the first, relatively high level for a relatively short period of time, i.e. temporarily, where after the depth of content is set to the second, lower level. “Is set” is within the concept of the invention to be interpreted broadly, the transition between the first and second level may be, but need not be, instantaneously. There could be a fluent transition, wherein the change in the depth of content between the first and second level is gradual. In fact, this is preferred, since it gives better results. In these embodiments the enhanced 3D effect is presented when it is appreciated most, i.e. when new elements are introduced into the image. New elements could be a new scene, or new elements, such as a car driving into the scene. The 3D effect is by the viewer mentally associated with the new element or the change in scenery and will be perceived to persist, even if the actual 3D effect, i.e. the depth content itself, has decreased to a low level. This alternative embodiment is based on the same insight: Temporarily increasing the depth content of the displayed image ‘tricks’ the viewer into believing that, even when an image of lower 3D effect is subsequently shown, said image has a high depth content.
The image display device in accordance with the invention comprises means for performing the method steps in accordance with the invention.
The method of conversion comprises the method steps in accordance with the invention relating to conversion of input image data to output image data.
A converter comprises means for performing the methods steps corresponding with the method of conversion in accordance with the invention.
These and other objects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.
In the drawings:
The Figures are not drawn to scale. Generally, identical components are denoted by the same reference numerals in the Figures.
In a barrier auto-stereoscopic display system, a barrier 31 is disposed in front of a display array 32. The left and right images of a stereo pair of images are sliced into vertical strips. The strips 32L of the left image and the strips 32R of the right image are alternately disposed on array 32. Slots 31A are formed in barrier 31. Slots 31A are positioned so that the left eye 4L of an observer can see only strips 32L of the left image and the right eye 4R can see only strips 32R of the right image of the pair. The observer reconstructs the full image in three dimensions.
Referring now to
The amount of difference between the image presented to the left and right eye determines the perceived depth. If there is no difference between the image presented to the left and right eye the image is flat and there is no 3D effect. The amount of depth can be adjusted by changing the difference between the left and right image. Basically the viewer perceives different parts of the image on different planes, one behind the other.
An inherent property of 3D display devices is that only one plane is in focus for the viewer, almost always the plane that corresponds to the actual position of the display screen. This means that the viewer receives conflicting depth cues: the image disparity (difference in image for left and right eye) and eye accommodation cue (the eyes focus on the screen, while the eyes turn towards the observed object at a distance that is different from the distance to the screen) never actually match. When the depth reproduction is relatively small (i.e. there is only small 3D effect) this does not present a problem, since there is only a small depth difference and all parts of the image are, due to the small depth difference more or less in focus and the eyes can accommodate any difference. However, for larger depth effects, viewing fatigue and even headaches result. What is not in focus is blurred and this causes fatigue to the eyes. This seems to present an insurmountable dilemma. One the one hand the stronger the 3D effect, the more attractive the 3D display becomes in respect of conventional 2D displays, but on the other hand a strong 3D effect leads to viewing fatigue and possibly headaches, which effect could be reduced by reducing the 3D effect, i.e. the perceived depth in the image, but this would reduce the attractiveness of the 3D display.
In an alternative embodiment of the invention, the depth content is increased upon introduction of new elements into the image. In these embodiments the enhanced 3D effect is presented when it is appreciated most, i.e. when new elements are introduced into the image. New elements could be a new scene, or new elements, such as a car driving into the scene. The 3D effect is by the viewer mentally associated with the new element and will be perceived to persist, even if the actual 3D effect, i.e. the depth content itself, has decreased to a low level. This alternative embodiment is based on the same insight: Temporarily increasing the depth content of the displayed image ‘tricks’ the viewer into believing that, even when an image of lower 3D effect is subsequently shown, said image has a high depth content.
In preferred embodiments the depth content, during application of the enhanced depth effect, when the cue is the introduction of a novel element, is only increased for a part of the image near the newly introduced element. The attention of the viewer is then strongly drawn to the newly introduced element, and the association that is made between depth perception and the element is increased.
It is remarked that in JP 10/040,420 a scheme is described in which, when a novel scene is introduced, and thus possibly an object at a substantially different position in depth than before is sown to which change the eyes have to accommodate, a depth conversion is made to ease the transformation and moderate the change in depth perception over a number of frame times. This is however, although a temporary depth content conversion is used, different from the present invention. In JP 10-040420 temporarily, after an image scene change, a depth content less than the actual depth content is used, followed by increasing the depth content until the depth corresponds to the final value. Once the depth value has reached the final value no further changes are made. i.e. there is no diminution.
The invention may also be embodied in a converting device even if not comprised in a display device. Such a converter may convert an incoming data stream (for instance from a DVD) and convert it into an output data stream which is then e.g. sent to a storing device, or to a receiver. Basically, such a converting device has the same elements as the display device shown in
It is remarked that it is not necessary that the first level is the same for all instances. The idea of the invention is to temporarily increase the depth of content. Within the concept of the invention it is possible that, for instance, upon introduction of novel elements the depth of content is set high, but not equally high for all novel elements. Small, relatively unimportant, novel elements could be associated with a relatively mild increase in depth of content, whereas upon introduction of important new elements a relatively large increase in the depth of content is implemented.
Likewise, when a cyclic variation is used, the cyclic variation could comprise various intermingled cycles, wherein the variation in depth of content of the various cycles may differ.
In short the invention can be described as follows:
In a method and device for generating and/or displaying 3D images the depth content of an image or a part of an image is temporarily increased where after the depth of content is decreased. This temporarily increase of the depth of content renders a large 3D effect to the image or the part of the image to which it has been applied. It has been found that this enhancement effect of an increased 3D effect lingers on in the perception of the viewer even after application. The actual average depth of content can be made relatively low thus reducing fatigue, while the perceived depth of content is relatively high.
The invention is also embodied in any computer program product for a method or device in accordance with the invention. Under computer program product should be understood any physical realization of a collection of commands enabling a processor—generic or special purpose—, after a series of loading steps (which may include intermediate conversion steps, like translation to an intermediate language, and a final processor language) to get the commands into the processor, to execute any of the characteristic functions of an invention. In particular, the computer program product may be realized as data on a carrier such as e.g. a disk or tape, data present in a memory, data traveling over a network connection—wired or wireless—, or program code on paper. Apart from program code, characteristic data required for the program may also be embodied as a computer program product.
Some of the steps required for the working of the method may be already present in the functionality of the processor instead of described in the computer program product, such as data input and output steps.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims.
In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim.
The word “comprising” does not exclude the presence of other elements or steps than those listed in a claim. The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a device claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The invention may be implemented by any combination of features of various different preferred embodiments as described above. In particular it is mentioned that any embodiment shown or claimed in relation to an encoding method or encoder has, unless otherwise indicated or impossible, a corresponding embodiment for a decoding method or decoder and such decoding methods and decoder are embodiments of the invention and claimed herewith.
Number | Date | Country | Kind |
---|---|---|---|
05112381.8 | Dec 2005 | EP | regional |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/IB06/54675 | 12/8/2006 | WO | 00 | 6/16/2008 |