Patent Application
20180124373
This application claims priority from European Patent Application No. 16306433.0 entitled “Method for Providing Continuous Motion Parallax Effect Using an Auto-Stereoscopic Display, Corresponding Device, Computer Program Product and Computer-Readable Carrier Medium”, filed on Oct. 31, 2016, the contents of which are hereby incorporated by reference in its entirety.
The present disclosure lies in the field of auto-stereoscopic displays. More precisely, the disclosure pertains to a technique for obtaining multi-view images to be displayed on an auto-stereoscopic display. The proposed technique allows providing a continuous motion parallax effect, and thus improves the immersive experience of an observer in front of the auto-stereoscopic display.
The present section is intended to introduce the reader to various aspects of art, which may be related to various aspects of the present disclosure that are described and/or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present disclosure. Accordingly, it should be understood that these statements are to be read in this light, and not as admissions of prior art.
One of the key elements to get a full immersive experience when using any display system is to be able to render what is called the motion parallax. Motion parallax offers depth cues by comparing the relative motion of different elements in a 3D scene: when an observer's head moves, closer 3D objects appear to move faster than those far away from the observer.
Some systems have been proposed to render motion parallax on standard 2D displays or on standard stereoscopic (two-views) displays. For example, some solutions exist to follow the location of an observer in front of the display (e.g. using eye-tracking techniques). It is then possible to render the motion parallax by slightly modifying the content displayed according to the observer's eyes position. However, one main problem of these systems lies in the time it takes to display the adapted content: the location tracking system takes time to evaluate the location of the observer, the adapted content must then be generated accordingly, and finally the display takes time to refresh the content on the screen. This lag between the time the observer is moving his head and the time the adapted content is available on the display is usually too high to be unnoticeable to the observer. As a result, for example, the observer can have the wrong feeling that an object displayed is moving the opposite way than the one expected, because the brain is anticipating a movement which does not occur quickly enough. The user experience is thus not optimal, and sometime deceptive and discomfortable.
Other types of display allow a better experience in term of motion parallax. This is for example the case with auto-stereoscopic displays. As schematically illustrated in relation with
An auto-stereoscopic display is usually capable of displaying more than two views at the same time. On the example of
Besides, because of the optical properties of the lenticular array 12, the same eight views are visible from several different locations in front of the auto-stereoscopic display 10. For example,
However, one drawback of the previously described technique is that a continuous motion parallax effect is restricted to a very limited zone, i.e. a narrow viewing zone. This is due to the optical properties of the lenticular array 12 previously described. For example, if the observer O3—initially seeing stereoscopic pair of views (V1; V2) from viewing zone Z3—moves his head to his right, to a position where he sees stereoscopic pair of views (V7; V8) from viewing zone Z2, he won't see a continuous and coherent motion parallax effect since pairs of views (V1; V2), (V8; V1) and (V7; V8) do not form coherent successive stereoscopic pairs of views. In the transition zone where the observer O3 sees the pair of views (V8; V1), as represented in
It would hence be desirable to provide a technique for providing motion parallax effect that would avoid at least one drawback of the prior art.
According to an aspect of the present disclosure, a method for obtaining a multi-view image to be displayed on an auto-stereoscopic display is provided. The auto-stereoscopic display is configured to display multi-view images comprising n views forming n−1 successive stereoscopic pairs of views. The proposed method for obtaining a multi-view image to be displayed comprises:
In that way, the content displayed on the auto-stereoscopic display can be automatically adapted depending on the location of the observer. In particular, the multi-view image to be displayed on the auto-stereoscopic display is obtained from n views determined so that they form a set of n−1 successive stereoscopic pairs of views centred on the observer location. In this manner, by iterating the proposed method, the content displayed on the auto-stereoscopic display may thus be constantly adapted so that the observer is always substantially located at the centre of a zone in which motion parallax effect is respected.
According to an embodiment, obtaining said multi-view image to be displayed takes into account said location of the observer in front of said auto-stereoscopic display.
In that way, the multi-view image to be displayed may be generated so that the current stereoscopic pair of views is still associated with the predetermined location of the observer in front of said auto-stereoscopic display. In that way, if the observer hasn't moved again after having reached said predetermined location, he will not notice any undesirable artefact when the image displayed on the auto-stereoscopic display changes to the newly obtained multi-view image: he still sees the scene with the same point of view. Besides, the multi-view image to be displayed may also be generated so that the optical properties of the auto-stereoscopic display are taken into account. More particularly, the fact that the same views are repeated at predetermined intervals due to optical properties of the lenticular array of the auto-stereoscopic display is taken into account: the multi-view image is generated according to the position of the observer in a way that makes adjacent views always coherent spatially and temporally. Thus, for any limited movement of the head, even fast, the observer is sure to see only adjacent and coherent content.
According to an embodiment, n is an even number, and the n views being determined responsive to said current stereoscopic pair of views defines a set of n−1 successive stereoscopic pairs of views in which the current stereoscopic pair of views occupies the position of rank n/2.
In that way, the current stereoscopic pair of views occupies the exact central position within the set of n−1 successive stereoscopic pairs of views. A continuous motion parallax effect is thus provided, whether the observer moves his head to his right or to his left.
According to another embodiment, n is an odd number, and the n views being determined responsive to said current stereoscopic pair of views defines a set of n−1 successive stereoscopic pairs of views in which the current stereoscopic pair of views occupies the position of rank (n−1)/2 or (n+1)/2.
In that way, the current stereoscopic pair of views occupies a substantially central position within the set of n−1 successive stereoscopic pairs of views. A continuous motion parallax effect is thus provided, whether the observer moves his head to his right or to his left, even if there is no exact central position within the set of n−1 successive stereoscopic pairs of views.
According to an embodiment, determining the n views responsive to said current stereoscopic pair of views further takes account of a direction of motion of said observer to reach said location of the observer in front of the auto-stereoscopic display.
In that way, depending of said direction of motion, it is possible to estimate the direction in which the observer is likely to move afterwards, and thus determine the n views of the multi-view image accordingly. In particular, while still occupying a substantially central position within the set of n−1 successive stereoscopic pairs of views, the current stereoscopic pair of views may be slightly offset within said set of n−1 successive stereoscopic pairs of views in order to give more latitude, in term of coherent motion parallax effect, in the direction of a presumed future motion of the observer.
According to an embodiment, determining said n views responsive to said current stereoscopic pair of views comprises interpolating at least one of said n views.
In that way, some views that were not acquired by the image acquisition system used to capture a scene may be interpolated within the device for implementing the proposed technique itself, for example within the auto-stereoscopic display itself. Others views already available are used to perform such an interpolation. In that way, it is possible to save some bandwidth on the link between a content provider and a receiving device, since there is no need to transmit all the views that may be useful to generate a new multi-view image: some of them may be interpolated at the auto-stereoscopic display level.
According to an embodiment, obtaining the current stereoscopic pair of views takes into account the position of the eyes of said observer.
The present disclosure also concerns a device for obtaining a multi-view image to be displayed on an auto-stereoscopic display. The auto-stereoscopic display is configured to display multi-view images comprising n views forming n−1 successive stereoscopic pairs of views. Such a device comprises:
According to one implementation, the different steps of the method for obtaining a multi-view image to be displayed on an auto-stereoscopic display as described here above are implemented by one or more software programs or software module programs comprising software instructions intended for execution by a data processor of an apparatus for obtaining a multi-view image to be displayed on an auto-stereoscopic display, these software instructions being designed to command the execution of the different steps of the methods according to the present principles.
A computer program is also disclosed that is capable of being executed by a computer or by a data processor, this program comprising instructions to command the execution of the steps of a method for obtaining a multi-view image to be displayed on an auto-stereoscopic display as mentioned here above.
This program can use any programming language whatsoever and be in the form of source code, object code or intermediate code between source code and object code, such as in a partially compiled form or any other desirable form whatsoever.
The information carrier can be any entity or apparatus whatsoever capable of storing the program. For example, the carrier can comprise a storage means such as a ROM, for example a CD ROM or a microelectronic circuit ROM or a magnetic recording means, for example a floppy disk or a hard disk drive.
Again, the information carrier can be a transmissible carrier such as an electrical or optical signal which can be conveyed via an electrical or optical cable, by radio or by other means. The program according to the present principles can be especially uploaded to an Internet type network.
As an alternative, the information carrier can be an integrated circuit into which the program is incorporated, the circuit being adapted to executing or to being used in the execution of the methods in question.
According to one embodiment, the methods/apparatus may be implemented by means of software and/or hardware components. In this respect, the term “module” or “unit” can correspond in this document equally well to a software component and to a hardware component or to a set of hardware and software components.
A software component corresponds to one or more computer programs, one or more sub-programs of a program or more generally to any element of a program or a piece of software capable of implementing a function or a set of functions as described here below for the module concerned. Such a software component is executed by a data processor of a physical entity (terminal, server, etc.) and is capable of accessing hardware resources of this physical entity (memories, recording media, communications buses, input/output electronic boards, user interfaces, etc.).
In the same way, a hardware component corresponds to any element of a hardware unit capable of implementing a function or a set of functions as described here below for the module concerned. It can be a programmable hardware component or a component with an integrated processor for the execution of software, for example an integrated circuit, a smartcard, a memory card, an electronic board for the execution of firmware, etc.
A non-transitory processor readable medium having stored thereon such a program is also disclosed.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the disclosure, as claimed.
It must also be understood that references in the specification to “one embodiment” or “an embodiment”, indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.
Embodiments of the present disclosure can be better understood with reference to the following description and drawings, given by way of example and not limiting the scope of protection, and in which:
a,
3
b and 3c illustrate how a multi-view image is generated, according to different embodiments of the present disclosure;
The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the disclosure.
The general principle of the present disclosure relies on a specific technique for obtaining multi-view images to be displayed on an auto-stereoscopic display.
As it will be described more fully hereafter with reference to the accompanying figures, it is proposed in one aspect of the present disclosure to take account of a location of an observer in front of an auto-stereoscopic display to adapt the content displayed, so that a continuous motion parallax effect that allows a more optimal immersive experience can be provided.
This disclosure may, however, be embodied in many alternate forms and should not be construed as limited to the embodiments set forth herein. Accordingly, while the disclosure is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that there is no intent to limit the disclosure to the particular forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the disclosure as defined by the claims. Like numbers refer to like elements throughout the description of the figures.
While not explicitly described, the present embodiments and variants may be employed in any combination or sub-combination.
Referring first to
At step 21, the current stereoscopic pair of views of I1 associated with the current location of an observer O is obtained. This current stereoscopic pair of views corresponds to the stereoscopic pair of views of I1 the observer actually sees, from his location in front of the auto-stereoscopic display. It can be easily determined, since characteristics of the auto-stereoscopic display are well known. For example, in the situation illustrated in
At step 22, the current stereoscopic pair of views is used to obtain a new multi-view image to be displayed on the auto-stereoscopic display. In one embodiment of the present disclosure, step 22 for obtaining a new multi-view image to be displayed on the auto-stereoscopic display comprises determining (221) n views responsive to the current stereoscopic pair of views, and generating (222) the multi-view image to be displayed from said n views previously determined. In other words, the n views are determined as a function of the current stereoscopic pair of views: these n views comprise the two views of the current stereoscopic pair of views, and the n−2 remaining views are selected so that they form, with the current stereoscopic pair of views, n−1 successive stereoscopic pairs of views in which the current stereoscopic pair of views occupies a substantially central position. According to an embodiment, if n is an even number, the n−2 remaining views are selected so that the current stereoscopic pair of views occupies the exact central position in the n−1 successive stereoscopic pairs of views, which is position with rank n/2.
According to another embodiment of the proposed disclosure, at step 22, a direction of motion of the observer in front of the auto-stereoscopic display is also taken into account to determine the n views that will be used to generate a new multi-view image to be displayed on the auto-stereoscopic display. Such information regarding motion direction is useful, since it helps to estimate the direction (left or right) in which the observer is likely to move afterwards. For example, referring back to
As it can be noticed on the examples of
In the embodiment illustrated in
By iterating the proposed method, the content displayed on the auto-stereoscopic display may thus be constantly adapted so that the observer is always substantially located at the centre of a zone in which motion parallax effect is respected. In other words, the multi-view image displayed on the auto-stereoscopic display is generated so that the observer is maintained, as far as possible, at a certain distance from transition zones, namely zones where the continuity of the motion parallax effect is broken, such as the one corresponding to the location of observer O3 in
According to the present disclosure, the observer can thus benefit from a continuous motion parallax effect all along the motion of his head. The implementation of the proposed technique has many advantages compared to prior art solutions: it offers time to refresh a multi-view image, allowing rendering a continuous horizontal motion parallax effect which is not limited to a narrow viewing zone, with unperceivable latency for the observer. The immersive experience is thus improved.
An apparatus 500 illustrated in
The processor 501 controls operations of the apparatus 500. The storage unit 502 stores at least one program to be executed by the processor 501, and various data, including for example parameters used by computations performed by the processor 501, intermediate data of computations performed by the processor 501, and so on. The processor 501 is formed by any known and suitable hardware, or software, or a combination of hardware and software. For example, the processor 501 is formed by dedicated hardware such as a processing circuit, or by a programmable processing unit such as a CPU (Central Processing Unit) that executes a program stored in a memory thereof.
The storage unit 502 is formed by any suitable storage or means capable of storing the program, data, or the like in a computer-readable manner. Examples of the storage unit 502 include non-transitory computer-readable storage media such as semiconductor memory devices, and magnetic, optical, or magneto-optical recording media loaded into a read and write unit. The program causes the processor 501 to perform a process for obtaining a multi-view image to be displayed on an auto-stereoscopic display according to an embodiment of the present disclosure as described previously. More particularly, the program causes the processor 501 to generate a multi-view image to be displayed on an auto-stereoscopic display. The views used to generate such multi-view image may be stored into storage unit 502.
The input device 503 is formed for example by a device for determining the location of an observer in front of an auto-stereoscopic display. For example, input device 503 is an eye-tracking device.
The output device 504 is formed for example by an auto-stereoscopic display to display the multi-view image generated by applying the method for obtaining a multi-view image previously described.
The interface unit 505 provides interfaces between the apparatus 500 and external apparatus. The interface unit 505 may be communicable with external apparatus via cable or wireless communication. For example, in an embodiment where the device for obtaining a multi-view image is not embedded in an auto-stereoscopic display, an external apparatus may be such an auto-stereoscopic display. The device for determining the location of an observer may also be an external apparatus, if such a device is not embedded in the device for obtaining a multi-view image according to the proposed technique.
Although only one processor 501 is shown on
These modules and units may also be embodied in several processors 501 communicating and co-operating with each other.
| Number | Date | Country | Kind |
|---|---|---|---|
| 16306433.0 | Oct 2016 | EP | regional |
