Patent Application
20090115800
The invention relates to a multi-view display device, comprising:
a structure of light-generating elements; and
a structure of optical means for directing light being generated by the structure of light-generating elements in a number of directions corresponding to respective views.
The invention further relates to a method of computing luminance values for respective light-generating elements of such a multi-view display device.
The invention further relates to an image processing apparatus for computing luminance values for respective light-generating elements of such a multi-view display device.
Since the introduction of display devices, a realistic 3-D display device has been a dream for many. Many principles that should lead to such a display device have been investigated. Some principles try to create a realistic 3-D object in a certain volume. For instance, in the display device as disclosed in the article “Solid-state Multi-planar Volumetric Display”, by A. Sullivan in proceedings of SID'03, 1531-1533, 2003, visual data is displaced at an array of planes by means of a fast projector. Each plane is a switchable diffuser. If the number of planes is sufficiently high the human brain integrates the picture and observes a realistic 3-D object. This principles allows a viewer to look around the object within some extent. In this display device all objects are (semi-)transparent.
Many others try to create a 3-D display device based on binocular disparity only. In these systems the left and right eye of the viewer perceives another image and consequently, the viewer perceives a 3-D image. An overview of these concepts can be found in the book “Stereo Computer Graphics and Other True 3-D Technologies”, by D. F. McAllister (Ed.), Princeton University Press, 1993. A first principle uses shutter glasses in combination with for instance a CRT. If the odd frame is displayed, light is blocked for the left eye and if the even frame is displayed light is blocked for the right eye.
Display devices that show 3-D without the need for additional appliances are called auto-stereoscopic display devices.
A first glasses-free display device comprises a barrier to create cones of light aimed at the left and right eye of the viewer. The cones correspond for instance to the odd and even sub-pixel columns. By addressing these columns with the appropriate information, the viewer obtains different images in his left and right eye if he is positioned at the correct spot, and is able to perceive a 3-D picture.
A second glasses-free display device comprises an array of lenses to image the light of odd and even sub-pixel columns to the viewer's left and right eye.
The disadvantage of the above mentioned glasses-free display devices is that the viewer has to remain at a fixed position. To guide the viewer, indicators have been proposed to show the viewer that he is at the right position. See for instance U.S. Pat. No. 5,986,804 where a barrier plate is combined with a red and green led. In case the viewer is well positioned he sees a green light, and a red light otherwise.
To relieve the viewer of sitting at a fixed position, multi-view auto-stereoscopic display devices have been proposed. See for instance US60064424 and US20000912. In the display devices as disclosed in US60064424 and US20000912 a slanted lenticular is used, whereby the width of the lenticular is larger than two sub-pixels. In this way there are several images next to each other and the viewer has some freedom to move to the left and right.
In WO 99/05559 a method for controlling pixel addressing of a display device to drive the display device as an multi-view auto-stereoscopic display when a lenticular screen is overlaid and image data for multiple views to be interlaced is provided. Based on data defining at least the lenticular screen lenticule pitch, and the global lenticular screen position relative to the display device, for each display color pixel, a derivation is made as to which of the N views it is to carry. The corresponding pixel data for the assigned view is then selected as the display pixel data. Although the image quality of a multi-view display device controlled on basis of the method as described in WO 99/05559 is relatively good, there is still room for improvement.
It is an object of the invention to provide a multi-view display device of the kind described in the opening paragraph with an improved image quality.
This object of the invention is achieved in that the multi-view display device further comprises computing means for computing luminance values for the respective light-generating elements on basis of an input signal, whereby for the computation of a particular luminance value for a particular light-generating element, the computing means is arranged to take into account a spatial displacement between a first location of the particular light-generating element and a second location of the corresponding one of the optical means.
Because of the spatial displacement between the particular light-generating element and the corresponding, i.e. cooperating or aligned to, optical means there is also a displacement observable in the generated light. Typically, the optical means introduce a spatial shift of the light in a plane parallel to the plane of the light-generating elements. For instance, suppose that a central point of a particular light-generating element is located at a particular location with coordinates (x,y). Suppose that the light being generated by that particular light-generating element originates from that particular location, or that the center of the light spot originates from that particular location with coordinates (x,y). The optical means, which is disposed at the second location in order to cooperate with the particular light-generating element, typically has a spatial displacement relative to the first location of the particular light-generating element, which differs from zero. That means that the center of the particular light-generating element and the center of the corresponding optical means are not aligned. Alternatively, one can say that the edge of the particular light-generating element and the edge of the corresponding optical means are not aligned, at the do not have mutually equal coordinates (x,y). The light being generated by the particular light-generating element, originating from the particular location (x,y) will be observable as if it was generated from another location (x+Δx,y+Δy). That means that the optical means introduce a spatial shift of the light beam in a plane parallel to the plane of the light-generating elements, whereby the spatial shift of the light Sx,y=(Δx,Δy) is related to the spatial displacement between the first location of the particular light-generating element and the second location of the corresponding one of the optical means.
The spatial shifts of the light beams are not mutually equal for all combinations of light-generating element and corresponding optical means. For achieving a relatively high image quality, the actual spatial shifts of the light beams, which are caused by the spatial displacement between the light-generating elements and the respective optical means, should be taken into account when computing the luminance values for the respective light-generating elements.
With particular luminance values is meant the particular value, e.g. driving voltage to be applied to the particular light-generating element in order to generate a particular amount of light.
In an embodiment of the multi-view display device according to the invention, the optical means are lenses. Typically, lenticular elements are provided by a lenticular sheet, whose lenticules, comprising (semi) cylindrical lens elements, extend in the column direction of the structure of light-generating elements with each lenticule overlying a respective group of two, or more, adjacent columns of light-generating elements and extending parallel with the light-generating element columns. Alternatively, the lenses are slanted relative to the structure of light-generating elements.
In an embodiment of the multi-view display device according to the invention, the structure of light-generating elements is based on any one of the set of display technologies comprising LCD, Plasma, CRT and organic LED. Typically, the structure of light-generating elements is a matrix display panel. The light may be actually created by the various light-generating elements. Alternatively, the matrix display panel is used as a spatial light modulator. Then the light is reflected or transferred by the light-generating elements.
In an embodiment of the multi-view display device according to the invention, the computing means is arranged to apply a shift to the sample position in the input signal, the shift being proportional to the spatial displacement. Computing the particular luminance value corresponds to extracting the appropriate value from the input signal. If the input signal is a continuous signal, extracting means sampling the input signal appropriately. The sampling for the particular light-generating element is not only based on the first location of the particular light-generating element in the structure of light-generating elements but also based on the spatial displacement between the first location of the particular light-generating element and the second location of the corresponding one of the optical means. That means that by selection/extraction of the particular luminance value the resulting spatial shift of the light beam is taken into account.
In an embodiment of the multi-view display device according to the invention, the computing means is arranged to apply a shift to the sample position in the input signal, for the computation of a set of luminance values for respective light-generating elements being located on a single row of the structure of light-generating elements and related to one of the views. As explained above, the spatial shift of the light beams is not a single constant value. It may be different for different combinations of light-generating elements and respective optical means. However it may be substantially mutually equal for a set of light-generating elements, e.g. for the light-generating elements being located on a single row of the structure of light-generating elements and related to one of the views. This embodiment of the multi-view display device according to the invention is arranged to apply shifts to the sample positions in the input signal for such a set.
In an embodiment of the multi-view display device according to the invention, the computing means is provided with a control signal representing shifts to be applied to the sample positions in the input signal, the shifts being proportional to respective spatial displacements between first locations of the light-generating elements and second locations of the respective corresponding optical means. The various shifts to be applied to the sample positions in the input signal may be constant in time. Alternatively, the various shifts also change as a function of time. This embodiment of the multi-view display device according to the invention is arranged to accept a control signal representing the actual shifts to be applied. The control signal is preferably based on a measurement of light being generated by the multi-view display device. To perform such a measurement the multi-view display device is provided with a predetermined input signal, e.g. representing a homogeneous image or representing an image with a predetermined pattern. By measuring the light being generated by the multi-view display device and comparing it with an expected output image it is possible to determine whether the appropriate shifts to the sample positions are applied or whether adjustments of the different shifts are needed, resulting in a modified control signal. Measuring the light being generated by the multi-view display device typically corresponds to acquiring an output image by means of an image acquisition apparatus like a camera.
It is a further object of the invention to provide a method of computing luminance values for respective light-generating elements of a multi-view display device of the kind described above resulting into an improved image quality.
This object of the invention is achieved that for the computation of a particular luminance value for a particular light-generating element, a spatial displacement between a first location of the particular light-generating element and a second location of the corresponding one of the optical means is taken into account.
It should be noted that it is known in the prior art to take into account the actual location of light-generating elements for a high quality rendering. See e.g. the article “subpixel image scaling for color matrix displays”, in Journal of the Society for Information Display, 11(1): 99-108, 2003, by M. Klompenhouwer and G. de Haan. However, taking into account a spatial shift of the light beams when rendering is novel. When computing the luminance values according to the method of the invention, phase compensation is applied during sampling, which is proportional to the spatial shift of the light beams in the optical domain of the multi-view display device.
It is a further object of the invention to provide an image processing apparatus for computing luminance values for respective light-generating elements of a multi-view display device of the kind described above with an improved image quality.
This object of the invention is achieved in that the image processing apparatus comprises:
input means for receiving an input signal; and
computing means for computing the luminance values for the respective light-generating elements on basis of the input signal, whereby for the computation of a particular luminance value for a particular light-generating element, the computing means is arranged to take into account a spatial displacement between a first location of the particular light-generating element and a second location of the corresponding one of the optical means.
Modifications of the multi-view display device and variations thereof may correspond to modifications and variations thereof of the method and the image processing apparatus being described.
These and other aspects of the multi-view display device, or the image processing apparatus and of the method, according to the invention will become apparent from and will be elucidated with respect to the implementations and embodiments described hereinafter and with reference to the accompanying drawings, wherein:
In the following example, a direct-view type of 3D-LCD lenticular array display device 100 having a slanted arrangement of lenticulars will be initially described with reference to
It will be understood that the Figures are merely schematic and are not drawn to scale. For clarity of illustration, certain dimensions may have been exaggerated while other dimensions may have been reduced. Also, where appropriate, the same reference numerals and letters are used throughout the Figures to indicate the same parts and dimensions.
Referring to
The light-generating elements 12 are substantially rectangular in shape and are regularly spaced from one another with the light-generating elements in two adjacent columns being separated by a gap extending in column (vertical) direction and with the display elements in two adjacent rows being separated by a gap extending in the row (horizontal) direction. The panel 10 is of the active matrix type in which each light-generating element is associated with a switching element, comprising for, example, a TFT or a thin film diode, TFD, situated adjacent the light-generating element.
The display panel 10 is illuminated by a light source 14, which, in this example, comprises a planar backlight extending over the area of the display element array. Light from the source 14 is directed through the panel with the individual light-generating elements being driven, by appropriate application of drive voltages, to modulate this light in conventional manner to produce a display output. The array of light-generating elements constituting the display produced thus corresponds with the structure of light-generating elements, each light-generating element providing a respective display pixel.
Over the output side of the panel 10, opposite that facing the light source 14, there is disposed a lenticular sheet 15 comprising an array of elongate, parallel, lenticules, or lens elements, acting as optical director means to provide separate images to a viewer's eyes, producing a stereoscopic display to a viewer facing the side of the sheet 15 remote from the panel 10. The lenticules of the sheet 15, which is of conventional form, comprise optically (semi) cylindrically converging lenticules, for example, formed as convex cylindrical lenses or graded refractive index cylindrical lenses. Autostereoscopic display device using such lenticular sheets in conjunction with matrix display panels are well known in the art although, unlike the conventional arrangement in such apparatuses, with lenticules extending parallel to the display pixel columns (corresponding to the display element columns), the lenticules in the apparatus of
In the purpose-built embodiment illustrated, the pitch of the lenticules is chosen in relation to the pitch of the display elements in the horizontal direction according to the number of views required, as will be described, and each lenticule, apart from those at the sides of the display element array, extends from top to bottom of the display element array.
The individually operable light-generating elements are driven by the application of display information in such a manner that a narrow slice of a 2D image is displayed by selected light-generating elements under a lenticule. The display produced by the panel comprises six interleaved 2D sub-images constituted by the outputs from respective light-generating elements. Each lenticule 16 provides six output beams from the underlying light-generating elements with view-numbers 1 to 6, respectively, whose optical axes are in mutually different directions and angularly spread around the longitudinal axis of the lenticule. With the appropriate 2D image information applied to the light-generating elements and with a viewer's eyes being at the appropriate distance to receive different ones of the output beams, then a 3D image is perceived. As the viewer's head moves in the horizontal (row) direction, then a number of stereoscopic images can be viewed in succession. Thus, a viewer's two eyes would see, respectively, for example, an image composed of all light-generating elements “1” and an image composed of all light-generating elements “2”. As the viewer's head moves, images comprised of all light-generating elements “3” and all light-generating elements “4” will be seen by respective eyes, then images comprised of all light-generating elements “3” and all light-generating elements “5”, and so on. At another viewing distance, closer to the panel, the viewer may, for example, see views “1” and “2” together with one eye and views “3” and “4” together with the other eye.
The plane of the light-generating elements 12 coincides with the focal plane of the lenticules 16, the lenticules being suitably designed and spaced for this purpose, and consequently position within the display element plane corresponds to viewing angle. Hence all points on the dashed line A in
The slanting lenticule arrangement can be applied to both monochrome and color displays. Considering, for example, the six-view scheme of
While the use of a slanted lenticular in the 6-view arrangement increases the horizontal resolution considerably, the vertical resolution is rather poor. This situation can be significantly improved, however, by applying the premise that each lenticule need not overlie and cooperate optically with a whole number of adjacent light-generating elements in a single row. In further examples, again using the same display panel, the lenticules are designed such that, rather than covering 3 or 4 light-generating elements on each row as: in the above-described arrangements, they instead cover 2½ or 3½ light-generating elements, that is, the pitch of the lenticular elements corresponds to 2½ and 3½ times the pitch of the light-generating elements in the row direction, to provide a 5-view and a 7-view system, respectively. In these, the output beams, 5 or 7, provided by each lenticule from the underlying light-generating elements have optical axes, which are in mutually different directions and spread angularly around the longitudinal axis of the lenticule. The arrangement for the seven-view system is shown in
In another embodiment, providing an 8-view system, and using the same display panel, the lenticules are slanted at the same angle as before but have a 33⅓% larger pitch and cover four light-generating elements on each row. Display elements of the 8 views are thus situated in groups comprising light-generating elements from two adjacent rows, four in each row. Each lenticule 16 in this case provides eight output beams from the underlying light-generating elements whose optical axes are in mutually different directions and angularly spread around the longitudinal axis of the lenticule. This arrangement has been found to give further improvement in vertical resolution.
While the matrix display panel in the above described embodiments comprises an LC display panel, it is envisaged that other kinds of electro-optical spatial light modulators and flat panel display devices, such as electroluminescent or plasma display panels, could be used.
As mentioned above, the number of pixels per lens does not have to be an integral number, although to date auto-stereoscopic displays have used regular repeating patterns of view pixels such as to give affixed arrangement of an integer number of views per lenticule. Applicant has recognized that this use of non-integer numbers of pixels per lens may be extended to enable the provision of a means whereby any arbitrary lenticular screen can be used on any (flat panel) pixel display simply by adjusting the mapping between the multiple perspective views and the pixels.
So, the optical means 408 introduce a spatial shift of the light in a plane parallel to the plane of the light-generating elements 402-406, whereby the spatial shift of the light Sx,y=(Δx,Δy) is related to the spatial displacement between the first location (x,y) of the particular light-generating element 402 and the second location of the corresponding optical means 408.
As described above, the arrangement of optical means is typically aligned with the structure of light-generating elements. A shift φ in the spatial domain, which is applicable for a particular light-generating element, may also be applied for a larger set of light-generating elements. For instance, such a set may comprise the light-generating elements corresponding to one particular view.
an input unit 502 for receiving an input signal 414;
a structure 10 of light-generating elements;
a structure 15 of optical means for directing light being generated by the structure 10 of light-generating elements in a number of directions corresponding to respective views; and
a computing unit 18 for computing luminance values for the respective light-generating elements on basis of the input signal 414, whereby for the computation of a particular luminance value for a particular light-generating element, the computing unit 18 is arranged to take into account a spatial displacement between a first location of the particular light-generating element and a second location of the corresponding one of the optical means, as described in connection with
The input unit 502 and the computing unit 18 may be implemented using one processor. Normally, these functions are performed under control of a software program product. During execution, normally the software program product is loaded into a memory, like a RAM, and executed from there. The program may be loaded from a background memory, like a ROM, hard disk, or magnetically and/or optical storage, or may be loaded via a network like Internet. Optionally an application specific integrated circuit provides the disclosed functionality.
The input signal may be a broadcast signal received via an antenna or cable but may also be a signal from a storage device like a VCR (Video Cassette Recorder) or Digital Versatile Disk (DVD). The signal is provided at the input connector 506. The image processing apparatus 500 might e.g. be a TV. Alternatively the image processing apparatus 500 does not comprise the optional display device but provides the output images to an apparatus that does comprise a display device. Then the image processing apparatus 500 might be e.g. a set top box, a satellite-tuner, a VCR player, a DVD player or recorder. Optionally the image processing apparatus 500 comprises storage means, like a hard-disk or means for storage on removable media, e.g. optical disks. The image processing apparatus 500 might also be a system being applied by a film-studio or broadcaster.
In the examples described above it is assumed that the input signal is a continuous signal representing the image data for all views of the multi-view display device 100. However it may be that the input signal is a discrete signal. It will be clear that in that case a resampling has to be performed. Typically this means that the following operations have to be performed:
computing a continuous signal on basis of the discrete input signal;
low-pass or band-pass filtering the continuous signal, taking into account the spatial resolution of the multi-view display device 100;
computing the luminance values by resampling the filtered continuous signal.
In a further alternative, the input signal represents the image data of a single view only, preferably in combination with depth or disparity information. The image data corresponding to the other views has to be rendered on basis of the provided image data and depth or disparity information. Then the rendering is e.g. as described in the article “Synthesis of multi viewpoint images at non-intermediate positions” by P. A. Redert, E. A. Hendriks, and J. Biemond, in Proceedings of International Conference on Acoustics, Speech, and Signal Processing, Vol. IV, ISBN 0-8186-7919-0, pages 2749-2752, IEEE Computer Society, Los Alamitos, Calif., 1997. Alternatively, the rendering is as described in “High-quality images from 2.5D video”, by R. P. Berretty and F. E. Ernst, in Proceedings Eurographics, Granada, 2003, Short Note 124.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be constructed as limiting the claim. The word ‘comprising’ not exclude the presence of elements or steps not listed in a claim. The word “a” or “an” preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements and by means of a suitable programmed computer. In the unit claims enumerating several means, several of these means can be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words are to be interpreted as names.
| Number | Date | Country | Kind |
|---|---|---|---|
| 05100268.1 | Jan 2005 | EP | regional |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/IB2006/050117 | 1/12/2006 | WO | 00 | 9/8/2008 |
