This invention relates to methods and apparatus for the holographic display of images.
Many small, portable consumer electronic devices incorporate a graphical image display, generally a LCD (Liquid Crystal Display) screen. These include digital cameras, mobile phones, personal digital assistants/organisers, portable music devices such as the iPOD (trade mark), portable video devices, laptop computers and the like. In many cases it would be advantageous to be able to provide a larger and/or projected image but to date this has not been possible, primarily because of the size of the optical system needed for such a display.
We have previous described, for example in WO 2005/059660, a method for image projection and display using appropriately calculated computer generated holograms displayed upon dynamically addressable liquid crystal (LC) spatial light modulators (SLMs). Broadly speaking in this technique an image is displayed by displaying a plurality of holograms each of which spatially overlaps in the replay field and each of which, when viewed individually, would appear relatively noisy because noise is added by phase quantisation by the holographic transform of the image data. However when viewed in rapid succession the replay field images average together in the eye of a viewer to give the impression of a reduced (low) noise image. The noise in successive temporal subframes may either be pseudo-random (substantially independent) or the noise in a subframe may be dependent on the noise in one or more earlier subframes with the aim of at least partially canceling this out, or a combination of both may be employed. More details of such OSPR-type procedures are described later.
a shows an example optical system for the holographic projection module of
An example of a suitable binary phase SLM is the SXGA (1280×1024) reflective binary phase modulating ferroelectric liquid crystal SLM made by CRL Opto (Forth Dimension Displays Limited, of Scotland, UK). A ferroelectric liquid crystal SLM is advantageous because of its fast switching time; binary phase devices are convenient but devices with three or more quantized phases may also be employed (use of more than binary phase enables the conjugate image to be suppressed, see WO 2005/059660). A single optical arrangement can be used for beam expansion prior to modulation, and for demagnification of the modulated light. Thus the lens pair L1 and L2 and the lens pair L3 and L4 may comprise at least part of a common optical system, used in reverse, in conjunction with a reflective SLM, for light incident on and reflected from the SLM.
b illustrates such a lens sharing arrangement, in which a polariser is included to suppress interference between light travelling in different directions, that is into and out of the SLM.
d shows a further arrangement in which just L2/L3 is shared between the collimation and demagnification stages. In this example a waveplate 34 is also employed to rotate the polarisation of the incident beam for the beamsplitter. In the optical arrangements of each of
A colour holographic projection system may be constructed by employing an optical system as described above to create three optical channels, red, blue and green superimposed to generate a colour image. In practice this is difficult because the different colour images must be aligned on the screen and a better approach is to create a combined red, green and blue beam and provide this to a common SLM and demagnifying optics. In this case, however, the different colour images are of different sizes; techniques to address this are described in our co-pending UK patent application no. GB0610784.1 filed 2 Jun. 2006, hereby incorporated by reference.
Referring again to
OSPR-type techniques substantially reduce the amount of computation required for a high quality holographic image display and the temporal averaging reduces the level of perceived noise. However in practice the level of perceived noise can be an order of magnitude worse than that predicted by theory because of (non-) uniformity of the image (uniformity is determined by the reciprocal signal energy variance—approximately the noise in the light parts of the image).
The inventor has recognised that a substantial contribution to this arises because the optical surfaces in a practical system are imperfect and, especially, because the SLM is not perfectly flat.
One solution to this, in an OSPR-type system, would be to calculate more subframes for each image frame, but in practice one is often already calculating as many subframes per image frame as the processing hardware and/or software allows. Another potential possibility would be to characterise an individual SLM and then compensate for the phase imperfections when calculating a hologram, but it would be preferable to avoid this particularly in a manufacturing process.
The inventor has identified techniques which address this issue; moreover these are not restricted to OSPR-type procedures for hologram calculation. Background prior art can be found in G2,320,156A and U.S. Pat. No. 5,048,935.
According to a first aspect of the invention there is therefore provided a method of compensating for spatial phase non-uniformities in a holographic image display system, the system including a substantially coherent light source illuminating an SLM, the method comprising displaying substantially the same hologram at a plurality of different positions on said SLM such that the displayed images replayed by said differently positioned holograms average to provide a displayed image with increased uniformity, in particular intensity uniformity.
Broadly speaking, the inventor has recognised that there is a property of holograms which can be exploited to compensate for spatial phase non-uniformities, which is that if a hologram is displaced perpendicular to the optical axis then there is no effect on the replayed image. This is because a position shift in the hologram plane merely results in a phase gradient in the image whereas the eye sees intensity. Thus one might imagine in a perfectly spatial-phase-uniform system that maintaining the SLM in a fixed position and moving the hologram on the display would have no visual effect. However, in a real system with spatial phase non-uniformities (aberrations) present on the display, at different positions on the display a particular pixel of the hologram, say the “top left hand corner” pixel of the hologram experiences different phase shifts due to said aberrations, resulting in independent intensity non-uniformities in the output image, which average out in the eye of an observer to provide an image with apparently enhanced intensity uniformity.
One might imagine that if the hologram were to be moved on the SLM to any substantial degree, the SLM would need to be much larger than the hologram in terms of numbers of pixels in the x- & y-directions. In fact the hologram can wrap around on the display so that the top left hand corner of the hologram may be “started” at any position on the SLM, once a boundary of the SLM is reached in the x- and/or y-direction the hologram wrapping around to continue at the opposite boundary. Optionally this wrap-around may neglect one or more rows/columns of boundary (edge) pixels of the SLM. The effect of placing the hologram in a variable location on the SLM is equivalent to moving the aberrations, effectively averaging these out.
In some embodiments the position of the hologram on the SLM may be dithered, that is moved over a relatively small range of positions, in which case the wrap-around may be omitted. However the degree to which the hologram is moved is preferably chosen taking into account the distance over which the phase across the SLM varies. In some measured SLMs a gradual variation over a substantial fraction of the area of the SLM could be seen, giving the appearance of “islands” of phase non-uniformity, and in embodiments, therefore, it is preferable to move the hologram over a distance of greater than 25%, 50% or 75% of the total number of pixels in the x- and/or y-direction.
In this latter case wrap-around is preferable. Although such wrap-around may be implemented in the software or hardware driving SLM, in other embodiments the SLM itself may incorporate a circular buffer. An SLM may incorporate a memory element for each pixel in which case each row and/or column of the SLM may be configured as a shift register and a circular buffer implemented by coupling the output at the end of a row/column back to its input. Such an arrangement has the advantage of reducing the load on the hologram calculation system since the hologram may be rapidly moved across the display, with wrap-around, by performing a circular shift along the row and/or column direction.
In some preferred embodiments the hologram is moved in both x- and y-directions to a set of substantially random (pseudo random) positions. In embodiments of the method at least five, at least ten or at least twenty different positions are employed for each subframe calculated, say, using an OSPR-type procedure.
Thus in another aspect the invention provides a spatial light modulator (SLM) for compensating for spatial phase non-uniformities in a holographic image display system, the SLM having a plurality of SLM pixels arranged in rows and columns, each having an associated pixel circuit including memory for storing pixel data indicating a value to display for the pixel on the SLM, and wherein the SLM further comprises circuitry to implement a circular shift register for one or both of said rows and columns to enable wrap-around rotation of said stored pixel data for displaying on said SLM.
The invention also provides a holographic image display system incorporating an SLM as described above.
The invention further provides a holographic image display system, the system including a substantially coherent light source illuminating an SLM, the system further comprising a mechanism for compensating for spatial phase non-uniformities, and wherein the mechanism is configured to move substantially the same hologram to a plurality of different positions on said SLM such that the displayed images replayed by said differently positioned holograms average to provide a displayed image with increased uniformity.
As previously mentioned, although in some preferred embodiments circuitry on the SLM performs a circular shift on the display data for the SLM, in embodiments this may instead be performed in software.
Thus the invention further provides processor control code to implement the above-described systems and methods, in particular on a data carrier such as a disk, CD- or DVD-ROM, programmed memory such as read-only memory (Firmware), or on a data carrier such as an optical or electrical signal carrier. Code (and/or data) to implement embodiments of the invention may comprise source, object or executable code in a conventional programming language (interpreted or compiled) such as C, or assembly code, code for setting up or controlling an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array), or code for a hardware description language such as Verilog (Trade Mark) or VHDL (Very high speed integrated circuit Hardware Description Language). As the skilled person will appreciate such code and/or data may be distributed between a plurality of coupled components in communication with one another.
Embodiments of the above described methods and systems may be incorporated into a consumer electronics device, or into an advertising or signage system, or into a helmet mounted or head-up display or, for example, an aircraft or automobile.
These and other aspects of the invention will now be further described, by way of example only, with reference to the accompanying figures in which:
a to 2d show, respectively, an example of an optical system for the holographic projection module of
a and 3b schematically show, respectively, a cross-section through an SLM illustrating variations in height and phase across the SLM surface, and typical phase variations across an SLM;
a and 4b show, respectively, a block diagram of a hologram data calculation system, and operations performed within the hardware block of the hologram data calculation system;
a to 7c illustrate operation of an embodiment of a method according to the invention, and phase wrap-around;
a and 8b illustrate averaging of phase non-uniformities; and
Some preferred implementations of the above described techniques are employed with an OSPR-type procedure, although applications of the techniques are not limited to such procedures. We therefore briefly describe such procedures. Further details can be found in GB0518912.1 (PCT/GB2006/050291) and GB0601481.5 (PCT/GB2007/050037), both hereby incorporated by reference.
Broadly speaking in our preferred method the SLM is modulated with holographic data approximating a hologram of the image to be displayed. However this holographic data is chosen in a special way, the displayed image being made up of a plurality of temporal sub-frames, each generated by modulating the SLM with a respective sub-frame hologram. These sub-frames are displayed successively and sufficiently fast that in the eye of a (human) observer the sub-frames (each of which have the spatial extent of the displayed image) are integrated together to create the desired image for display.
Each of the sub-frame holograms may itself be relatively noisy, for example as a result of quantising the holographic data into two (binary) or more phases, but temporal averaging amongst the sub-frames reduces the perceived level of noise. Embodiments of such a system can provide visually high quality displays even though each sub-frame, were it to be viewed separately, would appear relatively noisy.
The procedure is a method of generating, for each still or video frame I=Ixy, sets of N binary-phase holograms h(1) . . . h(N). In embodiments such sets of holograms form replay fields that exhibit mutually independent additive noise. An example is shown below:
Step 1 forms N targets Gxy(n) equal to the amplitude of the supplied intensity target Ixy, but with independent identically-distributed (i.i.t.), uniformly-random phase. Step 2 computes the N corresponding full complex Fourier transform holograms guv(n). Steps 3 and 4 compute the real part and imaginary part of the holograms, respectively. Binarisation of each of the real and imaginary parts of the holograms is then performed in step 5: thresholding around the median of muv(n) ensures equal numbers of −1 and 1 points are present in the holograms, achieving DC balance (by definition) and also minimal reconstruction error. The median value of muv(n) may be assumed to be zero with minimal effect on perceived image quality.
a (from GB0511962.3, filed 14 Jun. 2005, incorporated by reference) shows a block diagram of a hologram data calculation system configured to implement this procedure. The input to the system is preferably image data from a source such as a computer, although other sources are equally applicable. The input data is temporarily stored in one or more input buffer, with control signals for this process being supplied from one or more controller units within the system. The input (and output) buffers preferably comprise dual-port memory such that data may be written into the buffer and read out from the buffer simultaneously. The control signals comprise timing, initialisation and flow-control information and preferably ensure that one or more holographic sub-frames are produced and sent to the SLM per video frame period.
The output from the input comprises an image frame, labelled I, and this becomes the input to a hardware block (although in other embodiments some or all of the processing may be performed in software). The hardware block performs a series of operations on each of the aforementioned image frames, I, and for each one produces one or more holographic sub-frames, h, which are sent to one or more output buffer. The sub-frames are supplied from the output buffer to a display device, such as a SLM, optionally via a driver chip.
b shows details of the hardware block of
The purpose of the phase-modulation block is to redistribute the energy of the input frame in the spatial-frequency domain, such that improvements in final image quality are obtained after performing later operations.
The quantisation block takes complex hologram data, which is produced as the output of the preceding space-frequency transform block, and maps it to a restricted set of values, which correspond to actual modulation levels that can be achieved on a target SLM (the different quantised phase retardation levels may need not have a regular distribution). The number of quantisation levels may be set at two, for example for an SLM producing phase retardations of 0 or π at each pixel.
In some preferred embodiments the quantiser is configured to separately quantise real and imaginary components of the holographic sub-frame data to generate a pair of holographic sub-frames, each with two (or more) phase-retardation levels, for the output buffer.
In the OSPR approach we have described above subframe holograms are generated independently and thus exhibit independent noise. In control terms, this is an open-loop system. However one might expect that better results could be obtained if, instead, the generation process for each subframe took into account the noise generated by the previous subframes in order to cancel it out, effectively “feeding back” the perceived image formed after, say, n OSPR frames to stage n+1 of the algorithm. In control terms, this is a closed-loop system.
One example of this approach comprises an adaptive OSPR algorithm which uses feedback as follows: each stage n of the algorithm calculates the noise resulting from the previously-generated holograms H1 to Hn-1, and factors this noise into the generation of the hologram Hn to cancel it out. As a result, it can be shown that noise variance falls as 1/N2. An example procedure takes as input a target image T, and a parameter N specifying the desired number of hologram subframes to produce, and outputs a set of N holograms H1 to HN which, when displayed sequentially at an appropriate rate, form as a far-field image a visual representation of T which is perceived as high quality. More details can be found in GB0518912.1 and GB0601481.5 (ibid), hereby incorporated by reference in their entirety.
Referring now to
Some benefit can be obtained with just two different positions of the hologram on the SLM but preferably a larger number is employed, for example ten different positions in order to provide, potentially, a tenfold increase in uniformity. The degree of movement of a hologram depends upon the expected phase non-uniformity to be addressed and should preferably be sufficient to average out most of this phase non-uniformity. For example the hologram may be moved as far as an average distance over which a phase change of π is expected. In general the number of positions and degree of movement of the hologram may be chosen by routine experiment and/or characterisation of one or a batch of SLMs.
a and 8b illustrate, schematically, how the islands of phase non-uniformity shown in
Referring now to
The inventor has recognised that because the liquid crystal switches faster than data can be loaded into the display it is advantageous to be able to load the data just once and to move the hologram, for example as shown in
Applications for the described techniques and modulators include, but are not limited to the following: mobile phone; PDA; laptop; digital camera; digital video camera; games console; in-car cinema; navigation systems (in-car or personal e.g. wristwatch GPS); head-up and helmet-mounted displays for automobiles and aviation; watch; personal media player (e.g. MP3 player, personal video player); dashboard mounted display; laser light show box; personal video projector (a “video iPod®” concept); advertising and signage systems; computer (including desktop); remote control unit; an architectural fixture incorporating a holographic image display system; more generally any device where it is desirable to share pictures and/or for more than one person at once to view an image.
No doubt many effective alternatives will occur to the skilled person and it will be understood that the invention is not limited to the described embodiments and encompasses modifications apparent to those skilled in the art lying within the spirit and scope of the claims appended hereto.
Number | Date | Country | Kind |
---|---|---|---|
0701338.6 | Jan 2007 | GB | national |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/GB2007/050755 | 12/12/2007 | WO | 00 | 11/30/2009 |