Patent Application
20100277566
This invention relates to methods and apparatus for displaying images holographically.
We have previously described techniques for displaying an image holographically (see, for example, WO 2005/059660, WO 2006/134398, WO 2006/134404, WO 2007/031797, WO 2007/085874, and WO 2007/110668, all hereby incorporated by reference in their entirety). They have a range of applications including, for example, in hand held battery powered devices such as digital cameras, mobile phones, portable media players, laptop computers and the like. In these and other applications there is, however, a general need to increase the operating efficiency.
Background prior art can be found in EP1702243A; EP1498766; and Palima D. and Dania V. R., “Holographic projection of arbitrary light patterns with a suppressed zero-order beam”, 2007, Vol 46, No 20, Applied Optics.
We will describe techniques which are able to achieve a substantial increase in efficiency, in particular with so-called multiphase spatial light modulators as distinct from binary phase spatial light modulators (that is SLMs which have more than two different selectable phase delay values for a pixel as opposed to binary devices in which a pixel has only one of two phase delay values). Multiphase SLMs include continuous phase SLMs, although when driven by digital circuitry these devices are necessarily quantised to a number of discrete phase delay values.
The techniques we have previously described for efficient computation of holograms to display on an SLM in order to reproduce a desired image (ibid) we broadly refer to as OSPR (One Step Phase Retrieval)-type procedures. Details of an example procedure are given later, but the skilled person will understand that the schemes for increasing efficiency we describe later are not limited to any particular method for calculating a hologram for display on an SLM in order to reproduce an image (which here includes a frame of a video), albeit that employing an OSPR-type procedure is advantageous.
Each holographic subframe may be generated using an OSPR-type procedure in which, broadly speaking, an image is displayed by displaying a plurality of holograms each of which spatially overlaps in the replay field. Each of which, when viewed individually, would appear relatively noisy because noise is added, for example by phase quantisation by the holographic transform of the image data. However when viewed in rapid succession the replay field images average together in the eye of a viewer to give the impression of a low noise image. The noise in successive temporal subframes may either be pseudo-random (substantially independent) or the noise in a subframe may be dependent on the noise in one or more earlier subframes, with the aim of at least partially cancelling this out, or a combination may be employed.
An example of a suitable binary phase SLM is the SXGA (1280×1024) reflective binary phase modulating ferroelectric liquid crystal SLM made by CRL Opto (Forth Dimension Displays Limited, of Scotland, UK). A ferroelectric liquid crystal SLM is advantageous because of its fast switching time; binary phase devices are convenient but devices with three or more quantized phases (in the art, referred to as multiphase SLMs) may also be employed. Binary quantization results in a conjugate image whereas the use of more than binary phase suppresses the conjugate image (see WO 2005/059660).
A colour holographic projection system may be constructed by employing an optical system as described above to create three optical channels, red, blue and green superimposed to generate a colour image. In practice this is difficult because the different colour images must be aligned on the screen and a better approach is to create a time sequence of red, green and blue beams and provide these to a common SLM and demagnifying optics. In this case, however, the different colour images are of different sizes; techniques to address this are described in our co-pending UK patent application no. GB0610784.1 filed 2 Jun. 2006, hereby incorporated by reference.
Referring again to
OSPR-type techniques substantially reduce the amount of computation required for a high quality holographic image display and the temporal averaging reduces the level of perceived noise. However there is a general need to increase the optical efficiency of holographic image display techniques.
According to a first aspect of the invention there is therefore provided a method of displaying an image holographically using a spatial light modulator (SLM), said SLM having a plurality of SLM pixels, the method comprising: displaying a diffraction pattern on said pixels of said SLM; and illuminating said pixels of said SLM such that light diffracted by said diffraction pattern on said SLM pixels comprises a content of said displayed image, a variation in brightness of said displayed image across said displayed image being modulated by an intensity envelope determined by the diffraction pattern of an individual said pixel, for example a sine envelope; and wherein the method further comprises moving (a peak or centre of gravity) of said intensity envelope away from a zero order spot and towards a centre of said displayed image by imposing a pattern of phase delay across said SLM pixels, said pattern of phase delay repeating at a spatial interval corresponding to a pixel interval of said SLM.
In embodiments of the method the pixels are square or rectangular and the pixel diffraction pattern comprises a sine function (envelope), although theoretically pixels of a different shape, for example circular, may be employed which would have a different shape of diffraction pattern (for example a Bessel function if circular). In some embodiments of the method, for example those employing a binary phase SLM, a displayed image is offset from a zero order undiffracted spot because of the presence of a conjugate image. However where a multiphase SLM is employed there is no conjugate image but instead it is desirable to displace the displayed image away from the zero order undiffracted spot since otherwise this can interfere with the visual appearance of the displayed image. In either case the efficiency of the image display technique can be increased by displacing a peak or centre of gravity of the intensity envelope (determined by the diffraction pattern of an individual pixel, for example a sine envelope) away from the zero order spot and towards the centre of the displayed image, in particular by imposing a regular, repeating pattern of phase delay across each pixel.
In embodiments of the method the image used to determine the diffraction pattern, for example by a Fourier (or inverse Fourier) transform is multiplied by the inverse of the pixel diffraction pattern, for example an inverse sine function, prior to performing a holographic transform. This has the effect of partially compensating for light loss in the wings of the pixel diffraction pattern, for example converting a sine function into an approximate top hat function. However whether or not some compensation for the pixel diffraction pattern is applied prior to transforming the image data into hologram data for display on the SLM, there is an efficiency gain to be achieved by moving the peak or centre of gravity of the overall intensity envelope away from the zero order undiffracted spot. In the case of a binary SLM this effectively moves the peak or centre of gravity of the intensity envelope towards the displayed image of the conjugate image pair (that is the image of the pair which has the desired orientation). In the case of a multiphase SLM embodiments of the technique enable the displayed image to be moved away from the undiffracted spot, avoiding the visual impairment of the image this would otherwise cause, the peak or centre of gravity of the intensity envelope being correspondingly moved. Embodiments of the technique are particularly advantageous when employed in the context of a multiphase SLM (that is an SLM having pixels with more than binary phase values) since in this case the peak or centre of gravity of the intensity envelope can be moved away from the undiffracted spot allowing the efficiency benefit of losing the conjugate image (an effective doubling of the efficiency) whilst at the same time avoiding problems with superimposition of the displayed image upon a bright central spot which would otherwise make the technique impractical. The difficulties of obtaining a high efficiency image at the centre of the output field with a multiphase SLM caused by the practical impossibility of completely suppressing the zero order bright spot have not previously been recognised. (This spot is effectively a focus as compared with light distributed over pixels of the image and therefore only a very small amount of light is needed for this spot to compete in brightness with a pixel of the displayed image).
In preferred embodiments the pattern of phase delay across each SLM pixel comprises a phase delay which increases across a pixel and then repeats, rather in the manner of a blazed grating. For a two-dimensional array of pixels, this “blaze” is applied in only one of two orthogonal directions in the array. Across a pixel the phase delay increases in a direction opposite to which the peak of the intensity envelope is moved.
In some preferred embodiments, rather than the phase delay comprising a linear increase in phase a stepped pattern is employed across a pixel, effectively quantising the phase delay across a pixel into two or more (for example, four, eight and so forth) steps. This has the advantage of facilitating fabrication and yet being almost as efficient as a linear increase over a pixel. In embodiments the phase delay pattern is configured to displace the peak or centre of gravity of the intensity envelope by a distance, at said displayed image, which corresponds to a change in phase shift of substantially π/2 across a pixel. Thus the pattern of phase changes preferably chosen to give substantially the same displacement of the peak or centre of gravity of the intensity envelope as would be provided by a linear zero to π/2 ramp phase delay pattern across a pixel. In embodiments the aim is for the peak of the sine function or the centre of gravity of the top hat to be substantially at the middle of the displayed image and therefore a displacement corresponding to a phase delay of π/2 is preferred (recalling that a phase change of 2π would shift the envelope by one order).
In embodiments if the phase delay (or refractive index) is graded in one orthogonal direction, say the y-direction in the hologram plane then the intensity envelope is moved in the same direction, say the v-direction where the image is displayed. Thus in embodiments of the method image data defining an image to be displayed is located in one half of an image data plane (for example by padding the image) bisected in a direction perpendicular to the direction in which the intensity envelope is moved, prior to performing a holographic transform, for example a Fourier or inverse Fourier transform to determine the diffraction pattern from data in both halves of the image data plane. Thus in embodiments the image to be displayed is effectively embedded in, say, the upper half of an image data plane prior to performing the holographic transform, and the pattern of phase delay on the SLM is arranged (graded in this example in a vertical direction) so that the reconstructed or replayed image is in the upper half of the replay field and so that the intensity envelope is moved towards the upper half of the replay field so that the displayed image and intensity envelope preferably substantially coincide.
In some preferred embodiments employing a stepped pattern of phase delay, rather than quantising at values of, say, zero and π/2, the stepped phase delays are chosen to be intermediate between these two values. More particularly in the case of a single step within each pixel (two different phase delay values) the phase delay values are preferably π/4 apart; with four steps the phase delays are preferably π/8 apart and, in general, the difference between phase delay values is preferably π÷(2×the number of different phase delays across a pixel). This is a more accurate quantisation of the linear ramp than, say, in the two level case choosing values zero and π/2 and stems from the observation that phase delays wrap around and that, effectively, an arbitrary zero point may be selected.
In some preferred embodiments the SLM comprises a reflective SLM and the stepped phase delays are implemented as a series of stripes, for example one stripe per pixel, each stripe having a width of approximately half a pixel width (with this arrangement each pixel is effectively divided into two down the middle and therefore it could be considered that there are two stripes per pixel). In other arrangements there may be more stripes per pixel—for example in the case of four different phase delay values per pixel there may be three stripes per pixel (or four stripes per pixel, depending upon whether the “zero delay” stripe is counted).
Thus in a related aspect there is provided a spatial light modulator (SLM), in particular for use in a method or apparatus as described above, the SLM having a plurality of pixels and wherein said SLM has a pattern of phase delay across said pixels, said pattern comprising a phase delay which increases across a said pixel and repeats for each pixel.
In some preferred embodiments the SLM comprises a reflective liquid crystal SLM, more particularly a multiphase SLM. However other multiphase technologies may alternatively be employed.
In the case of a reflective SLM the SLM may conveniently be fabricated by etching a reflective, for example aluminium layer of the SLM to define the stripes. This is straightforward to implement with one (or more) additional etch steps at little additional cost.
The invention also provides apparatus for displaying an image holographically using a spatial light modulator (SLM), said SLM having a plurality of SLM pixels, the apparatus comprising: a system to display a diffraction pattern on said pixels of said SLM; a laser to illuminate said pixels of said SLM such that light diffracted by said diffraction pattern on said SLM pixels comprises a content of said displayed image, a variation in brightness of said displayed image across said displayed image being modulated by an intensity envelope determined by a diffraction pattern of an individual said pixel, for example a sine envelope; and wherein said apparatus is configured to move (a peak or centre of gravity of) said intensity envelope away from a zero order spot and towards a centre of said displayed image by imposing a pattern of phase delay across said SLM pixels, said pattern of phase delay repeating at a spatial interval corresponding to a pixel interval of said SLM.
Preferred embodiments of the apparatus include corresponding device features to those mentioned above in connection with preferred features of the aforementioned methods according to aspects and embodiments of the invention. In embodiments the apparatus may also include a system to perform a holographic transform on input image data to convert this to holographic data for display on the SLM. In some preferred embodiments of the apparatus, and of the foregoing methods, a diffraction pattern for display on the SLM is determined using an OSPR-type method (and thus multiple diffraction patterns may be displayed on the SLM to provide content for a single displayed image or frame of video).
In a related aspect the invention provides a method of displaying an image holographically using a multiphase pixelated spatial light modulator (SLM), the method comprising: displaying one or more holograms on said SLM using more than two different phase values for pixels of said SLM such that when said SLM is illuminated said image is displayed in a replay field of said hologram substantially without a conjugate image; and applying a modulating phase pattern to said displayed hologram to move said image displayed in a said replay field away from a zero order substantially undiffracted spot from said illuminated SLM.
The invention still further provides apparatus for displaying an image holographically using a multiphase pixelated spatial light modulator (SLM) using more than two different phase values for pixels of said SLM, the apparatus comprising: a system to display one or more holograms on said SLM such that when said SLM is illuminated said image is displayed in a replay field of said hologram substantially without a conjugate image; and wherein a phase modulation is applied to said pixels of said SLM such that, in operation, said image displayed in said replay field is displaced away from a zero order substantially undiffracted spot from said illuminated SLM.
In some preferred embodiments of the above-described methods and apparatus the SLM comprises a reflective SLM, such as reflective liquid crystal SLM, and the SLM incorporates a mechanism to apply the phase modulation, for example a stepped surface within the SLM to provide a regular, repeating phase delay pattern to provide a “blaze” in one direction across the SLM.
These and other aspects of the invention will now be further described, by way of example only, with reference to the accompanying figures in which:
a to 3f show, respectively, a model of holographic image formation using an SLM, movement of a sine attenuation peak as a result of applying a phase ramp to an SLM pixel, mapping of sine pixel diffraction envelopes to a displayed image for a binary phase and for a multiphase SLM; and mapping of sine pixel diffraction envelopes to a displayed image according to embodiments of the invention for a binary phase and for a multiphase SLM;
a to 4d show, respectively, a simplified view from above and vertical cross section view of a reflective liquid crystal SLM, a schematic illustration of a shift in the peak intensity of a pixel diffraction envelope, and phase shifts for two adjacent example pixels illustrating a smooth phase shift ramp (left), a four step ramp approximation (middle; Q=4 stripes) and a two step ramp approximation (right; Q=2 stripes);
a and 5b show, respectively, a simulated reconstruction field with sine attenuation without a peak-shifting phase pattern applied to the SLM, and with a Q=4 stripes peak shifting phase pattern applied to the SLM; and
a and 6b show, respectively, a block diagram of an OSPR hologram data calculation system, and operations performed within the system of
We will describe techniques for improving diffraction efficiency in holographic projection by shifting the SLM pixel diffraction (sine attenuation) envelope.
Images formed holographically suffer from an effect known as “sine attenuation”, caused by the square pixels on the SLM imparting a sinc-shaped intensity modulation centred on the zero-order over the whole of the reconstruction field, resulting in an undesirable decrease in intensity towards the edges of the field. This results in two problems.
We describe super-imposing a substantially identical phase ramp on each SLM pixel to shift the sine attenuation envelope so its centre lies at the centre of the image, minimising this drop in efficiency.
The reconstruction field obtained through holographic replay depends on the shape of the SLM, the hologram displayed on the device, the SLM's pixel sampling grid and the shape of the SLM pixel itself, as shown in
Referring to
Because a phase ramp in Fourier space corresponds to a position shift in image space, incorporating a phase ramp (with a gradient chosen to shift the y-direction by a quarter of the field height) into the SLM pixel unit would have the effect of shifting the sine attenuation envelope onto the centre of the image as desired (
b shows a simplified representation of the problem in the case of a binary phase SLM, where it can be seen that in the vertical direction the centre of the sine envelope is aligned with the centre of the replay field 300 whereas the displayed image 302 is displaced above this and is thus in the tail of the sine distribution. The dashed lines over the sine envelopes correspond to an approximate top hat function which is obtained by multiplying the input image by the inverse of the pixel sine function prior to applying a holographic, for example Fourier or Fresnel transform. The displayed image shown in
Referring now to
We now describe how the pixel diffraction envelope can be displaced as shown in
Referring now to
Referring again to
Referring now to
We next describe the effect of quantisation of the (conceptual) pixel phase ramp.
The above analysis referred to applying a linear phase ramp to each SLM pixel. In practice, it is preferable, for ease of fabrication, to divide each pixel into a finite number Q of stripes which together form a quantised grating to shift the sine envelope in the desired way. The larger the Q, the closer the pixel comes to resemble a smooth phase ramp, and so the closer the actual gain to the theoretical improvement possible of 23%. Because phase is relative, we can fix the phase shift of the first stripe to zero and define the others relative to that, therefore meaning that Q−1 additional mask steps are required when manufacturing the display. So choice of Q in a real application is dependent on the trade-off between cost due to the additional fabrication steps and the increasing efficiency gain. We have found that Q=2 (i.e. one additional mask step) gives an improvement of around 18%, with Q=4 (three additional mask steps) giving an improvement of 22%. Therefore there is little point going above Q=4. Indeed, due to relative ease of manufacture, Q=2 may represent the optimum trade-off between cost/complexity and performance.
On a pixel featuring a linear phase ramp (i.e. Q large or theoretically infinite), the phase change across the pixel should be π/2 (
As an example, for Q=2 the phase shifts are 0 and π/4 (
If this SLM sub-pixel structure is created by further etching the aluminium pixel layer in a device, how a given phase shift P relates to the required etch depths d is determined by the refractive index n of the liquid crystal material according to the relation
where λ may be chosen to be equal to the central design wavelength of the display, nominally green (532 nm). So, if we choose Q=2 and an LC material with n=1.4, an additional processing step would be employed involving etching half of each pixel's aluminium layer by an additional depth of 47.5 nm. Equally, instead of etching, additional aluminium of this thickness could be deposited onto the existing layer.
The images shown in
Some preferred implementations of the above-described techniques are employed with an OSPR-type procedure, although applications of the techniques are not limited to such procedures. We therefore briefly describe such procedures. Further details can be found in our earlier applications (ibid).
Broadly speaking in our preferred method the SLM is modulated with holographic data approximating a hologram of the image to be displayed. However this holographic data is chosen in a special way, the displayed image being made up of a plurality of temporal sub-frames, each generated by modulating the SLM with a respective sub-frame hologram. These sub-frames are displayed successively and sufficiently fast that in the eye of a (human) observer the sub-frames (each of which have the spatial extent of the displayed image) are integrated together to create the desired image for display. Temporal averaging amongst the sub-frames reduces the perceived level of noise even though each sub-frame, were it to be viewed separately, would appear relatively noisy.
The OSPR procedure is a method of generating, for each image I=Ixy, sets of N binary-phase holograms h(1) . . . h(N). In embodiments such sets of holograms form replay fields that exhibit mutually independent additive noise. An example is shown below:
Step 1 forms N targets Gxy(n) equal to the amplitude of the supplied intensity target Ixy, but with independent identically distributed (i.i.t.), uniformly-random phase. Step 2 computes the N corresponding full complex Fourier transform holograms guv(n). Steps 3 and 4 compute the real part and imaginary part of the holograms, respectively. In this example routine binarisation of each of the real and imaginary parts of the holograms is performed in step 5 (thresholding around the median muv(n), or zero aims to ensure substantially equal numbers of −1 and 1 points are present in the holograms, for DC balance).
a shows a block diagram of a hologram data calculation system to implement this procedure. Input image data is temporarily stored in one or more input buffers, with control signals supplied from a controller. The input (and output) buffers preferably comprise dual-port memory such that data may be written into the buffer and read out from the buffer simultaneously. The control signals comprise timing, initialisation and flow-control information so that one or more holographic sub-frames are produced and sent to the SLM per video frame period. The output from the input buffer comprises an image frame, I, and this becomes the input to a hardware block (although in other embodiments some or all of the processing may be performed in software) which performs a series of operations on each of the image frames, I, and for each one produces one or more holographic sub-frames, h, which are sent to an output buffer and supplied from there to a display device such as a SLM, optionally via a driver chip.
b shows details of the system of
In the OSPR approach we have described above subframe holograms are generated independently and thus exhibit independent noise. However the generation process for each subframe can take into account the noise generated by the previous subframes in order to cancel it out, effectively “feeding back” the perceived image formed after, say, n OSPR frames to stage n+1 of the procedure, forming a closed-loop system. Such an adaptive (AD) OSPR procedure uses feedback as follows: each stage n of the algorithm calculates the noise resulting from the previously generated holograms H1 to Hn-1, and factors this noise into the generation of the hologram Hn to cancel it out. As a result, noise variance falls as 1/N2 (where a target image T outputs a set of N holograms). More details can be found in WO2007/031797 and WO2007/085874.
Applications for the described techniques and modulators include, but are not limited to the following: mobile phone; PDA; laptop; digital camera; digital video camera; games console; in-car cinema; navigation systems (in-car or personal e.g. wristwatch GPS); head-up and helmet-mounted displays for automobiles and aviation; watch; personal media player (e.g. MP3 player, personal video player); dashboard mounted display; laser light show box; personal video projector (a “video iPod®” concept); advertising and signage systems; computer (including desktop); remote control unit; an architectural fixture incorporating a holographic image display system; more generally any device where it is desirable to share pictures and/or for more than one person at once to view an image.
No doubt many effective alternatives will occur to the skilled person and it will be understood that the invention is not limited to the described embodiments and encompasses modifications apparent to those skilled in the art lying within the spirit and scope of the claims appended hereto.
| Number | Date | Country | Kind |
|---|---|---|---|
| 0721571.8 | Nov 2007 | GB | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/GB2008/051005 | 10/27/2008 | WO | 00 | 7/16/2010 |
