The present invention generally relates to color holographic displays, and more particularly relates to a method and an apparatus for space division multiplexed color holographic displays.
The ability to display color three-dimensional (3D) objects is a desirable characteristic when developing a 3D holography system. A color holographic display provides a greatly improved sense of reality from reconstructed 3D objects as compared to mono-color reconstructed images. However, holography uses diffraction to reconstruct 3D objects, and holography requires single or narrow band wavelength reference light for good depth feeling. Also, holography is not a direct image of a hologram, making most color technologies used for two-dimensional displays unsuitable for 3D color holography.
Creating color in a display consists of two parts: color multiplexing and color mixing. Color multiplexing refers to selection of the correct color components for reconstruction. For example, one input 3D object point has red (R), green (G) and blue (B) color components such as aR, aG and aB. In reconstruction of an image for display, these color components should be equivalent to aR, aG and aB, not aG, aB and aR or other combinations. Correct selection of color components requires both the generation of holograms and the color reconstruction using an optical system to work with each other. Color mixing reconstructs different color components of the same 3D object point at the same position as specified by an input. Without color mixing, reconstruction will be color-separated or blurred.
Drawbacks of conventional holographic color display systems include color hologram generation that is computationally expensive, while providing inconsistent color reconstruction quality. Conventional systems also typically meet or exceed bandwidth and storage limitations because a required spatial light modulator and a storage device for hologram data each have their own refresh rate, bandwidth, and capacity limitations, as well as separate launching hardware and computer requirements. Further, conventional systems are typically complicated systems that are difficult to manage and expensive to operate.
Thus, what is needed is a 3D color holographic system with low computational load requirements, low bandwidth and low storage requirements, simple system design and control, and good color reconstruction quality. Furthermore, other desirable features and characteristics will become apparent from the subsequent detailed description and the appended claims, taken in conjunction with the accompanying drawings and this background of the disclosure.
According to the Detailed Description, a method for color holographic display is provided. The method for color holographic display includes modifying a plurality of color component light beams by passing light from each of a plurality of color component light sources through a corresponding one of a plurality of color sub-hologram masks to generate a corresponding plurality of modified color component light beams. The method also includes multiplexing the plurality of modified color component light beams to generate a multiplexed masked color light beam and creating a color holographic display of a three-dimensional object by shining the multiplexed masked color light beam onto modulated computer generated holographic information corresponding to the object in order to reconstruct each point of the object with a color from a portion of the multiplexed masked color light beam and at a location and intensity determined in response to a corresponding portion of the computer generated holographic information.
In accordance with another aspect, an additional method for color holographic display is provided. This method for color holographic display includes combining a plurality of color component light beams from each of a plurality of color component light sources to generate a multiplexed color light beam and modifying the multiplexed color light beam by filtering the multiplexed color light beam to generate a multiplexed masked color light beam. The method further includes creating a color holographic display of a three-dimensional object by shining the multiplexed masked color light beam onto modulated computer generated holographic information corresponding to the object in order to reconstruct the object by combining each portion of the multiplexed masked color light beam at a location and an intensity determined in response to a corresponding portion of the computer generated holographic information.
In accordance with yet another aspect, a method for generating a plurality of points for a color holographic display in a reconstructed object space of a three-dimensional object in a three-dimensional object space is provided. The method for generating the plurality of points of the color holographic display includes determining modulation values for each of a plurality of pixels in a two-dimensional representation of the three-dimensional object in response to locations and color intensity values of each of the plurality of points of the object in the three-dimensional object space and providing the modulation values for each of the plurality of pixels as computer generated holographic information to a spatial light modulator. The method further includes combining each of the modulation values with a corresponding predetermined wavelength of light by shining a multiplexed color light beam on the spatial light modulator programmed with the computer generated holographic information in order to generate the color holographic display of the three-dimensional object in the reconstructed object space.
In accordance with a further aspect, a color holographic display system is provided. The color holographic display system includes a plurality of component light sources, one or more light modifiers and a spatial light modulator. The one or more light modifiers are coupled to the plurality of light sources and generate a modified masked multiplexed light beam from light beams generated by and radiated from the plurality of component light sources. The one or more light modifiers include one or more light modifiers such as light masks, wavelength bandpass filters and light beam color multiplexer structures. The spatial light modulator receives computer generated holographic information and varies modulation of the modified masked multiplexed light beam shined on the spatial light modulator in response to the computer generated holographic information to create a color holographic display of a three-dimensional object by reconstructing the object through modulating the multiplexed masked color light beam at locations determined in response to the computer generated holographic information within a three-dimensional reconstructed object space.
The accompanying figures, where like reference numerals refer to identical or functionally similar elements throughout the separate views and which together with the detailed description below are incorporated in and form part of the specification, serve to illustrate various embodiments and to explain various principles and advantages in accordance with the present invention.
And
Skilled artisans will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures illustrating integrated circuit architecture may be exaggerated relative to other elements to help to improve understanding of the present and alternate embodiments.
The following detailed description is merely exemplary in nature and is not intended to limit the invention or the application and uses of the invention. Furthermore, there is no intention to be bound by any theory presented in the preceding background of the invention or the following detailed description.
While there are many conventional color holography display systems, these conventional display systems have drawbacks which hamper utilization of such systems. For example, a space division multiplexing (SDM) color holography system on multiple holograms generates one hologram for each of the three color components (red (R), green (G), and blue (B)), and launches these holograms to three individual spatial light modulators (SLMs). Each SLM is shined with a corresponding color reference light in accordance with the hologram launched to it. Different color reconstructions from the three SLMs at different positions of the optical system are optically combined. This conventional SDM color holography system on multiple holograms implementation, however, combines different reconstruction spaces thereby requiring a careful design of the optical system for display of such color holograms. While an SDM on multiple holograms implementation has no requirement for synchronization, such system requires increased computer generated hologram (CGH) computational load and hologram storage/transmission bandwidth three times that required for traditional single color holography.
Time Division Multiplexing (TDM) color holography display systems are another conventional method for color holography. Holograms for different color components are launched on an SLM sequentially and, when a hologram for one color component is launched, only the reference light for that color turns on. While there is only one reconstruction space and thus no optical combination is needed, TDM color holography display systems impose a two-times higher refresh rate requirement on the SLM device as compared to traditional single color holography or an SDM on multiple holograms color holography display system. The CGH computational load and hologram storage/transmission bandwidth are also three times that of single color holography.
A third type of conventional color holography display system is a SDM on single hologram (three quarters) color holography display system. Instead of implementing SDM on multiple holograms, this conventional technique utilizes different portions of the same hologram for different colors. SDM on single hologram (three quarters) divides a hologram into four sub-holograms, where different color components are launched on each sub-hologram and shined with reference light of a corresponding color. Although sub-holograms are implemented on a single SLM, each of the different colors still have different reconstruction spaces and it is difficult to combine these shifted reconstruction spaces. Shifting by hologram computation may be utilized to combine the reconstruction spaces, but the hologram has defection angle limitations. Thus, eventually the utilizable reconstruction space for combining each color is reduced to around one quarter of the reconstruction space provided by traditional single color holography. While mixing can be achieved by capturing a full color reconstruction projected on a capturing device such as a charge-coupled device (CCD) image sensor, when viewed by human eyes it is likely that a fixed portion of the reconstruction space would only show one color. None of these drawbacks is balanced by processing, storage or transmission savings because the CGH computational load and hologram storage/transmission bandwidth for a conventional SDM on single hologram (three quarters) system is the same as the requirements for a single color holography.
Another conventional spatial division multiplexing method used for color holography is angular multiplexing; more specifically an SDM on single hologram (angular selective) color holography system. In this fourth type of conventional color holography system, holograms for different color components are computed with reference light with different incident angles and superimposed to form a final hologram for all colors. During the reconstruction process, the color reference light shines the hologram at the same angles used for the CGH computation. Different color output light will have different output angles from the SLM, so combining different color reconstruction and optical design are difficult. Further, the CGH computational load for this method is around three times the load for traditional single color holography, while hologram storage/transmission bandwidth remains the same.
The drawbacks of these conventional systems is summarized in Table 1:
One method for color mixing in a color holographic display is by theoretically analyzing and numerically correcting distortion and mismatch introduced by each optical element in the holographic reconstruction setup, one by one. This is typically termed element by element correction. While element by element correction can provide accurate correction results because it is specific to the individual holographic reconstruction system, slight changes like angle or position change of a component will require a different set of correction values. While correction can be done at the element level, such correction is computationally tedious and time consuming. The drawbacks of such an element by element correction system are shown in Table 2 below:
In accordance with a present embodiment, a new space division multiplexing on a single SLM for color holographic three-dimensional display (referred to as SDM on single hologram (distributed sub-hologram)) is presented. The present embodiment, as will be seen from the description hereinbelow, distributes three sub-holograms for each of three color components, red (R), green (G) and blue (B), of a three-dimensional object on a single hologram. After shining on these sub-holograms a multiplexed masked color light beam generated by modifying a plurality of color component light beams by correspondingly masking reference light sources, reconstructions of different color components of the three-dimensional object will be automatically combined in a reconstructed object space to create a color holographic display of the object.
Referring to
Referring to
The color selective matrix M(xh,yh) 202 can be applied to various kinds of CGH algorithms and makes these algorithms suitable to generate holograms for SDM on single hologram (distributed sub-holograms). When computing the value for a hologram pixel, the color component and wavelength specified by matrix position within the color selective matrix M(xh,yh) 202 for that pixel are used. Examples of color selective matrix 202 generation is described hereinafter in regards to modifications of a conventional CGH algorithm, termed coherent ray trace (CRT), and another CGH algorithm using look-up tables, termed LUTs. The conventional CRT algorithm simulates light prorogation from an object point to a hologram pixel, as shown in equation 1, where I(xh,yh) is the complex hologram with xh,yh being its coordinates, N is the total number of object points, aj(xj,yj,zj) is the object point intensity at object point coordinate xj, yj,zj, λ is the wavelength of the reference light beam, Δx equals xh−xj, and Δy equals yh−yj:
Combining the CRT with the color selective matrix M(xh,yh), converts equation 1 to equation 2, where c is one of the colors, λc is the wavelength of the reference beam for c, and aj,c is the object point intensity for c:
For development of look-up tables (i.e., a horizontal LUT and a vertical LUT) in accordance with the present embodiment, the definition of the horizontal light modulation factor H (Δx,zj) is changed to H (Δx, zj,c) which is defined in equation 3:
and the definition of the vertical light modulation factor V (Δy,zj) is changed to V(Δy,zj,c), which is defined in equation 4:
In accordance with the present embodiment an off-line pre-computation step computes all possible H (Δx,zj,c) and V(Δy,zj,c) for all combinations of Δx,Δy,zj,c. During computation of holograms in accordance with the present embodiment, for each vertical line (xj,zj), which exists, aj,c (xj,yj,zj)≠0 (i.e. the line is not empty for color c), S(yh,c) is computed by equation 5:
S(yh,c)=Σj=0n-1aj,cV(Δy,zj,c) (5)
Then the vertical line (xj,zJ)'s contribution to the final hologram I(xh,yh)|(x
I(xh,yh)|(x
Finally, the final hologram I(xh, yh) is the sum of the contributions from all vertical line as shown in equation 7:
Calculation in accordance with the present embodiment, thus, only utilizes one-third of the computational requirements of the conventional holographic display methods such as TDM or SDM on different holograms.
Referring to
Two lenses 326, 328 and an aperture 330 cooperate to image the modified masked multiplexed light beam 324 through a beam splitter 332 onto an array of a spatial light modulator (SLM) 334, thereby providing portions of the modified masked multiplexed light beam 324 to portions of the array of the spatial light modulator 334. The spatial light modulator (SLM) 334 could, for example, be a SXGA-R3 SLM manufactured by Forth Dimension Displays of Fife, Scotland, UK including a SLM array with a 6400 pixel by 994 pixel array having a 13.86 μm pixel pitch.
The SLM 334 receives computer generated holographic (CGH) information 335 from a computational device 336 which could calculate the CGH information 335, store the CGH information 335, or both. For example, the computational device 336 could be a computer which determines modulation values for a plurality of pixels in a two-dimensional representation of a three-dimensional object in response to locations and color values of each of a plurality of points in the three-dimensional object and generates the computer generated holographic information 335 in response to the locations and color values for each of the plurality of points, the computational device 336 providing each of the modulation values as the computer generated holographic information 335 to corresponding pixels in the SLM 334 pixel array.
The SLM 334 varies modulation of the modified masked multiplexed light beam 324 shined thereon in response to the CGH information 335. In this manner, the SLM 334, utilizing lenses 338, 340 and an aperture 342, creates a color holographic display of an object 344 by reconstructing the object 344 in a reconstructed object space by diffracting portions of the multiplexed masked color light beam 324 at locations in the reconstructed object space determined in response to corresponding portions of the computer generated holographic information 335.
Those skilled in the art will understand that after the computer generated holographic information 335 is generated for a hologram, the computer generated holographic information 335 can be stored, loaded and finally launched to SLM 334 by the computational device 336 for the holographic reconstruction 344 of the three-dimensional object defined by the computer generated holographic information 335. In the SDM on single hologram (distributed sub-holograms) process in accordance with the present embodiment, each sub-hologram needs to be shined with reference light (i.e., a corresponding portion of the modified masked multiplexed light beam 324) which that sub-hologram is computed for. More specifically, pixel (xh,yh) on the hologram with a color value I(xh,yh) should be illuminated with light having wavelength λM(x
The three component light sources 302, 304, 306 (including a red light source 302, a green light source 304 and a blue light source 306) generate and radiate light beams for multiplexing in accordance with the present embodiment. The plurality of light masks 308, 310, 312 are coupled to respective ones of the component light sources 302, 304, 306 to generate corresponding modified masked light beams 314, 316, 318 as the light beams from the component light sources 302, 304, 306 pass through the light masks 308, 310, 312. Mirrors 402, 404 reflect the modified masked light beams 314, 316, 318 onto a beam combiner 406 to generate the modified masked multiplexed light beam 324. The beam combiner 406 is an X-beam splitter cube such as the cross dichroic prism X-cube beam splitter manufactured by Nitto Optical Company Ltd. of Tokyo, Japan. The beam combiner 406 advantageously allows the different color masks 308, 310, 312 to be easily adjusted in order to sharply image the modified masked multiplexed light beam 324 on the pixel array surface of the SLM 334 (
The masking of reference light to generate the modified masked multiplexed light beam 324 can be done either before or after the combining of the different color reference light beams. For masking before combination of the light beams, each color reference light beam from the component light sources 302, 304, 306 is masked by a black and white pattern made for that color (i.e., the masks 308, 310, 312). For masking after combination of the light beams, a band pass filter (alternatively shown in
With reference to
Referring to
While viewing angles resulting from conventional color multiplexing technologies are in a continuous range, the viewing angles resulting from the SDM on single hologram (distributed sub-holograms) process in accordance with the present embodiment is discontinuous. Gaps in the viewing angle range of the SDM on single hologram (distributed sub-holograms) process in accordance with the present embodiment are related to the figure size of the pixels in the M(xh,yh): i.e., the smaller the figure size, the smaller the gaps in the viewing angle range. If the figure size is small enough such that the gaps in the viewing angle range would be ignored by human eyes, the viewing angle range can be effectively considered as continuous. However, the figure size (which is related to the mask pattern size) cannot be too small and the mask pattern cannot be regular like lines, as both cases will cause deflection by the masks 308, 310, 312 on the reference beams 602, 604, 606 and make a resultant reconstruction unclear. In practice, the figure size is determined, e.g., by trial and error, to be the smallest figure size which can still be made and aligned. This results in a practical range of mask pattern sizes in the range from the size of the SLM pitch to a few millimeters. Referring to
After shining the modified masked multiplexed light beam 324 masked by M(xh,yh) to the CGH information 335 (
In the SDM on single hologram (distributed sub-holograms) process in accordance with the present embodiment, each component for each color will only affect one third of the final hologram. Thus, the total computational load will advantageously not increase as color is introduced in accordance with the present embodiment. Because only one hologram (i.e., only one hologram of CGH information 335) is used for reconstruction of all colors, the bandwidth and CGH information data storage requirement is the same as single color holography. Further, in the optical setup 300 (
After color multiplexing, reconstructions of different color components may not be matched together: aj,c at different positions in the reconstruction three-dimensional space, although all aj,c are computed by the same object coordinates xj,yj, zj in the CGH computation. This mismatch is primarily caused by small diffraction angle differences for different colors, imperfect optical elements and imperfections in other parameters of the optical setup. Color mixing needs to be performed to achieve good color reconstruction quality. The purpose of color mixing is to make color components from the same point of the three-dimensional object, as defined by the CGH information, to appear at the same point in the reconstructed object space.
Referring to
Therefore, as shown in the process 800, the modified masked multiplexed light beam 324 is shined on the pixel array of the SLM 334 after being reflected from a beam combiner 802. The modulated beam is then reflected back through the beam combiner 802 and focused and diffused by the lenses 338, 340 and the aperture 342 to project the reconstructed object 344 in the reconstructed object space. When CGH information corresponding to one or more reference objects 804 is utilized to create the object 344, the one of the one or more reference objects 804 is compared 806 to the reconstructed corresponding reference object 344 and object correction information is generated as horizontal, vertical and depth correction factors in response to object space correction of the color values for each of the plurality of points of the one or more reference images 804. The computational device 336 (
In order to get the correction parameters/values for object space correction, key depth layers are selected to do depth, column and row correction. The key depth layers could be front, back and center layers or could be a finer depth-division of a reference object. For each kind of correction, an initial correction value/parameter will be initially assigned. Thereafter, holograms of a test pattern are computed as a reference hologram with the correction value/parameter. This reference hologram is launched on the SLM 334 for reconstruction in space. After reconstruction, the correction value/parameter is updated by the difference (e.g., the computed difference 806) between the captured reconstruction 344 of the reference object and the desired outcome reconstruction 804 of the reference object. The test pattern in the CGH information is then computed with the updated correction value/parameter, and further tuning of the correction value/parameter continues until the reconstructed object 344 matches the targeted object 804, or until no further tuning in accordance with the parameters of the system 800 can be done. Column, row and depth corrections are done for selected key depth layers only (e.g., the first, center and the last depth layer). In this manner correction on other layers is automatically generated from correction on these selected key layers.
Exemplary reference images in accordance with the present embodiment are depicted in
Column, row and depth correction using the reference images 902, 904, 906 are performed at each selected key depth layer. Initially depth correction at the selected key depth layer is performed, and then column and row correction are performed for the selected key depth layer. The order of the column and row corrections is not important, and either can be performed first.
Depth correction obtains shift values of different colors in the z (depth) direction and causes different color components of selected object layer to appear at the same targeted depth plane during object reconstruction. Thus, the test pattern reference image 906 in accordance with the present embodiment is an object with multiple depth layers. For each iteration, the correction value of the most clear reconstructed layer on the targeted plane (i.e., the plane where the video capturing device is placed or focused upon) during object reconstruction is selected as the center for the next iteration, and smaller depth differences are used between each layer. The iterative computation stops when the center layer of the test pattern appears sharp within a targeted plane in the reconstructed object.
For column correction, the sample column correction pattern 902 contains vertical segments for different colors on the same line. Similarly, for row correction, the sample row correction pattern 904 contains horizontal segments for different colors on the same line. Depth, column and row correction are performed using the testing patterns 906, 902, 904 in order to obtain the correction values/parameters of depth levels, columns and rows, such as the scale and swift in the Z/depth, the scale and swift in the X/horizontal for the columns and the Y/vertical direction for the rows for the targeted depths. Distances between key columns, rows and depth patterns and positions of the first key columns, rows and depth layers are main parameters to be corrected; and positions of key columns, rows and depth layers need to be specified individually, if they do not appear at even distances.
Referring to
The correction process for each of the depth, column and row corrections 1006, 1008, 1010 are performed in accordance with the correction routine 1012. Correction is begun 1020 by assigning initial correction values 1022. The hologram CGH information for the correction pattern is then computed 1024 using the correction values. A color object of the computed correction pattern is reconstructed 1026 and the correction values are updated 1028 based upon the mismatch in the reconstructed correction pattern. If the target mismatch has not been met 1030, processing returns to compute 1024 the hologram CGH information for the correction pattern using the updated correction values and a color object of the computed correction pattern is reconstructed 1026 for updating 1028 the correction values based upon the mismatch in the reconstructed correction pattern. When the target mismatch is met 1030, the correction process is ended 1032 and processing returns to the next step (e.g., step 1008, step 1010 or step 1014).
The object space correction in accordance with the present embodiment is applicable for indirect display techniques like holography, as indirect displays can change output position in response to input parameters without any physical change on the actual display device. Treatment of the correction of all the parameters of the object data process and physical display device as correction of a black box of parameters greatly simplifies the color mixing process. In addition, the correction values are calculated once for each optical setup and stored in a look-up table thereby further simplifying the color mixing process. A comparison of the color mixing process in accordance with the present embodiment and the element-by-element object correction process is shown in Table 4:
The bandwidth and storage requirements of different color multiplexing methods for color holographic 3D display are analyzed in Table 2. It can be seen that SDM on single hologram (distributed sub-holograms) has low bandwidth and data storage requirements, the same as other SDM on single hologram methods.
Referring to
Referring to
Thus it can be seen that SDM on single hologram (distributed sub-holograms) achieves full color reconstruction space with low CGH computation load, low bandwidth/storage requirements, and simple optical setup design without synchronization. It also can be seen that object space correction in accordance with the present embodiment achieves good reconstruction quality with simple analysis steps and without increasing CGH computation load. While exemplary embodiments have been presented in the foregoing detailed description of the invention, it should be appreciated that a vast number of variations exist, including variations as to the materials and shapes used to form the optical setup 300.
It should further be appreciated that the exemplary embodiments are only examples, and are not intended to limit the scope, applicability, dimensions, or configuration of the invention in any way. Rather, the foregoing detailed description will provide those skilled in the art with a convenient road map for implementing an exemplary embodiment of the invention, it being understood that various changes may be made in the function and arrangement of elements and method of fabrication described in an exemplary embodiment without departing from the scope of the invention as set forth in the appended claims.