1. Field of the Invention
The capture of images for full depth viewing can be done with multiple coordinated imaging devices, and most frequently with just two. Presently, and for the most part, these images are stored, manipulated and then re-created as multiple images for viewing on flat screens. To see the images in full-depth observers are obliged to wear switching, polarized or anaglyph glasses. With varying degrees of success this has been done for many years. What has been done with much more difficulty is to re-create the images without the use of glasses, and more difficult still to do this without converting from capture to display format in milliseconds, that is, in real time.
Within the field of seeing without glasses (auto-stereoscopy) a number of techniques have been used to re-create full-depth images from flat screens. The most successful of these has been lenticular arrays, in which each cylindrical lens creates multiple points of view by bending the light from several LEDs, giving in aggregate an observer a vivid sense of depth. This is especially true at “sweet spots”, where a confluence of beams arrive in close coincidence.
A different technique physically divides the light from the emitting elements into left and right with small strips, so that each eye sees just one half of the full perspective. These are called “parallax barriers”. They have long been used, but are generally limited to just two points of view. Efforts to make them work well include liquid crystals, active barriers, reversed barriers and multiple barriers.
Another promising approach is a rear projection system with multiple points of view, possibly hundreds. This replicates the way we see scenes with our eyes, taking in innumerable snapshots from different perspectives to create full depth panoramas in our visual cortex. This approach requires as many cameras and as many projectors as there are points of view to recreate. These cameras and projectors must all be carefully coordinated both in capture and display. Even small failures, such as those of intensity or color balance, in any camera or projector will leave streaks in the display. For so many imaging devices storage requires considerable memory and streaming substantial bandwidth. Nonetheless multiple projectors can produce full-depth and full-parallax images of great quality.
A further technique, still under development, uses tiny flipping solid-state mirrors to guide light through narrow-angle screens to create images of differing viewpoints. This technique, totally coordinated through software, may be very successful one day.
2. Description of the Related Art
Renaissance artists used tricks of light and perspective to create full-depth effects by focusing a viewer's interest on the mains subjects of their compositions. One example are paintings of Mary holding the baby Christ (e.g. Georges de la Tour, 1644). In this and similar pictures the baby Christ is brightly lit, attracting the one's attention to Him, with onlookers receding into a darkening background. Today, with high-speed computers, a viewer's attention can be convincingly redirected to different parts of an action in milliseconds, mimicking the action of our eyes, giving us a full-depth effect. This process is called foveation, since the attention of the eye is drawn to the action by its most sensitive element, the fovea, and all the rest is reduced to peripheral (or less noticed) vision.
For the past century cinematographers have also used the separation of colors, in their simplest division of blue and red, to redirect the different perspectives, for example blue to the left eye and red to the right. The viewer uses glasses typically called anaglyph, since the spectrum is carved away at its blue and red extremes to minimize color (and image) overlap. This inexpensive technique is still used, though most viewers find the color differences to their two eyes somewhat disconcerting.
Almost a century ago another technique was introduced called parallax barrier in which the “blue” and “red” (in this case different perspectives) were separated in viewers' eyes by the parallax (or viewing) angle. In the 1990s Sharp developed an electronic flat-panel application of this technology to commercialization, briefly selling laptops with the world's only 3D LCD screens. Parallax barrier screens are still used but appear dark and generally have a limited viewing angle.
A later technique, born in the 1980s, was to use a system of cylindrical lenslets slanting at an angle close to 33° to the screen vertical, and with a number of separate perspectives (typically from four to nine for each lenslet) to create a full-depth effect. This technology has been very successful in advertising and signage. The screens are very bright, and the images can appear to come straight out at a viewer to give the viewer a brilliant effect of a product. These screens work since viewers are typically at some distance from the screens (optimally at 4 meters) and do not see the low resolution near the screens, where pixels are used up by the multiple perspectives. A viewer also has to be optimally situated in angle (at one “sweet spot” of several) to see a screen in full depth.
Since the cylindrical lenslet approach has been successful, attempts have been made to “convert” (or write to) the screens in real time. This conversion in real-time has had to overcome internal software obstacles, and so far no-one has been very successful. To this day all moving signage images are created by programmers frame by frame on their computers. Consequently computer-generation (or CG) is a very lengthy and expensive process.
A typical glasses-free signage screen is optimally of a size between 24″ and 48″. Because of the particular alignment of the optics the screen cannot be manufactured easily either smaller or larger. The plastic lenslets are fragile and must be carefully wiped, if cleaned at all. To preserve optical alignments the system is massively designed and in consequence heavy. A typical cost is $10,000 each. These are all barriers to universal acceptance by consumers.
What is required for today's glasses-free viewing is a 2D TV or monitor screen which can be modified inexpensively and display full-depth images in real-time using simple, unconverted code. The screen must light in weight. In other words, we require a solution which can be universally accepted by viewing audiences in their own homes.
It is also highly desirable to give the consumer a screen modifier which is easy to attach to a normal TV or monitor so that anyone can enjoy full-depth viewing inexpensively.
Happily, one such solution has been actualized in the following invention.
By allowing light to be guided from a TV to an observer through a series of apertures to a condensing lens where it is bent to become parallel or just slightly divergent, a modifier to a flat (and nowadays curved) TV screen can be made to simulate full-depth images to our eyes without our needing glasses. Substantially all the light so flowing is captured. The appearance of a scene is wholly natural. With the arrival of 4K screens the images can also be made in high definition, or HD.
For the natural re-creation of a scene in full-depth, the alignment, spacing, shape of the apertures must be very precise, as must be the separation of the apertures from the emitting elements, and the separation of apertures from a condensing lens. As an added refinement for directing the light there may be a series of lenses. With these lenses the images can also be magnified to enhance the full-depth effect.
The shape of the apertures and their proximity to emitting elements (such as LEDs) is driven by the need to shape light beams and eliminate overlaps. For example, to steer the beams from two adjacent emitting elements the apertures will appear in cross-section as crosses. In fact in our case the light-beams themselves may cross each other. The corresponding walls will appear as diamonds, ovals, rectangles and (in its most simple utilitarian form) circles. In their correct diameters the circles can be cylindrical in the form of threads or wires. Cylinders and other shapes can be drawn, deposited and printed. Precise printing can be done with 3D printers.
For large TV screens a series of lenses for shaping and condensing light will for utility be Fresnel lenses. For compactness the condensing lenses will be of short focal length. For appearance the condensing lenses will have finely divided sub-lenses, or lenslets.
For steering and shaping light beams the Fresnel lenses will be linear. For condensing the Fresnel lenses will be concentric.
Within the present invention we would prefer, for accuracy, that our added elements for creating full-depth viewing be incorporated by a manufacturer. However, the elements we add can also be made attachable and detachable at a reasonable cost (that is, at much less cost than the purchase of a new viewing system). Together these elements form light-weight adjustable screen modifiers.
With foreknowledge of the geometry of any television or lap-top, a screen modifier can be made to retrofit any type of viewing device.
Further, when upgrades to existing technology are created old screen modifiers can be quickly and effectively switched out for new without the expense of buying new TVs.
The means of attachment combined with the ability to convert flat to full depth vision efficiently becomes a part of this invention. This applies particularly to newer 4K TVs or monitors of any size.
Especially with the increasing resolution of TV screens such as UHD, or 4K, the viewing between full-depth and flat can be switched back and forth without any physical alteration to a set-up with a single click of a mouse, yet still remain high-definition (HD).
The utility driving the TV screens, thus modified, will be our own software.
This invention, with its many further advantages noted below, may be best understood by referring to the following descriptions together with the accompanying drawings, in which like numbers refer to like elements, and in which:
To give a sense of scale
In
Also in
In our example from
In our example from
In our example from
Picking the light from LED 7 (which happens to be most usually the “red” LED whose light is destined for the right eye) the main body of light from LED 7 passes unobstructed through the aperture to squarely encounter the element 20 of the Fresnel lens 2. This element 20 is a conventional wedge (or prism) and in this example the light enters an angle of 15° to normal and exits at an angle of less than 1° (the wedge's refractive index of 1.5 giving it a wedge angle of 10°). This “main body of light” 12, as we refer to it, continues on to the right eye at this small angle of 1° to give a comfortable viewing distance of about five feet.
Conversely in
As may be seen, Fresnel lens 2 is not a normal lens but a series of linear and opposing wedges (prisms) with the function of directing the light from the LEDs to a comfortable viewing distance. The observer can then see the main body of light from two different perspectives as full-depth in a natural manner
The pixel image format for
LRLR
LRLR
LRLR
with the pixels (LEDs) 6, 8 etc. creating the left-eye images and the pixels (LEDs) 7, 9 etc. creating the right-eye images.
In
In the case of all shapes we take particular care to minimize the effects diffraction, dispersion and aliasing. We will discuss this later.
As a corollary to previous figures,
In a further example at 10′ one or more viewers will see the edges of the screen at angles of up to 11°. It is still worth putting a small bias into the outlying Fresnel wedges to steer the main body of light towards the viewers.
It may be noted in these computations that an observer may be fairly comfortable viewing from a number of positions, since the images will track over a substantial range. We have simply computed the above as being optimal for a particular instance.
In any event, the structure of this particular Fresnel lens will be an accommodation with the relative viewing positions of an expected audience.
In a further refinement, in
In
Returning now to
We note that the choice of cylindrical sections 3 in forming the apertures coupled with their distance from the Fresnel lens 2 creates an ability to block or reject the side bands from adjacent LEDs almost totally. In the case of LED 8 a main body of light emerges between wire sections 41 and 42 to strike Fresnel lens section 19 squarely. This will be refracted through a large angle (shown as arrow 17) to be emitted out of viewing range. There is also a penumbra associated with LED 8 which emerges from wedge 20 at an angle similar to that of LED 7. This can (for example) be minimized either by increasing the wire diameter one or two hundredths of an inch or by reducing t and s, or both. In this configuration it is the only instance of overlap by adjacent bands. In the case of light emitted from adjacent LED 9 it can be shown that the cylindrical sections 41 and 42 occlude the emerging light almost totally, leaving only small residual penumbras. The light from the following LEDs is totally occluded. This continues for every other LED in the array.
We therefore see that increased separation of the Fresnel lens 2, the apertures 3 and the LEDs 4 may be necessary for mechanical or other reasons but it is not helpful within the scope of this invention. However the reduction of these dimensions will greatly reduce the side-bands.
Though we have chosen cylindrical sections as examples, because they are simpler conceptually and the easier to manufacture, should the oval or elongated structures from
We have also worked with reducing the cylinders to flat sections. The main body of light from LEDs 6 and 7 will emerge normally, but the side-bands will have very little to block them. Light from will all LEDs will escape far to the sides at increasingly grazing incidences.
On all sections we have considered the effects of diffraction. When light from the LEDs hits a cylindrical section the ability to diffract is spread over the surface, so there will be less optical interference than from a flat section with sharp edges. Also at an average wavelength of visible light of 550 nm (or 0.55μ) and an aperture width of 0.025″ (or 635μ) the diffraction effects at this ratio (over 1100:1) are negligible. Even with a 4K screen and apertures of 0.012″ similar results (i.e. over 550:1) obtain to create very small diffraction effects.
In
Taken together the four periods 71, 72, 73 and 74 cover a complete cycle of 2p for every aperture of array 4.
Slightly harder to envisage or execute is a sinc function for all the possible apertures in this invention. For example, for cylindrical sections 93, 94 and 95 we have tried cotton thread with the correct diameter and consistency. This has given us not unreasonable results and may be a very inexpensive solution. (See below).
In terms of viewing pleasure the appearance of the screen (modified as described) is less granular than that of lenticular arrays such as those of other manufacturers.
In terms of accessing multi-view lenticular arrays in real-time, this is difficult because manufacturers have designed them for synthetic computer-generated inputs. It is easier to access an aperture system because with pixels pre-assigned for left and right views, there are no impediments to streaming data in milliseconds to create full depth.
If it is desired to switch to 2D from 3D or back again this can be done either with a mouse-click or with a remote control button, and the pixels can be immediately re-assigned to their original functions. In the case of a 4K or UHD format both will still be in HD.
In
In
The pixel image format for
LRLR
RLRL
LRLR
with the pixels (LEDs) 6, 8 etc. creating the left-eye images and the pixels (LEDs) 7, 9 etc. creating the right-eye images.
If one turns the screen “upside-down” the same will be true if all pixel assignments are flipped between 6, 8 (left) and 7, 9 (right), which can be triggered instantly by a gravity sensor. This simply reverses the pixel image format given above. With this design one can do what no other manufacturer has so far done: achieve viewing in full depth from every orientation.
This would seem particularly useful for full-depth viewing of images in smaller devices such as tablets and cell phones, since the apertures would be at an exceedingly small distance, e.g. 0.010″ away from the LEDs and a Fresnel lens, if needed, at 0.010″ again. This inexpensive system of apertures and lenses could be embedded directly by a device manufacturer.
For smaller devices such as tablets and cell phones the Fresnel lenses are not strictly necessary, although an embedded Fresnel lens magnifying up to 5× may be desirable to enhance the full-depth effect, or for improving the view for those with poor eyesight.
In
Our example in
LMRS
LMRS
LMRS
LMRS
with the pixel columns 6 (L) and 7 (M) creating images on the left, and the pixels (LEDs) 7 (R) and 9 (S) creating images on the right, in relation to aperture 37. This aperture a (37) has a width very close top to properly separate the emitted light into beams 61, 62, 63 and 64. The width of the bars d (38) is 3p, so that a+d=4p because if a is optimized to be wider or narrower a+d always adds up to the period of four pixels.
It is very easy to see from here that if the apertures are made to run at 45° to the screen vertical up to the left (as in
LMRS
SLMR
RSLM
MRSL
with the pixels assigned as in
The apertures and LED assignments could just as easily run up to the right, which would mean rotating the screen clockwise through the left lower quadrant to see full depth continuously. For all diagonal arrangements full depth should be visible almost semi-circularly about this quadrant without inverting the LED assignments.
From
The same general rules apply for three, five or more viewpoints. The aperture width always remains p. For example the pixel image format for three viewpoints running vertically is
LMR
LMR
LMR
with L being the left, M being the middle, and R being the right pixel.
All assignments and re-assignments of pixels 6, 7, 8 and 9 for any particular purpose are done by adding to or rewriting the internal TV screen or monitor software. Generally the access time for writing to the screen (i.e. sending data) or re-assigning the pixels is in milliseconds, typically 20 ms for local data at 1920p, slightly longer for remotely streaming data, depending on the packet sizes and the vagaries of the Internet.
We have carried out extensive work on the creation of mesh apertures and the means of mounting them.
One method tried has been to string a wire, such as a black anodized aluminum wire with a 0.025″ cross-section, over a frame vertically on a 0.050″ period. This involves the use of up to 2,000 feet of wire on 960 passes (1,920 for 4K) without kinks or breakages. The wires provide a very clean section. Clear nylon filament has also been tried with good results. However wires, filaments, twine, thread, etc. suspended like this cannot easily maintain even separations over any length above about ¼″.
A method to overcome this is to affix the wires directly against, or to embed them in, a flat plate. We have already done this by CO2 laser cutting slots directly into Plexiglas plates. Unfortunately the laser cuts into the intermediate clear sections irregularly, reducing their ability to transmit light cleanly.
A method to simulate affixing cylindrical sections to a flat plate is to use a 3D printer to extrude a 0.025″ filament (black, frosted or clear) which will stick on a clear glass or Plexiglas sheet. To keep it consistent in over 960 passes the bead must be monitored and controlled in process optically. We would prefer perfect accuracy but we have found that with a diameter of 0.025″ a tolerance such as ±0.002″ in size and position is possible and acceptable.
Another method of creating a wire mesh 50 is shown in
Our preferred section is cylindrical for implementing wire apertures. Sometimes there is a certain flattening of the section as it is deposited on a glass, plastic or other transparent substrates but not to any functional detriment. We can deposit other sections including half-cylinder, oval, polygons and flat in various orientations and dimensions. However, for ease of conceptualization, manufacturability and use, cylindrical sections appear to work the most easily.
It makes little apparent difference to the results whether the sections are frosted, clear, grey or black. Frosted or clear are more desirable because they are less conspicuous.
As seen in
It is preferable that a holding frame 121 (which follows below) or a sub-frame 109 (which follows later) are designed so that they do not mar the TVs which they are enabling, either in attachment or in detachment.
A simple means of attachment which allows for adjustment is shown in
In
In all cases the amount of adjustment required to bring apertures and LEDs into alignment is small, for a large (55″) screen in the order of 0.020″, well within the stretching capacity of the particular Velcro used.
Here, for reference, we use the coordinates x, y and z for the different axes required for aligning the screen modifier apertures 3 with Fresnel lenses 2 with the monitor LEDs 4. The monitor axes are x (seen normally as horizontal), y seen normally as vertical, and z away from the TV towards the viewer.
For precision alignment we need two frames: a sub-frame 109 for the x and y axes adjustments and a top frame 108 for the z-axis adjustment. Sub-frame 109 is made with aluminum box tubing and top frame 108 made with aluminum angles for the combination to be adjustable on three axes, plus light, stiff and strong. These frames together are “light-weight structural elements”.
In
The thumb screws 105 (which can be set-screws) are set in four places 110 on the vertical edges and with a small differential adjustment (e.g. ±0.005″) can also do the vertical (or skew) alignment of the wires on sheet 100 with the LEDs 4 to bring them into precise vertical registration. Within one or two iterations (after some adjustments with top frame 108) the four thumb screws 105 can lock the sub-frame 109 into place. We note that springs 103 only, without thumb screws, are required top and bottom since generally no up or down alignment is required.
Top frame 108, which holds the aperture sheet 100 and the protective glass cover 111, is designed to snap over the sub frame 109 in such a manner that it is adjustable on the z-axis. This adjustment is achieved with thumb screw 106 which can raise or lower top frame 108. When this is adjusted perfectly, the frame can be locked with side screw 107.
To remove and replace holding frame 108 we can either loosen or remove the four screws 107, or if it is desired to return the TV 110 to its original condition, remove the sub-frame 109 entirely by loosening the thumb screws 105 and sliding off the pads or magnets.
Ultimately it will be better—certainly more convenient to customers—if all screen modifiers 100 are built into TVs 110 as original equipment so that all adjustments are pre-set and no external adjustments are required.
While the invention has been described and illustrated (in general) as one in which arrays of apertures may be combined with Fresnel lenses to separate left and right perspective views in order to create full-depth vision, to those skilled in the art it will be clear that other derivations of this technology are possible. These derivations include (but are not limited to): other separations of the elements 1, 2, 3, 4 and 5; differing angles across screen 4 for apertures 3; differing periods for 2 and 3; differing configurations and materials of the elements of aperture array 3; differing focal lengths and distances to single or multiple observers; differing manners, means and materials for attaching, adjusting, detaching and replacing screens containing elements 2, 3 and 5.
It may be understood that although specific terms are employed, they are used in a generic and descriptive sense and must not be construed as limiting. The scope of the invention is set out in the appended claims.