Three-dimensional displays have been proposed for a number of applications, however still further improvements in three-dimension displays and their use may be had.
The present disclosure is generally related to three-dimensional displays and methods for displaying three-dimensional images. Some embodiments may be related to zooming for 3D image capture and display in real-time.
In some aspects, a method of projecting a first three-dimensional image for viewing from a first reference point may be provided. The method may include capturing, at a first input, first input rays associated with an object. Each captured first input ray may have an associated position and angle at the first input. The method may further include determining, based on the positions and angles of the captured first input rays, first propagated output ray positions and angles for output at an output that is spaced a first distance from the first input. The first propagated output ray positions and angles may be propagated a non-zero propagation distance relative to the captured first input rays that is different from the first distance. The first propagated output rays may be outputted at the output to produce the first three-dimensional image of the object that appears, from the first reference point, to be shifted a second distance equal to the first distance minus the non-zero propagation distance.
In some embodiments, the non-zero propagation distance may be less than zero. The first propagated output rays may be outputted to produce a zoomed-in three-dimensional image of the object when viewed from the first reference point.
In some embodiments, the non-zero propagation distance may be greater than zero. The first propagated output rays may be outputted to produce a zoomed-out three-dimensional image of the object when viewed from the first reference point.
Optionally, first propagated output ray positions and angles may be determined by tracing first input rays backward to a faux input positioned the non-zero propagation distance from the first input in a direction of an object space.
In some embodiments, first propagated output ray positions and angles may be determined by tracing first input rays forward to a faux input positioned the non-zero propagation distance from the first input in a direction of an image space.
In certain embodiments, an array of image capture devices define the first input. The array of image capture devices may be further coupled with an inlet lenslet array.
Optionally, an array of display elements may be coupled with a lenslet array to define the output.
The method may further include projecting a second three-dimensional image for viewing from a second reference point at a different angle from the output than the first reference point by capturing, at a second input, second input rays associated with the object. Each captured second input ray may have an associated position and angle at the second input. The method may further include determining, based on the positions and angles of the captured second input rays, second propagated output ray positions and angles for output at the output that is spaced a third distance from the second input. The second propagated output ray positions and angles may be propagated a second non-zero propagation distance relative to the second captured input rays that is different from the third distance. The second propagated output rays may be outputted concurrently with outputting the first propagated output rays at the output to produce the second three-dimensional image of the object that appears, from the second reference point, to be shifted a fourth distance equal to the third distance minus the second non-zero propagation distance.
The method may further include capturing, at the first input, second input rays associated with the object after the object moved relative to the first input. Each captured second input ray may have an associated position and angle at the first input. Thereafter, the method may include redetermining, based on the positions and angles of the captured second input rays, second propagated output ray positions and angles for output at the output. The second propagated output ray positions and angles may be propagated the non-zero propagation distance relative to the captured second input rays. The second propagated output rays may be outputted at the output to produce an updated three-dimensional image of the object that appears, from the first reference point, to be shifted the second distance equal to the first distance minus the non-zero propagation distance.
In further aspects, a method of projecting a three-dimensional image for viewing from a reference point may be provided. The method may include receiving input rays associated with an object. Each received input ray may have a position and angle associated with an input. The method may further include determining, based on the positions and angles of the received input rays, a series of propagated output ray positions and angles for output at an output. The series of propagated output ray positions and angles may be propagated with increasing or decreasing propagation distance increments relative to the received input rays. The series of propagated output rays may be output at the output to produce the three-dimensional image of the object that appears, from the reference point, to be moved toward or away from the reference point.
In some embodiments, the series of propagated output ray positions and angles may be determined with increasing propagation distances relative to the received input rays. The series of propagated output rays may be outputted at the output to produce the three-dimensional image of the object that appears, from the reference point, to be moved away from the reference point.
In some embodiments, the series of propagated output ray positions and angles may be determined with decreasing propagation distances relative to the received input rays. The series of propagated output rays may be outputted at the output to produce the three-dimensional image of the object that appears, from the reference point, to be moved toward the reference point.
Optionally, the method may include capturing the input rays associated with the object at the input. An array of image capture devices may define the input in certain embodiments.
In still further aspects, a three-dimensional display may be provided that includes a processor coupled with a display. The processor may be configured to: (1) receive input rays associated with an object, each received input ray having a position and angle associated with an input; (2) determine, based on the positions and angles of the received input rays, a series of propagated output ray positions and angles for output at the display, the series of propagated output ray positions and angles being propagated with increasing or decreasing propagation distance increments relative to the received input rays; and (3) outputting the series of propagated output rays at the display to produce the three-dimensional image of the object that appears, from the reference point, to be moved toward or away from the reference point.
The processor may be configured to determine the series of propagated output ray positions and angles with increasing propagation distances relative to the received input rays. The series of propagated output rays may be outputted at the display to produce the three-dimensional image of the object that appears, from the reference point, to be moved away from the reference point.
The processor may be configured to determine the series of propagated output ray positions and angles with decreasing propagation distances relative to the received input rays. The series of propagated output rays may be outputted at the display to produce the three-dimensional image of the object that appears, from the reference point, to be moved toward the reference point.
The three-dimensional display may include an input for capturing the input rays associated with the object. An array of image capture devices may define the input.
In still further aspects, a method of displaying an image may be provided that includes collecting input rays at an input plane. The collected input rays may include a first set of rays associated with a first portion of a target object and a second set of rays associated with a second object disposed between the target object and the input plane. The second object may be obstructing rays from a second portion of the target object from the input plane. The method may include determining a position of the second object by back-tracing collected input rays received at the input plane. The first set of rays may be differentiated from the second set of rays based on the position of the second object determined by back-tracing the collected input rays received at the input plane. The second portion of the target object may be reconstructed by estimating a third set of rays associated with the second portion of the target object based on the first set of rays differentiated from the second set of rays and associated with the first portion of the target object. Thereafter, an image of the first portion of the target object and the reconstructed second portion of the object may be outputted at an output (e.g., a display or the like). Embodiments of the present disclosure may further include a system, with appropriate hardware and/or software (e.g., processors, detectors, lenslet arrays, displays, etc.), for carrying out the method generally described above.
In some embodiments, the input plane may include a lenslet array with a plurality of detectors.
Optionally, determining the position of the second object by back-tracing collected input rays received at the input plane may include determining a boundary of the second object.
In some embodiments, determining the position of the second object by back-tracing collected input rays received at the input plane may include determining a contour of the second object.
In some embodiments, determining the position of the second object by back-tracing collected input rays received at the input plane may include differentiating the second object from the target object based on a change in contrast.
In some embodiments, determining the position of the second object by back-tracing collected input rays received at the input plane may include differentiating the second object from the target object based on a change in color.
Optionally, the method may include determining a position of the first object by back-tracing collected input rays received at the input plane.
The image of the first portion of the target object and the reconstructed second portion of the object may be outputted by outputting the image without an image of the second object.
Reconstructing the second portion of the target object may include estimating a position and direction of rays that would be collected at the input plane but for the obstruction by the second object.
The second portion of the target object may be reconstructed based on a color or a shape of the first portion of the target object.
Embodiments of the disclosure covered by this patent are defined by the claims below, not this summary. This summary is a high-level overview of various aspects of the disclosure and introduces some concepts that are further described in the Detailed Description section below. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used in isolation to determine the scope of the claimed subject matter. The subject matter should be understood by reference to appropriate portions of the entire specification of this patent, any or all drawings, and each claim.
The disclosure will be better understood upon reading the following description and examining the figures which accompany it. These figures are provided by way of illustration only and are in no way limiting on the disclosure.
Further details, aspects, and embodiments of the disclosure will be described by way of example only and with reference to the drawings. In the drawings, like reference numbers may be used to identify like or functionally similar elements. Elements in the figures may be illustrated for simplicity and clarity and have not necessarily been drawn to scale.
3D Display Ray Principles
Some optical principles for three-dimensional (3D) image capture and display are discussed first. These can be implemented using various technologies, including digital technology, whether in real-time or not. For simplicity, it is assumed that the magnification is unity, though this and other effects can be added for various visual effects for the final image(s).
Here ni, nm, nf are the indices of refraction, before the input surface, between the input and output, and after the output surface, respectively. L is the propagation distance, for the specific ray that was propagated. For arbitrarily shaped input and output surfaces, Equation (1) can be applied for each plane containing and rotated about z.
For simplicity, it may be assumed that ni=nm=nf=1 and that each input and output surface are two-dimensional (2D) planes perpendicular to the z-axis, though the formalism is not limited to these cases. Some, if not most, three-dimensional (3D) displays that do not change the appearance of objects, can be described as an output of rays that have propagated through space, hence by Equation (1).
The device length (Ld) is defined to be the distance from the input plane to the output plane. L is the propagation distance, or rather an “effective” propagation distance when it is different from Ld. When the two are the same, then the system describes a cloaking system, as illustrated in
When L>0 for Equation (1), rays may be traced forward as if the image capture device (input plane) was placed at a distance |L| in the direction of the observer (image space) (at the “faux” input plane in
Accordingly, in some embodiments of the invention, a three-dimensional display may be provided to project a zoomed-in or zoomed-out image of the object associated with the input rays. In some embodiments, the display may provide a first image of the object and may thereafter output a series of images that transitions the first image of the object toward a more zoomed-in or more zoomed-out image of the object. Optionally, the first image of the object may be a zoomed-in or zoomed-out image of the object. In some embodiments, the first image of the object may be an identity transform of the input rays associated with the object.
In some embodiments, the display may be a real-time display where the output is recalculated by changes in the input. In certain embodiments, display may not be real-time and the input rays associated with the object may be prerecorded and transmitted to or accessed by the display when output of an image of the object is desired.
3D Display Implementations
Physically realizing the 3D ray propagation principles laid out can be done by using an array of image capture devices/detectors for the input plane and an array of display elements for the output plane. For example, an array of cameras on a 2D plane can be used for the input plane, and the output can be a combination of display screen and lenticular lenslet arrays or fly's eye lenslet arrays. Equation (1) may be used to determine the proper ray propagation between these two input and output planes, or even for arbitrarily shaped input and output surfaces. By adjusting L, recalculating, and reprocessing the output using Equation (1), the object(s) can appear to be moved along the longitudinal direction z, provided enough information was captured by the original input plane.
The detectors may be connected to receive images in real-time, and process them quickly with a computer/processor, to output on the display with delays that are not noticeable. One method to do this could be to connect interne protocol cameras, and connect them through routers and switches, or use some other methods to connect the detectors. Then, one or more camera(s)/detector(s) may be accessed at a time to process the image(s), thus requiring less bandwidth for the connections and less computational power than simultaneously accessing all cameras/detectors.
In some real-time embodiments, the input plane may be associated a first device and the output plane may be associated with a second device that is different from the first device that is communicatively coupled with the first device (e.g., tethered, wireless, etc.). The first and second devices may be mobile phones, tablets, laptops, or the like. Optionally, the first device may act as the input plane for the display on the second device and the second device may act as the input plane for display on the first device. Alternatively, one device may be a portable electronic device that receives the input rays associated with an object, and the other may be a stand-alone display that is configured to calculate the series of propagated output ray positions and angles for output at the display. The propagated output ray positions and angles may be propagated with increasing or decreasing propagation distance increments relative to the received input rays.
While embodiments described herein are generally discussed for real-time capture and output for three-dimensional displays, it should be understood that in some embodiments non-real-time three-dimensional display may be performed. For example, the input rays may be associated with an object may be stored on a computer readable media. A processor may access or otherwise receive the input rays from the computer readable media and may process the input rays to calculate the series of propagated output ray positions and angles for output at the display
A simplified method to capture images for 3D is shown in
3D Display Beyond Opaque Objects
Here a method to “see through” opaque objects or simulate such viewing is described. Suppose rays are collected at the input plane and the rays are propagated backward (L<0 in Equation (1)) beyond an opaque object.
We suggest an example method for reconstructing object 1 rays. First, we must determine the z position of object 2, which can be done by back-tracing the rays that arrive at the input plane. For example, the two rays in
One or more computing devices may be adapted to provide desired functionality by accessing software instructions rendered in a computer-readable form. When software is used, any suitable programming, scripting, or other type of language or combinations of languages may be used to implement the teachings contained herein. However, software need not be used exclusively, or at all. For example, some embodiments of the methods and systems set forth herein may also be implemented by hard-wired logic or other circuitry, including but not limited to application-specific circuits. Combinations of computer-executed software and hard-wired logic or other circuitry may be suitable as well.
Embodiments of the methods disclosed herein may be executed by one or more suitable computing devices. Such system(s) may comprise one or more computing devices adapted to perform one or more embodiments of the methods disclosed herein. As noted above, such devices may access one or more computer -readable media that embody computer-readable instructions which, when executed by at least one computer, cause the at least one computer to implement one or more embodiments of the methods of the present subject matter. Additionally or alternatively, the computing device(s) may comprise circuitry that renders the device(s) operative to implement one or more of the methods of the present subject matter.
Any suitable computer-readable medium or media may be used to implement or practice the presently-disclosed subject matter, including but not limited to, diskettes, drives, and other magnetic-based storage media, optical storage media, including disks (e.g., CD-ROMS, DVD-ROMS, variants thereof, etc.), flash, RAM, ROM, and other memory devices, and the like.
The subject matter of embodiments of the present invention is described here with specificity, but this description is not necessarily intended to limit the scope of the claims. The claimed subject matter may be embodied in other ways, may include different elements or steps, and may be used in conjunction with other existing or future technologies. This description should not be interpreted as implying any particular order or arrangement among or between various steps or elements except when the order of individual steps or arrangement of elements is explicitly described.
Different arrangements of the components depicted in the drawings or described above, as well as components and steps not shown or described are possible. Similarly, some features and sub-combinations are useful and may be employed without reference to other features and sub-combinations. Embodiments of the invention have been described for illustrative and not restrictive purposes, and alternative embodiments will become apparent to readers of this patent. Accordingly, the present invention is not limited to the embodiments described above or depicted in the drawings, and various embodiments and modifications may be made without departing from the scope of the claims below.
This patent application claims the benefit of U.S. Provisional Application No. 62/378,016, filed Aug. 22, 2016, the subject matter of which is incorporated in its entirety by this reference.
Number | Date | Country | |
---|---|---|---|
62378016 | Aug 2016 | US |