Due to the binocular nature of human vision, two eyes “see” from slightly different perspectives. Our brains produce a stereoscopic (“3D”) combination of these images with depth perception. Compared with a projection of a three-dimensional image onto a two-dimensional surface (“2D imaging”), 3D imaging encodes more object position information, particularly depth. Currently 3D imaging requires either two integrated and matched cameras for simultaneous 3D imaging, or repositioning of a single camera for sequential 3D image capture. One method is a dedicated 3D camera with dual lens and dual image sensors. Despite the possibly higher cost, the versatility of this method is low. Another current method for 3D imaging is to acquire two “sequential” images with the same camera, each image acquired at a slightly different perspective. The images are later combined into a 3D format through post-processing. Due to the minimal requirements on camera type and model, this method is particularly popular among smartphone users. However the intrinsic drawback of this method is that it only works with still subjects. Video is not possible, as the method does not work for moving subjects where the position changes drastically between the two independent shots. Problems of cost, design complexity, alignment and timing synchronization limit the utility of the present methods.
One example of the present invention is an attachment to an image-capturing device, including single shutter devices such as a camera, a laptop computer or a mobile phone, that enables such an image-capturing device to take 3D images and videos.
Accordingly, an example embodiment of the present invention can comprise a left-eye channel; a right-eye channel; and a beam combiner. The left-eye channel is configured to relay a left-eye beam to the beam combiner; the right-eye channel is configured to relay the right-eye beam to the beam combiner; and the beam combiner is configured to combine the left-eye beam with the right-eye beam and to form a single combined beam.
In another example, the present invention is a system for acquiring a stereoscopic image. The system can comprise an attachment for acquiring a stereoscopic image as described above and an image acquisition device.
In another example, the present invention is a kit comprising an attachment for acquiring a stereoscopic image, as described above, and a device for viewing a stereoscopic image.
In another example, the present invention is a kit comprising an attachment for acquiring a stereoscopic image, as described above, and a computer-readable media with computer code instructions stored thereon, the computer code instructions causing an image display apparatus to display the stereoscopic image.
In another embodiment, the present invention is a system for acquiring a stereoscopic image. The system comprises a detector having a frame rate; a left-eye channel; a right-eye channel; a beam controller, the left-eye channel configured to relay a left-eye beam to the detector, the right-eye channel configured to relay a right-eye beam to the detector. The beam controller is configured to alternatively transmit either the left-eye beam or the right-eye beam to the detector at a switching rate that is equal to or less than the frame rate of the detector.
In another example, the present invention is a method for acquiring a stereoscopic image. The method comprises acquiring a left-eye image, carried by a left-eye beam; acquiring a right-eye image, carried by a right-eye beam; and combining the left-eye image and the right-eye image to form a single combined image.
In another example, the present invention is a method for acquiring a stereoscopic image. The method comprises acquiring a left-eye image carried by a left-eye beam; acquiring a right-eye image, carried by a right-eye beam; and alternatively transmitting either the left-eye beam or the right-eye beam to a detector at a switching rate, the detector having a frame rate, the switching rate being equal to or less than the frame rate of the detector.
In another example, the present invention is an apparatus for acquiring a stereoscopic image. The apparatus comprises means for acquiring a left-eye image, carried by a left-eye beam; means for acquiring a right-eye image, carried by a right-eye beam; and means for combining the left-eye image and the right-eye image into a single combined image.
In another example, the present invention is an apparatus for acquiring a stereoscopic image. The apparatus comprises means for acquiring a left-eye image carried by a left-eye beam; means for acquiring a right-eye image, carried by a right-eye beam; and means for alternatively transmitting either the left-eye beam or the right-eye beam to a detector at a switching rate, the detector having a frame rate, the switching rate being equal to or less than the frame rate of the detector.
The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawings will be provided by the Office upon request and payment of the necessary fee.
The foregoing will be apparent from the following more particular description of example embodiments of the invention, as illustrated in the accompanying drawings in which like reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating embodiments of the present invention.
A description of example embodiments of the invention follows.
An example embodiment of the present invention is a device that utilizes color filters or polarization filters to generate two separate beams carrying two separate images of the same object taken from different perspectives. The two perspectives correspond to the human left and right eye perspectives. The two separate beams are directed into a single camera objective (a single shutter camera), resulting in a single image with 3D information encoded in the color or polarization. A significant advantage of such a design is that the same lens is used for each eye perspective, thus permitting to capture a stereoscopic image with a single-shutter image acquisition device. Another advantage is that the images are captured simultaneously and work for both static and moving subjects, allowing 3D video capture. The device can be an attachment to a conventional, i.e. single shutter image capturing device.
In the example embodiments in which the device of the present invention is an attachment, the device can further include a means for securing the attachment to an image acquisition device. Examples of such means include a clip, a screw and a bolt, a pin, or any other mechanical device suitable for securing two items together. One example of such means is a metallic platform that is affixed around the lens, such as a washer affixed to the image acquisition device by a magnet. Another example of such means is a “dovetail” attachment arrangement whereby the attachment can be slid in place. Another example of such means is a conventional threaded camera mount and a number of components that link this camera base to the attachment such that the attachment can be positioned in front of the objective lens of the camera.
In certain example embodiments, such means for securing permit rotation of the attachment relative to the image acquisition device. Such rotation can allow taking the stereoscopic images in either the “portrait” or a “landscape” configuration. It will be appreciated by a person of ordinary skill in the art that the methods described herein, in their reliance on a single objective (single shutter) image acquisition devices, advantageously permit acquiring either a portrait or a landscape images.
In further embodiments, the attachment described herein includes a mechanism that permits tilting the left and right eye angles so that they are parallel or crossed at an arbitrary or fixed position. This is important for the perspective in capturing the stereoscopic view.
In further embodiments, the attachment described herein includes a mechanism that permits moving the left-eye channel relative to the right-eye channel by an arbitrary distance. For example, the left-eye channel and right-eye channel can be separated by about 6 cm. In other examples, the channels can be moved apart beyond 6 cm, or moved closer than 6 cm to each other. Such movement can be accomplished by any mechanism known to a person of ordinary skill in the art, such as a threaded component. Another example embodiment of the present invention is a device that includes a single shutter image acquisition device. The device further includes means for generating two separate beams carrying two separate images of the same object taken from different perspectives. In one example embodiment, each of the two images is captured in a corresponding frame by the image acquisition device. In another example embodiment, both images are captured within a single frame of the image acquisition device. Described in detail below, means for generating two separate beams can include two additional shutters, movable mirrors that control which beam reaches the objective lens, two polarization filters in combination with a polarization control elements, such as a liquid crystal element or Pockels cells.
Additionally, such a single-shuter image acquisition device useful for practicing the present invention can include a photodetector (CCD or CMOS) that employs a polarization mosaic array described, for example, in U.S. Pat. No. 7,420,542, incorporated herein by reference in its entirety.
To understand the present invention in more detail, some basic knowledge on how digital cameras capture color images is beneficial.
An example embodiment of the present invention is a device for acquiring a stereoscopic image that can be an attachment to a single-shutter image acquisition device. The attachment comprises a left-eye channel, a right-eye channel, and a beam combiner. The left-eye channel is configured to relay a left-eye beam to the beam combiner, the right-eye channel is configured to relay the right-eye beam to the beam combiner. The beam combiner configured to combine the left-eye beam with the right-eye beam and to form a single combined beam. Such an embodiment is a device 200 illustrated in
Referring to
Referring to
As used herein, the term “anaglyph” refers to the stereoscopic effect achieved by means of encoding each eye's image using filters of different (usually chromatically opposite) colors. Anaglyph images contain two differently filtered colored images, one for each eye. When viewed through the “color coded” “anaglyph glasses,” each of the two images reaches one eye, revealing an integrated stereoscopic image. The visual cortex of the brain fuses this into perception of a three dimensional scene or composition.
In the example shown in
The single combined beam 220 is then directed through a Bayer's mask 222 onto a detector 224. As a result, a standard color camera sensor can be used to obtain a 3D image. The image captured by a camera in the example embodiment of
Device 100 shown in
A further example embodiment of the present invention is device 500, shown in
The mirror M3 (212) of the device 200 is replaced with a second dichroic filter D1 (532) that transmits the compensating beam 534 but reflects the colored beam 536.
Image post-processing, using methods well-known to one of ordinary skill in the art, can be performed either by the image acquisition device 550 or by a special or general purpose computer to render a color-compensated 3D image acquired using the device 500.
A further example embodiment of the present invention is the device 600 illustrated in
As used herein, the term “color separation element” means “an optical filter capable of separating a “white light” into its component colors, such as a “rainbow” filter. As used herein, the phrase “color component ray” means a ray having a maximum intensity at one specific value of wavelength.
Referring to
The device 600 can be used to capture color compensated 3D images without the use of a third, color-compensating channel as that of the device 500 of
A person of ordinary skill in the art of image acquisition will appreciate that while the foregoing discussion referred to a Bayer mask as a means of color location at the CCD/CMOS sensor, other suitable technologies can also be employed with the devices described herein. One example of a Bayer mask alternative suitable for practicing with the present invention is a Foveon X3CMOC sensor. A Foveon X3 sensor uses an array of photosites, each of which consists of three vertically stacked photodiodes, organized in a two-dimensional grid. Each of the three stacked photodiodes responds to different wavelengths of light, thus capturing one of the three component colors.
A further example embodiment of a device of the present invention is a device that relies on polarization, rather than color, to convey the depth-of-field information. In this embodiment, an image acquisition device to which the device of the invention attaches or of which the device of the invention is a part includes a polarization mosaic array mask such as the one described in U.S. Pat. No. 7,420,542, mentioned above. An example embodiment of such a device 900 is schematically depicted in
The device 950 includes a detecting module 914. The detecting module 914 includes Bayer mask (or a color array) 916, a polarization mosaic array 918, and a CCD/CMOC sensor 920.
An alternative example embodiment of the device 900 is a device 1100, schematically shown in
Although not shown in
As stated above, in one example embodiment, the present invention includes an image acquisition device having a frame rate. Using such an image acquisition device, each of the left-eye and right-eye images can be captured in its corresponding frame by the image acquisition device. In such an embodiment, the device employs a beam controller configured to alternatively transmit either the left-eye beam or the right-eye beam to the detector at a switching rate that is equal to or less than the frame rate of the detector. A beam controller can be implemented in a variety of ways known to a person of ordinary skill in the art.
One example embodiment of such a device is illustrated in
Another example embodiment of a device of the present invention is a device 1300 depicted in
The movable mirror 1306 can be actuated by any means known to a person of ordinary skill in the art, including a galvanometer or a solenoid. A proper alignment of the movable mirror 1306 with the optical axis of an objective lens of the image acquisition device can be ensured by mechanical stops that arrest the mirror's movement.
An example embodiment of the present invention is a device 1400 schematically depicted in
The device 1400 further includes a polarization selecting element 1414. The polarization selecting element 1414 can be any optical element known to transmit electromagnetic waves having a desirable polarization. Examples include liquid crystal-based polarizers, including spatial light modulators (SLMs), adjustable polarization filters, Pockels cells, or a mosaic polarization arrays described above with reference to
An image acquisition device 1450 can be a single-shutter device, and may be separate from the device 1400.
An example embodiment of the present invention is a device 1500 schematically depicted in
In the embodiments of the present invention described above that employ capturing each of the left-eye and right-eye image in its corresponding frame of the image acquisition device (i.e. the embodiments that employ a beam controller configured to alternatively transmit either the left-eye beam or the right-eye beam to the detector at a switching rate that is equal to or less than the frame rate of the detector), the device of the present invention can further include a module configured to synchronize the beam controller and the detector. Synchronization can be accomplished, for example, by using a voltage sine wave or square wave to switch shutters on or off or to switch, or gate, the detected beam. This could be phase-locked with a trigger signal coming from an output of the camera so that the frames were perfectly synchronized with the shutter or beam transmitting elements. Alternatively the element controlling the shutters of the device described herein could produce a pulse that would tell the camera to take two or more pictures in rapid succession.
The stereoscopic images acquired using the embodiments of the present invention that rely on the color to encode the left-eye or the right-eye perspective can be viewed using a device for viewing the stereoscopic image is selected from goggles, a lenticular array, or a parallax array.
The stereoscopic images acquired using the embodiments of the present invention that rely on the polarization to encode the left-eye or the right-eye perspective can be viewed using the device for viewing the stereoscopic image (goggles, a lenticular array, or a parallax array) having polarization filters.
Additionally, the stereoscopic images can be viewed on a display configured to reproduce stereoscopic images. For example, left and right images can be displayed on a 3D TV screen where a passive system displays the left image at one polarization and the right image at an orthogonal polarization. The images can be displayed simultaneously or sequentially. In another example of a viewing device, left and right images are displayed, and goggles are actively synchronized to block the left and right eyes so that only the correct image is on when its corresponding eye is in the open goggle state. Another type of viewing device includes a display with a lenticular array attachment that enables a different set of pixels to be visible from the left or right eye perspectives. Similarly, a parallax system also will permit pixels to be visible by either the left or right eyes. Another example of a viewing device employs a side by side viewing system and a viewer that relaxes the eyes to see the stereo image. In any of the above embodiments, the present invention can include computer code instructions causing an image display apparatus to display the stereoscopic image.
It should be understood that a camera device or other electronic device, such as a handheld device, computer, or server, capturing or storing the stereographic images may be equipped with control logic, such as in the form of hardware, firmware, or software, configured to arrange the stereographic images for playback via a visual display. The stereographic images may be stored sequentially in left/right/left/right/ . . . order, in left and right image files, or in some other arrangement and encoded in some manner known in the art, such as having metadata associated with each image to indicate a sequence number and left or right channel indicator corresponding with corresponding parameters during image capture operations. The same or different control logic can be used to cause a display to produce graphical representations of the stereographic images to a viewer. The display may be a 2D display that coordinates with passive or active 3D viewing glasses, or the display may be a display that displays 3D images without the viewer's need to wear 3D viewing glasses. In the case of software, the electronic device includes a non-transitory computer-readable medium and a processor configured to read processor instructions stored thereon. The instructions may be a sequence of instructions in any suitable software language capable of performing operations disclosed herein. Similarly, the processor may be any form of processor that, after loading the instructions, can perform the operations or subset of operations disclosed herein. The non-transitory computer-readable medium may be any computer memory device, such as RAM, ROM, CD-ROM, or other memory storage device known in the art. Typical computer buses, input/output ports, hardware interfaces, processor logic, peripheral controllers, display controllers, and so forth may be included in the electronic device or display device or other device configured to operate in cooperation therewith.
Accordingly, in certain example embodiments, the present invention can be a kit. For example, in one example, the present invention is a kit comprising an attachment for acquiring a stereoscopic image, as described above, and a device for viewing the stereoscopic image. In another example, the present invention is a kit comprising an attachment for acquiring a stereoscopic image, as described above, and a computer-readable media with computer code instructions stored thereon, the computer code instructions causing an image display apparatus to display the stereoscopic image.
While this invention has been particularly shown and described with references to example embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the invention encompassed by the appended claims.