Patent Grant
10012831
The present invention relates generally to optical scanning, and particularly to methods and devices for monitoring timing and other parameters of a scanned optical beam.
Optical scanners are used in a wide range of applications. Some scanners use a rotating mirror to scan a beam.
For example, PCT International Publication WO 2012/020380, whose disclosure is incorporated herein by reference, describes apparatus for mapping, which includes an illumination module. This module includes a radiation source, which is configured to emit a beam of radiation, and a scanner, which is configured to receive and scan the beam over a selected angular range. Illumination optics are configured to project the scanned beam so as to create a pattern of spots extending over a region of interest. An imaging module is configured to capture an image of the pattern that is projected onto an object in the region of interest. A processor is configured to process the image in order to construct a three-dimensional (3D) map of the object.
In one of the embodiments described in this PCT publication, an illumination module comprises one or more beam sensors, such as photodiodes, which are coupled to a processor. These sensors are positioned at a selected angle or angles within the angular range that is scanned by a mirror so as to receive the scanned beam periodically and thus verify that the scanner is operating.
U.S. Patent Application Publication 2013/0207970, whose disclosure is incorporated herein by reference, describes depth engines that generate 3D mapping data by measuring the time of flight (TOF) of a scanning beam. A light transmitter, such as a laser, directs short pulses of light toward a scanning mirror, which scans the light beam over a scene of interest. The scanner may comprise a micromirror produced using microelectromechanical system (MEMS) technology. A receiver, such as a sensitive, high-speed photodiode (for example, an avalanche photodiode) receives light returned from the scene via the same scanning mirror. Processing circuitry measures the time delay between the transmitted and received light pulses at each point in the scan. This delay is indicative of the distance traveled by the light beam, and hence of the depth of the object at the point. The processing circuitry uses the depth data thus extracted in producing a 3D map of the scene.
U.S. Patent Application Publication 2013/0250387, whose disclosure is incorporated herein by reference, describes scanning apparatus that includes a transmitter, which is configured to emit a beam comprising pulses of light, and a scanning mirror, which is configured to scan the beam over a scene. A receiver is configured to receive the light reflected from the scene and to generate an output indicative of the pulses returned from the scene. A grating is formed on an optical surface in the apparatus and is configured to diffract a portion of the beam at a predetermined angle, so as to cause the diffracted portion to be returned from the scanning mirror to the receiver. A controller is coupled to process the output of the receiver so as to detect the diffracted portion and to monitor a scan of the mirror responsively thereto.
Embodiments of the present invention that are described hereinbelow provide apparatus and techniques for monitoring the operation of an optical scanner.
There is therefore provided, in accordance with an embodiment of the invention, scanning apparatus, including a transmitter, which is configured to emit a beam including pulses of light, and a scanner, which is configured to scan the beam along a scan axis over a specified angular range. A scattering line extends across a path of the scanned beam. A receiver is configured to receive the light scattered from the scattering line and to generate an output indicative of an intensity of the scattered light. A controller is coupled to process the output of the receiver so as to monitor operation of the scanner.
In one embodiment, the scattering line includes a filament mounted so as to extend across the path of the scanned beam. In another embodiment, the apparatus includes a transparent optical element positioned in the path of the scanned beam, wherein the scattering line is scribed across the transparent optical element.
In some embodiments, the receiver is further configured to receive the light reflected from a scene beyond the scattering line within the specified angular range, wherein the output is further indicative of the pulses returned from the scene. In a disclosed embodiment, the controller is configured to process the output of the receiver so as to generate a three-dimensional map of the scene based on the returned pulses.
Typically, the controller is configured to calculate one or more operating parameters of the scanner responsively to the output of the receiver. In some embodiments, the scanner is configured to scan the beam in mutually-opposing first and second scan directions in alternation along the scan axis, and the controller is configured to calculate a scan offset between the first and second scan directions based on the output of the receiver.
Additionally or alternatively, the scanner is configured to scan the beam in a periodic scan pattern, and the one or more operating parameters calculated by the controller include at least one of a frequency, a phase, and an amplitude of the periodic scan pattern.
In one embodiment, the scattering line is one of at least first and second scattering lines, which extend at different, respective locations across the path of the scanned beam, and the output of the receiver includes first and second outputs that are indicative of the intensity of the light scattered from the first and second scattering lines, respectively. The controller is configured to calculate the one or more operating parameters responsively to a relation between the first and second outputs.
There is also provided, in accordance with an embodiment of the invention, a method for scanning, which includes scanning a beam including pulses of light along a scan axis over a specified angular range. A scattering line is positioned across a path of the scanned beam. The light scattered from the scattering line is received, and an output indicative of an intensity of the scattered light is generated and processed so as to monitor the scanning.
The present invention will be more fully understood from the following detailed description of the embodiments thereof, taken together with the drawings in which:
Accurate TOF-based depth mapping requires that the position of the scanning spot be precisely known at each point at which a depth measurement is made along the scan path. MEMS-based scanning mirrors in particular, although compact and low in cost, do not always provide the desired level of accuracy. Parameters such as the scan frequency, phase, amplitude, and zero-offset may vary substantially relative to desired baseline values. As a specific example, when the mirror is driven in a bi-directional raster scan (in which odd-numbered lines are scanned in one direction and even-numbered lines in the opposite direction), offset between the odd and even lines along the scan direction may vary. Unless corrected—either by closed-loop control of the scanner or by digital adjustment of the measured TOF values—inaccuracies of the scan parameters can degrade the quality of the depth map.
Embodiments of the present invention that are described herein provide techniques for monitoring scanner operation that are simple, effective and inexpensive to implement. The term “monitoring” is used in the present description and in the claims in the broad sense of measuring scanning parameters, such as scan angle as a function of time, as well as detecting deviation of the scanning parameters from certain ideal or baseline values.
In the disclosed embodiments, a transmitter emits a beam comprising pulses of light, and a scanner, such as a rotating mirror, scans the beam along a scan axis over a specified angular range. In some embodiments, as described further hereinbelow, this scan axis is one of two (or more) axes along which the beam is scanned. For purposes of monitoring the scan, a scattering line is extended across the path of the scanned beam. The scattering line may comprise, for example, a filament, such as a wire, mounted so as to extend across the path of the scanned beam, or a line that is scribed across a transparent optical element that is positioned in the path of the scanned beam.
A receiver receives the light scattered from the scattering line and generates an output that is indicative of the scattered light intensity as a function of the scan angle, i.e., the angle of deflection of the scanner (as determined, for example, by the angle of rotation of the rotating mirror). A controller processes this output in order to monitor operation of the scanner. The receiver may be dedicated to the purpose of such monitoring. Alternatively, it can be advantageous to use this same receiver to receive the light from the scanned beam that is reflected from a scene beyond the scattering line, such that the receiver output is further indicative of the pulses returned from the scene. In this latter case, the controller may process the output of the receiver, for example, so as to generate a 3D map of the scene based on the returned pulses.
In some embodiments of the present invention, the controller calculates one or more operating parameters of the scanner using the output of the receiver. For example, when the scanner scans the beam in mutually-opposing scan directions in alternation along the scan axis, as described above, and there is a scan offset between the two directions, the controller can estimate and compensate for this offset based on the difference in the output of the receiver between the directions. Additionally or alternatively, the controller can process the receiver output in order to extract the frequency, phase, and/or amplitude of a periodic scan pattern. It can be useful, particularly for this latter sort of calculation, to extend two (or more) parallel scattering lines at different, respective locations across the path of the scanned beam. In this case, the receiver will provide multiple outputs, in the form of two (or more) scattering peaks that are indicative of the intensity of the light scattered from the different scattering lines, and the controller uses the relation between these outputs in calculating the operating parameters of the scanner.
Although the embodiments that are shown in the figures and described in detail herein relate to a particular scanner design for use in 3D mapping, the principles of the present invention are equally applicable, mutatis mutandis, to other sorts of optical beam scanners, both for 3D mapping and for other applications. Such scanners may use not only MEMS mirrors to scan the transmitted beam, but also scanning mechanisms of other sorts, including, but not limited to, mechanically-rotating mirrors and acousto-optical and electro-optical deflectors.
The receiver that senses the scattered optical pulses may have a detection area that is scanned, using either the same scanning mechanism as the transmitted beam or its own scanning mechanism, as in the embodiments shown in the figures. Alternatively, the receiver may comprise an image sensor or other detector with a fixed field of view. The approach adopted in the pictured embodiments, in which the same receiver is used for both depth mapping and scan monitoring, is advantageous, inter alia, in that it can make use of existing components to perform the scan monitoring functions, and requires essentially no additional hardware components. In alternative embodiments, however, a dedicated receiver may be used for scan monitoring.
All of the above alternative implementations are considered to be within the scope of the present invention.
Reference is now made to
Scanning device 20 is built around a scanner 22, comprising an adjacent transmit mirror 24 and receive mirror 26, which are mounted together within a gimbal 28. Alternatively, two or more receive mirrors may be mounted side-by-side in gimbal 28, parallel to transmit mirror 24, or the same mirror may be used for both the transmitted and received beams (as in the above-mentioned U.S. Patent Application Publication 2013/0207970). Scanner 22 may be made from a substrate, such as a semiconductor wafer, which is etched to define base 32, gimbal 28, and transmit and receive mirrors 24, 26 in a MEMS process. (A reflective coating is deposited on the mirrors as a part of the process.) Alternatively, other mirror configurations may be used, as will be apparent to those skilled in the art.
Mirrors 24 and 26 rotate about respective hinges 30 relative to gimbal 28, while gimbal 28 rotates about hinges 34 relative to a base 32. Hinges 30 (and hence the axes of rotation of mirrors 24 and 26) are parallel to one another, along the X-axis in the figures, so that the rotation of mirror 24 will scan a reflected beam along a scan axis that is parallel to the Y-axis. Hinges 34 are oriented so that the axis of rotation of gimbal 28, shown as being oriented along the Y-axis, is perpendicular to the mirror axes. Gimbal 28 and mirrors 24 and 26 may be driven to rotate about their respective axes by any suitable sort of drive, such as the magnetic drives described in the references cited above in the Background section, or other types of magnetic and electrical scanner drives that are known in the art.
The dimensions and masses of transmit and receive mirrors 24 and 26 and hinges 30 may desirably be chosen so that the mirrors rotate about their respective hinges 30 by oscillation at or near a resonant frequency. On the other hand, gimbal 28 may be driven to rotate relative to base 32 in a non-resonant mode, typically at a frequency substantially lower than the resonant frequency of mirrors 24 and 26. The fast rotation of mirrors 24 and 26 about the X-axis and the slower rotation of gimbal 28 about the Y-axis may be coordinated so as to define a raster scan of the transmitted and received beams over an area of interest. Alternatively, the rotations of mirrors 24, 26 and gimbal 28 may be controlled to generate scan patterns of other sorts.
A transmitter 36 emits pulses of light, which are collimated by a collimating lens 38 and directed by a reflecting area 46 of an output plate 40 toward transmit mirror 24. The transmit mirror scans the beam toward a scene via a transmitting area 48 of plate 40, which is transparent to the light emitted by the transmitter. (The term “light,” in the context of the present description and in the claims, refers to optical radiation of any wavelength, including visible, infrared, and ultraviolet radiation.) Light reflected back from the scene and through transmitting area 48 is directed by receive mirror 26 toward reflective area 46, and from area 46 to a collection lens 42, which focuses the reflected light onto a receiver 44. In alternative optical layouts (not shown in the figures), light reflected back from the scene may be directed by receive mirror 26 toward a collection lens, without reflection from reflective area 46. Additionally or alternatively, reflective area 46 may be eliminated from the transmit path, as well.
Receiver 44 typically comprises a high-speed optoelectronic detector, which generates an output indicative of the time of flight of the pulses to and from points in the scene. In one embodiment, transmitter comprises a pulsed laser diode, while receiver 44 comprises an avalanche photodiode, but any other suitable sorts of emitting and sensing components may alternatively be used in device 20. Scanner 22 scans the transmitted and received beams of light together over a predefined angular range, so that at each point in the scan, receiver 44 receives light from the same area of the scene that is illuminated at that point by transmitter 36.
For purposes of monitoring the scan parameters, a scattering line in the form of a filament 50 is mounted on or adjacent to the surface of transmitting area 48, and thus extends across the path of the scanned, transmitted beam. Filament 50 is oriented perpendicular to the “fast” scan axis of the beam reflected from mirror 24, i.e., perpendicular to the Y direction. In one embodiment, filament 50 comprises a titanium wire, having a diameter of about 100 μm and a matte surface finish, to reduce specular reflections. The diameter of the transmitted beam that is incident on area 48 is typically about 1.5 mm, and several successive pulses from transmitter 36 typically strike filament 50 at different angles in each scan line. Alternatively, filament 50 may comprise other sorts of metal wires or non-metallic materials, of larger or smaller diameters. Further alternatively, the scattering line may be scribed (for example, by etching or cutting) across transmitting area 48.
Receiver 44 receives the light that is scattered from filament 50 and generates an output that is indicative of the scattered light intensity. An example of such an output is shown in
Controller 52 calculates one or more operating parameters, such as the scan angle, of scanner 22 based on the receiver output due to scattering from filament 50. The calculated scan angle can be combined with the readout of receiver 44 in computing a 3D map with enhanced angular accuracy. More generally, controller 52 can apply this calculated scan angle, as well as other scan parameters, either in controlling the elements of scanning device 20 to operate with greater precision or in correcting the data extracted from the scanning device to compensate for scan inaccuracies. For example, controller 52 may compute and apply appropriate shifts to the coordinates of the TOF data in order to improve the accuracy of the 3D map generated by device 20.
The signals shown in
The scan offset between the two directions is indicative of a timing skew between the scan directions, which will result in an erroneous coordinate offset in the depth map between the odd and even scan lines. In addition, due to geometrical and mechanical factors, operating parameters of scanner 22 may vary over time, and when a two-axis scanner is used, as illustrated in
To address these problems, controller 52 processes curves 64 in order to calculate accurate scan parameters continually during operation of scanner 22. For this purpose, for example, controller 52 may fit the measured occurrence times of curves 64 to a parametric model of the scan angle α(t) as a function of time having the general form: α(t)=A sin(ω(t−t0)+φ)+b. The controller thus derives the scan parameters A, ω, t0, φ and b, and is able to calculate α(t) accurately at any time t during the scan. In this manner, controller 52 is able to associate a precise scan angle value with each depth value derived from the receiver output and can correct the 3D map output, for example by buffering and interpolating the depth values to compensate for angular distortion and errors.
The fitting process can be improved if the location and angular orientation of filament 50 are known, typically by means of pre-calibration, and can be refined still further if a pair of scattering lines is extended across the beam path. Even without such calibration, however, a single scattering line still enables controller 52 to extract useful scan coordinates from curves 64. Referring back to
Additionally or alternatively, controller 52 may calculate and correct other operating parameters of scanner 22 based on the timing of scattering curves 64. For example, the time span between successive scattering curves is indicative of the period, and hence the frequency, of oscillation of mirror 24. Furthermore, controller 52 can compute the difference between the measured time T1 and/or T2 and a pre-calibrated baseline time T0, at which curves 64 are expected to occur, as an indication of the phase offset of the scan.
In an alternative embodiment, not shown in the figures, two or more filaments or scattering lines of other sorts are extended across transmitting area 48. In this case, the signals output by receiver 44 will contain a double output curve, with multiple scattering peaks due to the multiple scattering lines. Controller 52 can then compute the relation between the two peaks, including their respective times of occurrence and the difference between these times, as an indication of the frequency, phase, and/or amplitude of the scan, and use the results of this computation in calculating and correcting scan parameters.
It will be appreciated that the embodiments described above are cited by way of example, and that the present invention is not limited to what has been particularly shown and described hereinabove. Rather, the scope of the present invention includes both combinations and subcombinations of the various features described hereinabove, as well as variations and modifications thereof which would occur to persons skilled in the art upon reading the foregoing description and which are not disclosed in the prior art.
| Number | Name | Date | Kind |
|---|---|---|---|
| 3796498 | Post | Mar 1974 | A |
| 4850673 | Velzel et al. | Jul 1989 | A |
| 5406543 | Kobayashi et al. | Apr 1995 | A |
| 5477383 | Jain | Dec 1995 | A |
| 5606181 | Sakuma et al. | Feb 1997 | A |
| 5648951 | Kato | Jul 1997 | A |
| 5691989 | Rakuljic et al. | Nov 1997 | A |
| 5742262 | Tabata et al. | Apr 1998 | A |
| 5781332 | Ogata | Jul 1998 | A |
| 6002520 | Hoch et al. | Dec 1999 | A |
| 6031611 | Rosakis et al. | Feb 2000 | A |
| 6229598 | Yoshida | May 2001 | B1 |
| 6288775 | Tanaka | Sep 2001 | B1 |
| 6560019 | Nakai | May 2003 | B2 |
| 6583873 | Goncharov et al. | Jun 2003 | B1 |
| 6611000 | Tamura et al. | Aug 2003 | B2 |
| 6707027 | Liess et al. | Mar 2004 | B2 |
| 6927852 | Reel | Aug 2005 | B2 |
| 6940583 | Butt et al. | Sep 2005 | B2 |
| 7112774 | Baer | Sep 2006 | B2 |
| 7227618 | Bi | Jun 2007 | B1 |
| 7304735 | Wang et al. | Dec 2007 | B2 |
| 7335898 | Donders et al. | Feb 2008 | B2 |
| 7700904 | Toyota et al. | Apr 2010 | B2 |
| 7952781 | Weiss et al. | May 2011 | B2 |
| 8384997 | Shpunt et al. | Feb 2013 | B2 |
| 8807766 | Hung | Aug 2014 | B2 |
| 9201237 | Chayat | Dec 2015 | B2 |
| 20040012958 | Hashimoto et al. | Jan 2004 | A1 |
| 20040082112 | Stephens | Apr 2004 | A1 |
| 20050030305 | Brown | Feb 2005 | A1 |
| 20050178950 | Yoshida | Aug 2005 | A1 |
| 20060001055 | Ueno et al. | Jan 2006 | A1 |
| 20060072100 | Yabe | Apr 2006 | A1 |
| 20060215149 | LaBelle et al. | Sep 2006 | A1 |
| 20060252167 | Wang | Nov 2006 | A1 |
| 20060252169 | Ashida | Nov 2006 | A1 |
| 20060269896 | Liu et al. | Nov 2006 | A1 |
| 20070019909 | Yamauchi et al. | Jan 2007 | A1 |
| 20080198355 | Domenicali et al. | Aug 2008 | A1 |
| 20080212835 | Tavor | Sep 2008 | A1 |
| 20080240502 | Freedman et al. | Oct 2008 | A1 |
| 20080278572 | Gharib et al. | Nov 2008 | A1 |
| 20090090937 | Park | Apr 2009 | A1 |
| 20090096783 | Shpunt et al. | Apr 2009 | A1 |
| 20090183125 | Magal et al. | Jul 2009 | A1 |
| 20090185274 | Shpunt | Jul 2009 | A1 |
| 20090237622 | Nishioka | Sep 2009 | A1 |
| 20100007717 | Spektor et al. | Jan 2010 | A1 |
| 20100013860 | Mandella et al. | Jan 2010 | A1 |
| 20100142014 | Rosen et al. | Jun 2010 | A1 |
| 20110019258 | Levola | Jan 2011 | A1 |
| 20110069389 | Shpunt | Mar 2011 | A1 |
| 20110075259 | Shpunt | Mar 2011 | A1 |
| 20110114857 | Akerman et al. | May 2011 | A1 |
| 20110141480 | Meissner | Jun 2011 | A1 |
| 20110187878 | Mor et al. | Aug 2011 | A1 |
| 20110188054 | Petronius et al. | Aug 2011 | A1 |
| 20110228251 | Yee et al. | Sep 2011 | A1 |
| 20110295331 | Wells et al. | Dec 2011 | A1 |
| 20120038986 | Pesach | Feb 2012 | A1 |
| 20120140094 | Shpunt et al. | Jun 2012 | A1 |
| 20120140109 | Shpunt et al. | Jun 2012 | A1 |
| 20130038881 | Pesach et al. | Feb 2013 | A1 |
| 20130038941 | Pesach et al. | Feb 2013 | A1 |
| 20130207970 | Shpunt et al. | Aug 2013 | A1 |
| 20140225824 | Shpunt et al. | Aug 2014 | A1 |
| 20140291491 | Shpunt et al. | Oct 2014 | A1 |
| 20150109586 | Masuda | Apr 2015 | A1 |
| Number | Date | Country |
|---|---|---|
| 1213244 | Apr 1999 | CN |
| 1651971 | Aug 2005 | CN |
| 1725042 | Jan 2006 | CN |
| 1748120 | Mar 2006 | CN |
| 201378231 | Jan 2010 | CN |
| 101874221 | Oct 2010 | CN |
| 102200431 | Sep 2011 | CN |
| 102009046911 | May 2011 | DE |
| 201118178 | Jun 2011 | JP |
| 2007043036 | Apr 2007 | WO |
| 2007105205 | Sep 2007 | WO |
| 2008120217 | Oct 2008 | WO |
| 2010004542 | Jan 2010 | WO |
| 2012020380 | Feb 2012 | WO |
| 2012066501 | May 2012 | WO |
| Entry |
|---|
| Fienup, J.R., “Phase Retrieval Algorithms: A Comparison”, Applied Optics, vol. 21, No. 15, pp. 2758-2769, Aug. 1, 1982. |
| International Application PCT/IL2008/01592 Search Report dated Apr. 3, 2009. |
| Sazbon et al., “Qualitative Real-Time Range Extraction for Preplanned Scene Partitioning Using Laser Beam Coding,” Pattern Recognition Letters 26 , pp. 1772-1781, year 2005. |
| Gerchberg et al., “A Practical Algorithm for the Determination of the Phase from Image and Diffraction Plane Pictures,” Journal Optik, vol. 35, No. 2, pp. 237-246, year 1972. |
| Moharam et al. “Rigorous coupled-wave analysis of planar-grating diffraction”, Journal of the Optical Society of America, vol. 71, No. 6, pp. 818-818, Jul. 1981. |
| Eisen et al., “Total internal reflection diffraction grating in conical mounting” ,Optical Communications 261, pp. 13-18, year 2006. |
| O'Shea et al., “Diffractive Optics: Design, Fabrication and Test”, SPIE Tutorial Texts in Optical Engineering, vol. TT62, pp. 66-72, SPIE Press, USA 2004. |
| Ezconn Czech A.S. “Site Presentation”, 32 pages, Oct. 2009. |
| Luxtera Inc., “Luxtera Announces World's First 10GBit CMOS Photonics Platform”, 2 pages, Carlsbad, USA, Mar. 28, 2005 (press release). |
| Bradley et al., “Synchronization and Rolling Shutter Compensation for Consumer Video Camera Arrays”, IEEE International Workshop on Projector-Camera Systems—PROCAMS 2009, 8 pages, Miami Beach, Florida, 2009. |
| Marcia et al., “Fast Disambiguation of Superimposed Images for Increased Field of View”, IEEE International Conference on Image Processing, 4 pages, San Diego, USA, Oct. 12-15, 2008. |
| Btendo, “Two Uni-axial Scanning Mirrors Vs One Bi-axial Scanning Mirror”, 4 pages, Kfar Saba, Israel, Aug. 13, 2008. |
| Microvision Inc., “Micro-Electro-Mechanical System (MEMS) Scanning Mirror”, 1 page, years 1996-2009. |
| European Patent Application # 11150668.9 Partial European Search Report dated Apr. 1, 2011. |
| Garcia et al., “Three-dimensional mapping and range measurement by means of projected speckle patterns”, Applied Optics, vol. 47, No. 16, pp. 3032-3040, Jun. 1, 2008. |
| Garcia et al.., “Projection of Speckle Patterns for 3D Sensing”, Journal of Physics, Conference series 139, 7 pages, year 2008. |
| International Application PCT/IB2011/053560 Search Report dated Jan 19, 2012. |
| International Application PCT/IB2011/055155 Search Report dated Apr. 20, 2012. |
| International Application PCT/IB2013/051986 Search Report dated Jul. 30, 2013. |
| International Application # PCT/IB2014/062244 Search Report dated Oct. 30, 2014. |
| Awtar et al, “Two-axis Optical MEMS Scanner,” Proceedings of the ASPE Annual Meeting ,Paper No. 1800, 4 pages, year 2005. |
| CN Application # 201380015507.8 Office Action dated Apr. 29, 2016. |
| Number | Date | Country | |
|---|---|---|---|
| 20170038581 A1 | Feb 2017 | US |
