The present disclosure relates in general to three-dimensional (3D) surface contouring techniques, and more particularly to a device and method for using a spatial light modulator as a dynamic diffraction grating to reflect structured light in one of many types of patterns onto a surface of an object to ultimately determine through triangulation the 3D contour of the object's surface using the interference of two beams or spots of light.
In the field of three-dimensional surface contouring for accurately and rapidly determining the 3D coordinates of an object, there are many known techniques available, some of which involve the use of projecting various structured light patterns onto the object. The structured light pattern is typically formed in fringes (i.e., alternating bright and dark or different colored “stripes” or regions) on a surface of the object. In some cases, a spatial light modulator in the form of a diffraction grating of either a transmissive or reflective type is used to form grating patterns and to vary the phase of these patterns. The resulting fringe patterns on the surface of the object are then viewed by a camera device such as a charge coupled device (CCD), and processed by a computer or processor using various known triangulation techniques to ultimately determine the 3D surface contour of the object.
However, drawbacks with this type of approach include the fact that the diffraction grating is of a “static” type which must be moved by some type of manual means to effectuate a shift in the phase of the grating patterns. This results in a relatively slow phase shifting speed, which leads to less than optimum performance of the overall system. Also, such a system may require multiple separate diffraction gratings, each having a different grating period, to create a fringe pattern having the required spacing between the fringe lines (also known as pitch of the fringe lines). Besides the multiple gratings, it may also be necessary to provide associated translation stages and optical component feedback mechanisms, both of which are generally relatively expensive. Such a system may also require a relatively large amount of processor capability to process the camera captured images.
Other known prior art 3D object surface contouring systems are based on the direct projection of laser light, the projected image being essentially a replica of a pattern formed in a spatial light modulator such as, for example, in a digital micromirror device.
It is desirable to create very pure sinusoidal patterns having an infinite depth of field. A way to do this is to use a reflective or transmissive device as a dynamic diffraction grating device in a relatively highly accurate and less expensive 3D object surface contouring measurement system to form various types of structured light patterns by reflection of light off of the grating which then provides the reflected light through a pinhole plate to create by filtering two focused spots of light corresponding to the +1 and −1 order modes, and then allowing the light from the two spots of light to interfere at the surface of an object. The interference creates periodic sine waves that vary in intensity, thereby representing fringe patterns whose images may then be captured by a camera device and processed using known triangulation techniques to determine the 3D surface contour of the object. The reflective dynamic diffraction grating may comprise a digital micromirror device (DMD) comprised of a two-dimensional array of a plurality of movable reflective light switches or mirrors formed using microelectromechanical systems (MEMS) technology. The dynamic diffraction grating may be referred to in general as a spatial light modulator (SLM) of which the grating may be a particular type of SLM.
A method for determining three-dimensional coordinates of a first object point on a surface of an object includes the steps of: providing a source, a projector, and a camera, the projector including a spatial light modulator, a projector lens, and a pinhole plate, the camera including a camera lens and a photosensitive array, the projector having a projector perspective center, the camera having a camera perspective center, the line segment connecting the projector perspective center and the camera perspective center being the baseline, the length of the baseline being the baseline length; projecting a first light from the source to the spatial light modulator. The method also includes, in a first instance: spatially modulating the first light with the spatial light modulator to produce a first modulator pattern of light having a first pitch; sending the first modulator pattern of light through the projector lens to form a first plurality of light spots; filtering the first plurality of spots with the pinhole plate to pass a first pair of light spots while blocking other spots from among the first plurality of spots; propagating light from the first pair of light spots onto the object to obtain a first fringe pattern on the object, the first object point being illuminated by the first fringe pattern; imaging the first object point with the camera lens onto a first array point of the photosensitive array to obtain a first electrical data value from the photosensitive array. The method further includes, in a second instance: spatially modulating the first light with the spatial light modulator to produce a second modulator pattern of light having a pitch equal to the first pitch, wherein the second modulator pattern is spatially shifted relative to the first modulator pattern; sending the second modulator pattern of light through the projector lens to form a second plurality of light spots; filtering the second plurality of spots with the pinhole plate to pass a second pair of light spots while blocking other spots from among the second plurality of spots; propagating light from the second pair of light spots onto the object to obtain a second fringe pattern on the object, the first object point being illuminated by the second fringe pattern; imaging the first object point with the camera lens onto a first array point of the photosensitive array to obtain a second electrical data value from the photosensitive array. The method still further includes: determining the three-dimensional coordinates of the first object point based at least in part on the first electrical data value, the second electrical data value, and the baseline length; and storing the three-dimensional coordinates of the first object point.
Referring now to the drawings, exemplary embodiments are shown which should not be construed to be limiting regarding the entire scope of the disclosure, and wherein the elements are numbered alike in several FIGURES:
Referring to
More specifically, in the prior art embodiment of
The diffraction grating 104 may comprise a series of lines etched into a piece of glass. In an embodiment, the etch depth is d=λ/2 (n−1), where λ is the wavelength of the laser and n is the index of refraction of the glass. This creates a π/2 phase shift between the etched and non-etched regions, thereby minimizing the zero-order diffraction. The fringe shift is created by moving the grating perpendicular to the lines. The motion causes a phase change of ΔΦ in the +1 beam and −ΔΦ in the −1 beam. When the beams interfere after the pinhole plate 136 this causes a 2ΔΦ shift of the fringe pattern.
In an embodiment, there may be three channels in the projector. The channels differ by grating line pitch and pinhole position. For example, channel 1 may have a grating pitch of 228.6 microns and the motion for the 2π/3 phase shift may be 38.1 microns. Channel 2 may have a grating pitch of 200.0 microns and the motion for the 2π/3 phase shift may be 33.3 microns. Channel 3 may have a grating pitch of 180.0 microns and the motion for the 2π/3 phase shift may be 30.0 microns.
The grating pitch may be selected to make unwrapping relatively easier. In an embodiment, the phase unwrapping may use the Diophantine method. This requires the fringe pitch (and thus the grating pitch) to be multiples of relatively prime numbers. For example, pitch 1=8/7*pitch 2 and pitch 2=10/9*pitch 3. The relative ratios of 8:7 for channels 1 and 2 and 10:9 for channels 2 and 3 may make the unwrapping calculations relatively faster. Other methods for unwrapping phase are well known to those of ordinary skill in the art.
Not shown in
Referring to
The system 200 includes a source 210, a projector 230, a camera 260, and a processor 232. In an embodiment, the source 210 includes a laser, a fiber delivery system 212, and a collimating lens 216. The projector 230 includes a spatial light modulator 224, a projector lens 240, and a pinhole plate 252. The camera includes a lens 262 and a photosensitive array 265. The processor 232 communicates with the spatial light modulator 224 and the camera 260.
The system 200 includes the laser light source 204 that provides a beam of light 208 to an optical fiber 212. Other types of light sources may be utilized. The light beam 208 travels through the optical fiber 212 and then passes to a collimator lens 216 that provides a collimated light beam 220 to a digital micromirror device (DMD) 224. The DMD 224 is an array of microelectromechanical systems (MEMS) technology mirrors that can be individually addressed using electronic (e.g., digital) words. Typically, the array comprises 800×600 or 1024×768 individually addressable digital light switching elements or “pixels.” Other 2D array sizes are commercially available. One example of such a commercially available DMD 224 is a digital light processing (DLP®) microchip provided by Texas Instruments. Such a DLP® device 224 is based on MEMS technology and provides an all-digital implementation. The basic component of the DLP® device 224 is a reflective digital light switch (i.e., mirror) or pixel.
As indicated above, the DLP® device 224 may include an array having thousands of such pixels. In an embodiment, each mirror is 10-16 microns across and can rotate between two positions of ±10°, where +10° represents an “on” position at which the light beam 220 impinging on the particular mirror or pixel is reflected towards a desired object, and where −10° represents an “off” position at which the light beam 220 impinging on a particular mirror or pixel is “rejected” or directed away from the object as part of an “off beam” 228 (
By selecting a suitable pattern of light on the surface of the DLP® device 224, two plane waves of light corresponding to +1, −1 orders created by the DLP® pattern can be generated. These may be sent through a lens that focuses them to two small spots of light 248. The two spots of light pass through holes in a pinhole plate. Other light is not desired and is blocked by the pinhole plate. A pinhole plate used in this way is acting as a spatial filter. The light that emerges from the pinhole plate 252 are directed towards the surface 256 of the object whose surface contour is desired to be accurately and rapidly measured. The light from the pinholes overlap in an interference region 237, indicated in
A camera 260 is then used to capture images of the fringe patterns at the surface 256 of the object. The image data from the camera is provided to the processor 232, which controls the DLP® device 224—specifically, to control the phase of the grating patterns produced by the DLP® device 224 and ultimately the phase of the fringe patterns on the surface 256 of the object. The processor then may use known triangulation techniques to determine or calculate the 3D contour of the surface 256 of the object.
The camera device 260 includes a lens 262 and a photosensitive array 265. Light reflected or scattered off of a particular point 258 passes through all points of the lens and is focused onto a point 267 on the surface of the photosensitive array. The lens 262 has an optical axis, which is typically an axis of symmetry passing through the centers of the lens elements. There is a point in the lens 262 which is a perspective center 263. This is a point through which a ray may be drawn from the object point 258 to the array point 267. A real lens has aberrations which will make it depart slightly from the straight line path through the perspective center 263. However, measurements are carried out for each lens to characterize these aberrations and compensate for them in measurements.
A line drawn from the perspective center 241 of the projector lens 240 and the perspective center 263 of camera lens 262 is called the baseline 251, and the length of the baseline 251 is called the baseline length. The principle of the method of triangulation is to determine the lengths and angles of a triangle having vertices 263, 241, 258. The length of the baseline 251 and the values of the two angles a1 and a2 are used to find the length of the side from point 258 to point 263. The pixel position of the point 267 is used to determine the angles of the point 258 in relation to the optical axis 270. In this way, the coordinates of each point on the surface of the object may be determined.
Referring to
Referring to
Due to the discrete nature of the DMD array 224, the diffraction/holographic grating will become pixilated. For the diffraction grating example, the grating pitch is an even integer number of columns. This limits the possible grating pitches. The limit may affect projectors that need an integer ratio of grating pitches. There is also a limit imposed by the phase shift. If a 120° shift is desired then the pitch must be a multiple of 6 columns. If different (but known) phase shifts are acceptable then this issue disappears.
In an embodiment, grating periods are selected to be multiples of 6, 12, 18, 24, etc. These can all be shifted by ⅙th of the pattern to create a 120° phase shift. In another embodiment, grating periods that are not multiples of 6 are selected, and phases are shifted to be as near as possible to one-sixth of the grating period. For example, a 13 pixel period may be shifted by 2 then 4 pixels to give phase shifts of 0°, 111°, and 222°. The wrapped phase calculation takes into account these specific phase shift values. With this embodiment, the Diophantine method can be used. For example, if the pixel spacing is 10 micrometers in the DLP® device 224, grating pitches of 23, 20, and 18 pixels may be selected to get close to 8:7 and 10:9 ratios. The phase shifts are 125° for channel 1 (4 pixel shift), 108° for channel 2 (3 pixel shift), and 120° for channel 3 (3 pixel shift).
It is also possible to use more than three phase shifts—for example, four, five, or even more phase shifts. Additional phase shifts enable other patterns to be used. For example, a grating with a period of 8 pixels may have shifts of 1, 2, and 3 pixels to produce phase shifts of 90°, 180°, and 270°.
In another embodiment, the DMD 224 can be replaced with a different type of spatial light modulator (SLM) of which the DMD 224 is one example. The SLM can be used to vary the intensity as described above for the DMD. Some types of SLMs can be used in a “phase-only” mode in which the phase, rather than the intensity, of the reflected light is varied. An SLM used in a “phase-only” mode acts as a phase grating, allowing 100% of the light to be projected onto the object to be measured rather than 50% of the light as in the case of a DMD device. A disadvantage of an SLM that is not a DMD is the relatively slow write time. The refresh rate is 10-30 Hz, compared to 30 kHz for the DMD 224.
In another embodiment, an SLM that is not the DMD type is used in transmission mode, rather than reflection mode. Examples of companies that sell non-DMD SLMs include Hamamatsu (http://sales.hamamatsu.com/en/products/solid-state-division/lcos-slm.php); Boulder (http://www.bnonlinear.com/products/index.htm); and Meadowlark (http://www.meadowlark.com/products/slmLanding.php).
In another embodiment, the light source can be strobed (flashed on and off at defined intervals). This may be necessary if the DMD or SLM needs to be refreshed. For example, if the SLM requires time to change the liquid crystal from one state to another the light source can be turned off while the SLM changes and then turned back on when it has finished. Typically SLMs require 10-100 ms to switch.
Embodiments of the present invention provide for several advantages over prior art designs such as that of
While preferred embodiments have been shown and described, various modifications and substitutions may be made thereto without departing from the spirit and scope of the invention. Accordingly, it is to be understood that the present invention has been described by way of illustrations and not limitation.
The presently disclosed embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims, rather than the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.
The present application claims the benefit of U.S. Provisional Patent Application No. 61/507,304, filed on Jul. 13, 2011, the contents of which are hereby incorporated by reference in their entirety.
Number | Name | Date | Kind |
---|---|---|---|
3762809 | Kato et al. | Oct 1973 | A |
4017727 | Yamamoto | Apr 1977 | A |
5166508 | Davis | Nov 1992 | A |
5870191 | Shirley et al. | Feb 1999 | A |
6040910 | Wu et al. | Mar 2000 | A |
6438272 | Huang et al. | Aug 2002 | B1 |
6549730 | Hamada | Apr 2003 | B1 |
6690474 | Shirley | Feb 2004 | B1 |
6768589 | Deutsch et al. | Jul 2004 | B2 |
6909105 | Heintzmann et al. | Jun 2005 | B1 |
6985308 | Smith et al. | Jan 2006 | B1 |
7046410 | Deutsch et al. | May 2006 | B2 |
7139128 | Smith et al. | Nov 2006 | B2 |
7196789 | Senturia et al. | Mar 2007 | B2 |
7379241 | Smith | May 2008 | B2 |
7505641 | Senturia et al. | Mar 2009 | B1 |
7595892 | Judell et al. | Sep 2009 | B2 |
7751063 | Dillon et al. | Jul 2010 | B2 |
7763841 | McEldowney | Jul 2010 | B1 |
7791027 | McAllister et al. | Sep 2010 | B2 |
20020163573 | Bieman et al. | Nov 2002 | A1 |
20030052607 | Paquette | Mar 2003 | A1 |
20030058455 | Ebihara et al. | Mar 2003 | A1 |
20030072011 | Shirley | Apr 2003 | A1 |
20040081441 | Sato et al. | Apr 2004 | A1 |
20040125205 | Geng | Jul 2004 | A1 |
20050002677 | McAllister et al. | Jan 2005 | A1 |
20080180693 | Dillon et al. | Jul 2008 | A1 |
20090257063 | Smith et al. | Oct 2009 | A1 |
20100008588 | Feldkhun et al. | Jan 2010 | A1 |
20100182311 | Kim | Jul 2010 | A1 |
20100290060 | Mohazzab et al. | Nov 2010 | A1 |
20110043876 | Kurashige et al. | Feb 2011 | A1 |
20120262550 | Bridges | Oct 2012 | A1 |
20120262573 | Bridges | Oct 2012 | A1 |
Number | Date | Country |
---|---|---|
10321888 | Dec 2004 | DE |
1944569 | Jul 2008 | EP |
2157455 | Feb 2010 | EP |
2001127852 | May 2001 | JP |
2002013919 | Jan 2002 | JP |
2002090126 | Mar 2002 | JP |
2006258438 | Sep 2006 | JP |
2006107929 | Oct 2006 | WO |
2010021972 | Feb 2010 | WO |
2010096062 | Aug 2010 | WO |
2010096634 | Aug 2010 | WO |
Entry |
---|
International Search Report of the International Searching Authority for Application No. PCT/US2012/045361; Date of Mailing Oct. 22, 2012. |
International Search Report of the International Searching Authority for Application No. PCT/US2012/045908; Date of Mailing Oct. 8, 2012. |
International Search Report of the International Searching Authority for Application No. PCT/US2012/045909; Date of Mailing Oct. 8, 2012. |
Swanson, G., et al., High-Precision Surface Profiling with Broadband Accordion Fringe Interferometry, Proceedings of SPIE, vol. 4189, Feb. 12, 2001; pp. 161-169, XP55038933, ISSN: 0277-786X, DOI: 10.1117/12.417193 the whole document. |
Written Opinion of the International Searching Authority for Application No. PCT/US2012/045909; Date of Mailing Oct. 8, 2012. |
Written Opinion of the International Searching Authority for Application No. PCT/US2012/045361; Date of Mailing Oct. 22, 2012. |
Written Opinion of the International Searching Authority for Application No. PCT/US2012/045908; Date of Mailing Oct. 8, 2012. |
Hornbeck, L.J., Digital Light Processing for High-Brightness, High-Resolution Applications, Texas Instruments Inc., Digital Video Operations, Dallas, Texas, Electronic Imaging, EI '97 Projection Displays III, Co-Sponsored by IS&T and SPIE, An Invited Paper, Feb. 10-12, 1997, San Jose, California. |
Hornbeck, L.J., Digital Light Processing and MEMS: Timely Convergence for a Bright Future, Plenary Session, SPIE Micromachining and Microfabrication '95, Austin, Texas (Oct. 24, 1995). Color reprint available from Texas Instruments Digital Imaging Group, 214-995-2426. |
Frankowski, G., DLP-Based 3D Metrology by Structured Light or Projected Fringe Technology for Life Sciences and Industrial Metrology, GFMesstechnik GmbH, Germany, Proc. SPIE Photonics West 2009. |
VLSI Membrane Mirror Light Modulator for Multi-spectral Scene Projectors, Optron Systems, Inc., Public Release No. AFFTC PA 08430, http://www.optronsystems.com/html/technology—mems.htm. [Retrieved Jul. 12, 2012]. |
Frankowski, G., et al., Real-time 3D Shape Measurement with Digital Stripe Projection by Texas Instruments Micromirror Devices DMD, GFMesstechnik GmbH, Warthestr. 21, D-14513 Teltow/Berlin, Proc. of SPIE—vol. 2958 (2000), pp. 90-106. |
Rusinkiewicz, Szymon, et al., Real-Time 3D Model Acquisition, ACM Transactions on Graphics, (Proc. SIGGRAPH), Jul. 2002, http://gfx.cs.princeton.edu/pubs/Rusinkiewicz—2002—R3M/index.php, [Retrieved Jul. 11, 2012]. |
Texas Instruments, Using Lasers with DLP DMD Technology, Lasers & DLP, TI DN 2509927, Copyright Sep. 2008, Texas Instruments Incorporated. |
Georgia Tech. (Apr. 11, 2011). Trimensional 3D Scanner iPhone app. [Video file]. Retrieved from https://www.youtube.com/watch?v=a3IQcF2jO8k. |
Geng, Jason. “Structured-Light 3D Surface Imaging: A Tutorial”, Advances in Optics and Photonics, vol. 3, No. 2, Mar. 31, 2011, p. 128, XP55033088, DOI: 10.1364/A0P.3.000128, the whole document. |
International Preliminary Report on Patentability for International Application Serial No. PCT/US2012/035931. International filing date May 1, 2012. Date of Issuance Nov. 5, 2013. |
International Preliminary Report on Patentability for International Application Serial No. PCT/US2012/045361. International filing date Jul. 14, 2011. Date of Issuance Jan. 14, 2014. |
International Preliminary Report on Patentability for International Application Serial No. PCT/US2012/045908. International filing date Jul. 9, 2012. Date of Issuance Jan. 14, 2014. |
International Preliminary Report on Patentability for International Application Serial No. PCT/US2012/045909. International filing date Jul. 9, 2012. Date of Issuance Jan. 14, 2014. |
International Search Report of the International Searching Authority for Application No. PCT/US2012/035931; Date of Mailing Jul. 26, 2012. |
Written Opinion of the International Searching Authority for Application No. PCT/US2012/035931; Date of Mailing Jul. 26, 2012. |
Number | Date | Country | |
---|---|---|---|
20130016362 A1 | Jan 2013 | US |
Number | Date | Country | |
---|---|---|---|
61507304 | Jul 2011 | US |