Structured light systems with static spatial light modulators

Abstract

Structured light systems are based on temporally modulated light sources and static spatial light modulators.

Description

TECHNICAL FIELD

The disclosure is related to structured light systems and static spatial light modulators.

BACKGROUND

Structured light systems project known light patterns onto an object. Surface contours of the object make the patterns appear distorted when viewed with a camera at a vantage point separated from the pattern projector by a baseline distance. Geometrical relationships are used to interpret the distortions to determine the distance from the projector to points on the object. In this way, three dimensional spatial coordinates of the surface of the object may be obtained.

Many conventional structured light systems are based on projecting patterns that are periodic in one dimension, such as stripe patterns. Successive spatially phase-shifted replicas of a pattern are projected. Conventional projectors, such as those based on digital-micromirror-array spatial light modulators, are able to produce grayscale patterns at approximately 200 Hz. The update rate is limited by the use of pulse width modulation to produce grayscale (analog pixel brightness) from a digital light modulator (binary pixel brightness).

Depth resolution of a structured light system depends on how well the spatial phase of a periodic pattern can be resolved, and that in turn depends on accurate measurements of pixel brightness across an image. When the update rate of a pattern projector is limited to a few hundred Hertz, noise from low-frequency sources such as 60 Hz lighting may degrade structured light system performance.

What are needed are structured light systems with faster pattern projection rates that enable synchronous detection of the phase of spatial patterns.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a conceptual diagram of a structured light projector with multiple light sources and static spatial light modulators.

FIG. 2 is a conceptual diagram of a structured light projector with one light source and multiple static spatial light modulators.

FIG. 3 is a conceptual diagram of a structured light projector with one light source and a spatial light modulator based on a moving mask.

FIG. 4 illustrates different positions of the moving mask of FIG. 3.

FIG. 5 illustrates an alternative mask scheme for a projector such as that of FIG. 3.

FIG. 6 illustrates a structured light system using any of the projectors of FIGS. 1-3.

DETAILED DESCRIPTION

Structured light systems with static spatial light modulators are capable of projecting light patterns much faster than is possible with conventional, reconfigurable spatial light modulators. A static spatial light modulator is one that has a fixed spatial pattern that it imparts to light passing through it. An example of a static spatial light modulator is a photomask: a glass plate with a metal coating adhered to it that varies from opaque to partially transmitting to clear in different regions of the plate. A static spatial light modulator may be moveable, but its spatial pattern cannot be reconfigured.

Three examples of structured light systems based on static spatial light modulators are discussed below. In one example, multiple light sources each illuminate different static spatial light modulators. This system permits pattern rates as fast as the light sources can be temporally modulated, well into the megahertz range. In two other examples, one temporally modulated light source is combined with a movable, static spatial light modulator. Different parts of the modulator are illuminated at different times. The speed of these systems depends on how fast different parts of a static modulator can be moved in front of a light source. Such systems may permit pattern rates in the kilohertz range.

FIG. 1 is a conceptual diagram of a structured light projector with multiple light sources and static spatial light modulators. In FIG. 1, four light sources 105, 106, 107 and 108 illuminate four static spatial light modulators 115, 116, 117 and 118, respectively. Homogenizers 110, 111, 112 and 113 serve to make the illumination of the modulators uniform. The homogenizers may be realized as light tunnels and spatial filters, for example. Finally, four lenses 120, 121, 122 and 123 project images of the spatially modulated light onto an object 125.

Light sources 105-108 may be diode lasers or light emitting diodes. These light sources are modulated temporally with standard diode driver electronics. The modulation frequency may be as high as 1 MHz or more, although modulation that fast may not be necessary. Static spatial light modulators 115-118 may be photomasks with variable thickness metal coatings. Areas on the mask where the coating is thin or nonexistent pass the most light while areas where the coating is thicker pass less light. Lenses 120-123 may be individual lenses as illustrated, lenses in a lens array, or part of a more complex optical system.

In the figure, the masks are shown with stripe patterns having sinusoidal optical density variations in one direction. Although the masks are shown as if they were in the plane of the page, in fact they are perpendicular to the page.

Light sources 105-108 are modulated such that they are turned on and off in succession. Thus the projected pattern that is incident upon object 125 changes from an image of static modulator 115 to one of static modulator 116 to one of static modulator 117, etc. The system of FIG. 1 uses four light sources and static modulators to project four successive patterns. The patterns are four spatial phases, e.g. 0, 90, 180 and 270 degrees, of one base pattern. Structured light depth capture may also be performed with three phases, e.g. 0, 120 and 240 degrees. In that case only three light sources, masks and associated optics are needed.

FIG. 2 is a conceptual diagram of a structured light projector with one light source and multiple static spatial light modulators. The system of FIG. 2 is similar to that of FIG. 1 except that in FIG. 2 a mechanical carrier holds the four different static light modulator masks. The carrier moves the masks sequentially through an optical system. The masks are otherwise the same as 115-118 of FIG. 1.

In FIG. 2, light source 205 illuminates one of masks 216, 217, 218 or 219 that are fixed in carrier 215. Homogenizer 210 ensures that the light source illuminates a mask uniformly. As in the system of FIG. 1, the homogenizer may be realized as a light tunnel and spatial filter. Lens 220 projects an image of a mask onto object 225.

Masks or static spatial light modulators 216-219 move back and forth in the apparatus of FIG. 2 as indicated by the arrows and in sync with modulation of light source 205. Thus the light source is turned on for a brief time while mask 216 is in the light path from homogenizer 210 to lens 220. Next the light source is turned on for a second brief time when mask 217 is in the light path, etc.

Light source 205, which may be a diode laser or light emitting diode, can be modulated at high speed. However, the system can only produce new images as fast as carrier 215 can position masks in the light path. This may limit pattern projection to kilohertz rates. Comparing the systems of FIGS. 1 and 2, the former permits much higher pattern rates, but the latter uses fewer optical components. As mentioned in connection with FIG. 1, in FIG. 2 carrier 215 and its masks are shown as if they lie in the plane of the page when in fact they are perpendicular to the plane of the page.

A third example of a structured light system based on a static spatial light modulator is shown in FIG. 3 which is a conceptual diagram of a structured light projector with one light source and a spatial light modulator based on a moving mask. The system of FIG. 3 is similar to that of FIG. 2 except that one mask with extra periods of a one-dimensional periodic pattern is moved in front of an aperture or frame to produce different spatial phases of the pattern.

In FIG. 3, light source 305 illuminates mask 312. Homogenizer 310 ensures uniform illumination of the mask. Lens 320 projects an image of the mask onto object 325. However, the image only includes the part of mask 312 that lies within frame 315. As in the systems of FIGS. 1 and 2, light source 305 may be a diode laser or light emitting diode, homogenizer 310 may include a light tunnel and spatial filter, and mask 312 and frame 315 are shown as if they lie in the plane of the page, when in fact they are oriented perpendicular to the plane of the page and perpendicular to the direction of propagation of light from light source 305 to lens 320.

Frame 315 is stationary with respect to the light source, homogenizer and lens. Mask 312 moves as indicated by the double arrow. Mask 312 moves such that successive spatial phases of the mask pattern are projected. The mask may move in steps and dwell in positions corresponding to specific spatial phases, or it may move smoothly and appear frozen at different spatial phases by short pulses of light emitted by the light source.

FIG. 4 illustrates different positions of the moving mask 312 of FIG. 3 with respect to frame 315. Inspection of the figure shows that the pattern of the mask is periodic and that the four positions shown in the figure correspond to four spatial phases of the pattern being aligned with the frame.

In one mechanical arrangement the mask stops at each position (0, 90, 180, 270) briefly. During the time the mask is stopped, a light source (e.g. 305) is turned on to illuminate the mask. The light source is then turned off while the mask is moved to a new position. Alternatively the light source may be modulated so that it emits short pulses of light. The mask may move smoothly in the direction indicated by the double arrow. The light pulses may be made short enough that movement of the mask is negligible during each pulse.

The short pulse approach simplifies the mechanics of moving the mask since in that case it need only oscillate back and forth. On the other hand, stopping and starting the mask at each phase may allow longer duration illumination in each position and thereby make signal detection easier.

FIG. 5 illustrates an alternative mask scheme for a projector such as that of FIG. 3. In FIG. 5, masks 510, 512, 514, etc are carried on a wheel 515. The wheel may be inserted in an apparatus such as that shown in FIG. 3 in place of mask 312. The wheel operates analogously to a color wheel in a color image projector that uses only one (reconfigurable) spatial light modulator and projects successive red, green and blue images. When the diameter of the wheel is small successive phases of a spatial pattern (e.g. 0, 90, 180 as shown in the figure) may be carried on it. On the other hand if the diameter is made larger, then a continuous pattern of stripes radiating out from the center of the wheel and having, e.g. sinusoidal, variation in the tangential direction may provide a substitute for discretely framed patterns.

Any of the projectors of FIGS. 1-3, including any of the variations discussed in connection with FIGS. 4 and 5, may be used in a structured light system such as that illustrated in FIG. 6. In FIG. 6 a pattern projector located at “P” generates a sinusoidal stripe pattern 605 that illuminates a three-dimensional surface located a distance, z, away from the projector. A camera located at point “C” views the pattern 610 that stripes 605 make when they illuminate the surface. The camera records the (X, Y) location of points in pattern 610 as they appear on the camera's image sensor 615. For example point 620 on the surface corresponds to point 621 in the camera. The camera is separated from the projector by baseline distance, d.

Pattern projector “P” may be any of the projectors based on static spatial light modulators as described above. Frequency and phase information describing the temporal modulation characteristics of projected patterns is communicated between projector P and camera C by a SYNC connection 630. Camera C (and associated processors and memory, not shown) demodulate the spatial phase of patterns that appear on the camera's image sensor. This spatial phase demodulation is aided by temporal demodulation of the same signal. Since projector P is capable of projecting images at kilohertz or even megahertz frame rates, the temporal demodulation allows the camera to separate the desired signal (i.e. the image intensity at each camera pixel) from lower frequency noise.

Static spatial light modulators may be used in systems that project grayscale light patterns much faster than is possible with conventional, reconfigurable spatial light modulators since the pattern repetition rate depends on the speed of light modulation (or in some cases mask movement) rather than the time required to reconfigure light modulator elements. These systems may be especially useful when the diversity of spatial objects to be measured is limited. Depth capture in an industrial production line setting can be a more predictable environment than gesture recognition for games, as an example.

The above description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the principles defined herein may be applied to other embodiments without departing from the scope of the disclosure. Thus, the disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A structured light system comprising: a first light source for illuminating a first static spatial light modulator;a second light source for illuminating a second static spatial light modulator; and,a third light source for illuminating a third static spatial light modulator, such that when the first, second and third light sources are turned on and off in succession, three spatial phases of one base spatial pattern are projected in succession.

2. The system of claim 1, the first, second and third light sources being diode lasers.

3. The system of claim 1, the first, second and third light sources being light emitting diodes.

4. The system of claim 1, the first, second and third static spatial light modulators being made from glass plates with metal coatings adhered to them.

5. The system of claim 1, the three spatial phases being 0, 120 and 240 degrees.

6. The system of claim 1 further comprising: a fourth light source for illuminating a fourth static spatial light modulator, such that when the first, second, third and fourth light sources are turned on and off in succession, four spatial phases of one base spatial pattern are projected in succession.

7. The system of claim 6, the four spatial phases being 0, 90, 180 and 270 degrees.

8. The system of claim 1 further comprising: a camera separated from the first, second and third static spatial light modulators by a baseline distance, the camera receiving information from the first, second and third light sources pertaining to temporal modulation characteristics of projected patterns.

9. The system of claim 1 further comprising: first, second and third homogenizers to make illumination of the respective spatial light modulators by the light sources uniform.

10. A structured light system comprising: a light source; and,a mechanical carrier that moves four different static spatial light modulator masks sequentially through an optical system in sync with temporal modulation of the light source, such that four spatial phases of one base spatial pattern are projected in succession.

11. The system of claim 10, the light source being a diode laser.

12. The system of claim 10, the light source being a light emitting diode.

13. The system of claim 10, the masks being made from glass plates with metal coatings adhered to them.

14. The system of claim 10, the four spatial phases being 0, 90, 180 and 270 degrees.

15. A structured light system comprising: a light source;a static spatial light modulator illuminated by the light source;a frame; and,a projection lens, the lens projecting an image of a part of the static spatial light modulator that appears within the frame,wherein the static spatial light modulator moves with respect to the frame such that different spatial phases of a one-dimensional periodic pattern are projected in sync with modulation of the light source.

16. The system of claim 15, the light source being a diode laser.

17. The system of claim 15, the light source being a light emitting diode.

18. The system of claim 15, the static spatial light modulator having successive phases of the one-dimensional periodic spatial pattern carried on a wheel and the wheel rotating in sync with modulation of the light source.

19. The system of claim 15 further comprising: a camera separated from the static spatial light modulator by a baseline distance, the camera receiving information from the light source pertaining to temporal modulation characteristics of projected patterns.