LIGHT PIXEL PROJECTION MODULE

Abstract

A light pixel projection module includes a pixel light source, a light pixel projection assembly for projecting a light pixel generated by the light pixel generating assembly, and an optical time-of-flight (ToF) measurement assembly for measuring a distance between the projection module and an external object. The ToF measurement assembly includes a ToF light source, a beam splitting optical device for splitting an incident light beam into a reflected main beam component and a transmitted and attenuated secondary beam component, and an APD-based ToF photodetector for light detection. The beam splitting optical device is arranged in the optical path of light beams emitted by the ToF light source such that it splits each light beam emitted by the ToF light source into a main beam component leaving the module and heading towards the external object and a secondary beam component remaining within the module and hitting the ToF photodetector.

Description

The present application claims the priority of the German patent application No. 10 2020 201 453.1. The disclosure of the above German patent application is hereby incorporated into the present application by way of reference.

The present disclosure also pertains to a light pixel projection module for integration into an electronic device, the light pixel projection module being adapted to emit pixels of light that are to be projected onto a surface, the light pixel projection module comprising:

• • a light pixel generating assembly including a pixel light source;
  • a light pixel projection assembly for projecting a light pixel generated by the light pixel generating assembly; and
  • an optical time-of-flight—ToF—measurement assembly for measuring a distance between the projection module and an external object,

the ToF measurement assembly including:

a) a ToF light source for emitting light, which is to be directed onto the external object and reflected off the external object;

b) a beam splitting optical device for splitting an incident light beam into a reflected main beam component and a transmitted and attenuated secondary beam component; and

c) an APD-based ToF photodetector for light detection.

Such a light pixel projection module is known from document US 2019/0310489 A1.

This known solution has the drawback that it needs complex and power-consuming fast electronics 124, 170 to synchronise the photodetection with the driving of the IR laser 510 during ToF measurements.

An object of the present disclosure is therefore to provide a light pixel projection module with simple and power-saving ToF measurement capabilities.

According to the present disclosure, this object is achieved in that the beam splitting optical device is arranged in the optical path of light beams emitted by the ToF light source such that it splits each light beam emitted by the ToF light source into:

i) a main beam component leaving the module and heading towards the external object; and

ii) a secondary beam component remaining within the module and hitting the ToF photodetector.

Thanks to the beam splitting optical device of the present disclosure, a small part of the ToF light beam is diverted onto the ToF photodetector when it leaves the module. The detection of this secondary beam component by the ToF photodetector can then be used as the trigger for the necessary time measurement. Accordingly, one can dispense with the complicated synchronising electronics of the prior art.

According to preferred embodiments, the projection module of the present disclosure may include one, several or all of the following features, in all technically possible combinations:

• • the beam splitting optical device is a layer stack with an upper mirror section acting as a largely reflecting mirror, and a lower light attenuating section arranged underneath the upper mirror section;
  • the layer stack is deposited over the active surface area of the ToF photodetector;
  • the upper mirror section is an optical longpass filter with a cutoff wavelength that is higher than the maximum wavelength of the light beams emitted by the ToF light source such that the secondary beam component of a light beam split by the beam splitting optical device amounts to no more than 5% of the light beam's overall intensity;
  • the lower light attenuating section is a neutral density filter, preferably with a fractional transmittance of no more than 0.01%;
  • the beam splitting optical device also acts as the light pixel projection assembly;
  • a package, which houses the elements of the projection module, the ToF light source and the ToF photodetector thus sharing the same package;
  • a photodetector assembly for detecting a drift in the irradiance of the pixel light source and/or the ToF light source;
  • the photodetector assembly includes one or more silicon photomultipliers;
  • the beam splitting optical device covers both the photodetector assembly and the ToF photodetector;
  • the ToF photodetector includes at least one single photon avalanche diode or SPAD;
  • the ToF photodetector consists of a single SPAD and the ToF measurement assembly further comprises a ToF light scanning device for scanning the main beam component over the external object to obtain a 3D image thereof.

The present disclosure also relates to an image projection device for projecting a pixelated image onto a surface, the image projection device including a projection module as defined above and a light scanning module for distributing the light pixels projected by the projection module onto different parts of the surface, thus creating the pixelated image.

The present disclosure also relates to a method of making ToF measurements using a projection module as defined above.

Preferred embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings, in which:

FIG. 1 is a functional block diagram of a first embodiment of a light pixel projection module according to the present disclosure;

FIG. 2 is a functional block diagram showing the different elements of the supply and monitoring electronics of the projection module of FIG. 1;

FIGS. 3a and 3b are perspective views of a possible structural implementation of the first embodiment of FIGS. 1 and 2; and

FIG. 4 is a functional block diagram of a second embodiment of the light pixel projection module according to the present disclosure.

FIG. 1 shows a first embodiment 1300 of the light pixel projection module according to the present disclosure. The light pixel projection module 1300 is delimited by the dashed polygon. The light pixel projection module 1300 is a device that can be used to project changing light beams onto a screen to create pixels of an image for entertainment or professional applications. It is an optoelectronic component, which can be integrated into an electronic device, such as a smart phone, a wearable or tablet computer.

The light pixel projection module 1300 is adapted to emit pixels of light P that are to be projected onto a surface S.

The light pixel projection module 1300 comprises a light pixel generating assembly 1302, a light pixel projection assembly 1304, an optical time-of-flight measurement assembly 1306 (delimited by the dashed envelope), a photodetector assembly 1308 and supply and monitoring electronics 1310.

In the present example, the light pixel generating assembly 1302 consists of three laser diodes 1312, 1314 and 1316. The first laser diode 1312 is a blue laser and emits light BL at wavelength of e.g. 475 nm. The second laser diode 1314, is a green laser and emits light GL at a wavelength of e.g. 515 nm. The third laser diode 1316 in a red laser and emits light RL at a wavelength of e.g. 635 nm. The power of each laser diode may be up to 50 mW. Preferably, each laser diode emits intermittent individual light pulses with a duration of e.g. 2 ns (full width at half maximum, or FWHM). The three laser diodes 1312, 1314 and 1316, and thus the light pixel generating assembly 1302 may together be qualified as a pixel light source.

In FIG. 1, the light pixel projection assembly 1304 is implemented as a layer stack with an upper mirror section 1318 and a lower light attenuating section 1320 arranged underneath the upper section 1318. In the present example, the layer stack 1304 is deposited on top of the photodetector assembly 1308. The upper mirror section 1318 may be an optical longpass filter. This longpass filter 1318 may be implemented in multilayer coating technology using e.g. silicon dioxide and titanium dioxide. The lower light attenuating section 1320 may be a neutral density filter, preferably with a fractional transmittance of no more than 0.01%. The lower light attenuating section 1320 may be implemented as a metal layer filter (e.g. based on aluminium) with micrometric area apertures.

The optical time-of-flight (TOF) measurement assembly 1306 is configured for measuring distances between the projection module 1300 and an external object O. The TOF measurement assembly 1306 includes a TOF light source 1322, a beam splitting optical device 1304, an avalanche photodiode (APD) based TOF photodetector 1324, and TOF measurement electronics 1326.

The TOF light source 1322 is configured to emit light IL, which is to be directed onto the external object O, and reflected off the external object O. Preferably, the TOF light source 1322 is an infrared (IR) laser diode. The infrared laser diode 1322 may emit light at a wavelength of around 905 nm. The power of the infrared laser diode 1322 may be up to 1 to 10 W, but preferably is of the order of 50-100 mW. The infrared laser diode 1322 may in particular emit light pulses with a duration of around 2 ns (full width at half maximum, FWHM). In the present embodiment, all four laser diodes 1312, 1314, 1316 and 1322 are grouped together and form a multi-wavelength laser illumination block. Having all light sources of the projection module 1300 as a group in one place simplifies the setup of the module and the control of the light sources.

The pixel light source 1302 (i.e. the red, green and blue lasers) and the time-of-flight light source 1322 (i.e. the IR laser) are both oriented with respect to the layer stack 1304 such that the light beams BL, GL, RL and IL emitted by those light sources hit the outer surface 1328 of the layer stack 1304 at an angle a of around 45°.

In the embodiment shown in FIG. 1, the light pixel projection assembly (i.e. the layer stack) 1304 also acts as a beam splitting optical device. This means that the layer stack 1304 splits an incident light beam, for example the light beam IL, into a reflected main beam component ILm and a transmitted and attenuated secondary beam component ILs. More precisely, the layer stack 1304 is arranged in the optical path of light beams emitted by the TOF light source 1322 and the pixel light source 1302 such that it splits each light beam (e.g. BL) into a main beam component (BLm) leaving the module 1300 and a secondary beam component (BLs) remaining within the module 1300 and hitting the TOF photodetector 1324 or the photodetector assembly 1308. To that effect, the upper mirror section 1318 acts as a largely reflecting mirror. As mentioned above, the largely reflecting mirror 1318 may be an optical longpass filter. Its cut-off wavelength is higher than the maximum wavelength of the light beams IL emitted by the TOF light source 1322. Since the time-of-flight light source 1322 typically is an infrared light source, this means that the optical longpass filter 1318 reflects virtually all of the light, regardless of whether it comes from the time-of-flight light source 1322 or the pixel light source 1302. The secondary beam components ILs, BLs, GLs and RLs of the light beams IL, BL, GL and RL split off by the layer stack 1304 may amount to no more than 5% of the light beams' overall intensity.

The TOF photodetector 1324 is arranged beneath the layer stack 1304. In the present example, the photodetector assembly 1308 and the TOF photodetector 1324 are integrated into a single monolithic photodetector chip 1330. The layer stack 1304 is deposited over the entire top surface of the photodetector chip 1330 and thus over the active surface areas of the TOF photodetector 1324 and of the photodetector assembly 1308. In other words, the layer stack 1304 covers both the photodetector assembly 1308 and the TOF photodetector 1324.

In the first embodiment of FIG. 1, the TOF photodetector 1324 is implemented as an array of single photon avalanche diodes (SPADs). The SPAD array 1324 may have a total of 64×64 pixels with independent outputs.

The TOF measurement electronics 1326 for reading out the signals delivered by the SPAD array 1324 may be a time-to-digital converter or TDC.

The photodetector assembly 1308 may comprise four silicon photomultipliers (SiPMs) 1332, 1334, 1336 and 1338. Each silicon photomultiplier is associated with one of the lasers 1312, 1314, 1316 and 1322. The first silicon photomultiplier 1332 is provided to monitor the irradiance of the blue laser 1312. The second silicon photomultiplier 1334 is provided to monitor the irradiance of the green laser 1314. The third silicon photomultiplier 1336 is provided to monitor the irradiance of the red laser 1316. The fourth silicon photomultiplier 1338 is provided to monitor the irradiance of the infrared laser 1322. All four silicon photomultipliers may be of the same type. Alternatively, each SiPM may have its own specific design, which is optimised for light detection in the relevant colour.

In FIG. 1, the four silicon photomultipliers 1332 to 1338 are arranged next to and flush with each other. The arrangement is such that, when a blue light beam BL is emitted by the blue laser 1312 and split by the beam splitter 1304, the secondary beam component BLs hits the first silicon photomultiplier 1332, when a green light beam GL is emitted by the green laser 1314 and split by the beam splitter 1304, the secondary beam component GLs hits the second silicon photomultiplier 1334, when a red light beam RL is emitted by the red laser 1316 and split by the beam splitter 1304, the secondary beam component RLs hits the third silicon photomultiplier 1336, and when an infrared light beam IL is emitted by the infrared laser 1322 and split by the beam splitter 1304, the secondary beam component ILs hits the fourth silicon photomultiplier 1338.

The purpose of the photodetector assembly 1308 is to detect a potential drift in the irradiance of the laser light sources. The four silicon photomultipliers 1332 to 1338 may have 10×10 pixels each and may have multiplexed outputs.

The supply and monitoring electronics 1310 is shown in more detail in FIG. 2 and identified therein by a dashed polygon. The purpose of the supply and monitoring electronics 1310 is to supply the photodetector assembly 1308 and the TOF photodetector 1324 with the electric power that is necessary for their operation and also to process the output signals D1 to D4 delivered by the four silicon photomultipliers 1332 to 1338. To that effect, the supply and monitoring electronics 1310 includes a power supply 1340, a discriminator stage 1342, a converter stage 1344 and a processing stage 1346. As indicated by the arrows, the power supply 1340 provides the necessary operating voltage individually to the first three silicon photomultipliers (blue, green and red) 1332, 1334 and 1336. The fourth silicon photomultiplier 1338 and the SPAD array 1324 form an integrated pair and are jointly powered by the power supply 1340.

In the present example, both the discriminator stage 1342 and the converter stage 1344 have four individual channels for the four individual signals D1 to D4 of the four silicon photomultipliers. The discriminator stage 1342 acts as a thresholder that removes low-level artefacts from the output signals D1 to D4. The purpose of the converter stage 1344, which follows the discriminator stage 1342, is to count or integrate the signals. Accordingly, the converter stage 1344 may be a counter or analogue to digital converter (ADC). The output of the converter stage 1344 is then used to monitor the intensity or irradiance of the lasers, as represented by the arrow M in FIG. 2. If there is a drift in the intensity of the light delivered by the lasers, one may then take corrective action by adjusting the operation of the lasers driver 1348. In this way, one can ensure an optimal brightness of the projected images, which are obtained with the help of the projection module 1300.

The projection module 1300 of FIGS. 1 and 2 operates as follows:

The red, green, blue and infrared lasers emit very fast, high-power light pulses RL, GL, BL and IL. Most of the laser light is reflected by the layer stack 1304. The red, green and blue light reflected by the layer stack 1304 is used for laser projection. To this effect, the reflected components BLm, GLm and RLm may be shaped by a beam shaper 1311 and scanned over e.g. a screen S by a light scanning module 1313. The light scanning module 1313 may for example be a micro-electromechanical system or MEMS with a scanning mirror. In this way, one obtains a pixelated image on the surface of the screen S. The projection module 1300 and the light scanning module 1313 together form an image projection device 1315. In this image generating process, pixels P are projected by the layer stack 1304 out of the projection module 1300.

A small part of the light emitted by the four lasers is transmitted and attenuated by the layer stack 1304. This small part reaches the four silicon photomultipliers 1332 to 1338 and the SPAD array 1324. The red, green, blue and infrared laser light transmitted by the layer stack 1304 is monitored at single pulse level by the four independent silicon photomultipliers. In case there is an unwanted drift in the intensity of the pulses emitted by the red, green and/or blue laser, the control of the lasers driver 1348 is adjusted accordingly to correct the brightness of the image projected with the help of the projection module 1300 onto the screen S.

Optionally, the fourth silicon photomultiplier 1338 may also be used to detect stray infrared light inside the projection module 1300, for example, during the time intervals between the detection of consecutive infrared laser pulses by the fourth silicon photomultiplier 1338. This measurement of the amount of stray infrared light and thus of the IR noise present in the projection module 1300 may be used to improve the signal-to-noise ratio and/or the accuracy of the time-of-flight measurements.

The time-of-flight measurements work as follows: the secondary component ILs of the infrared light that reaches the SPAD array 1324 triggers the time acquisition for each pixel of the SPAD array. Concurrently, the reflected main beam component ILm leaves the module and heads towards the external object O. Infrared light from the main beam component ILm then reflects off the external object O and returns back to the SPAD array 1324. This generates the respective stop signals for each pixel of the SPAD array and the time acquisition is completed. The resulting time-of-flight measurements can be used to construct a 3D image of the surface of the external object O. Preferably, the return of infrared light from the external object O triggers a new round of time acquisitions, which is stopped once the infrared light has travelled back and forth for a second time between the SPAD array 1324 and the external object O. This process can be repeated several times in a closed-loop.

We will now turn to FIGS. 3, which show a possible structure of the projection module 1300 of the present disclosure. As shown in FIGS. 3, all the elements of the projection module 1300 may be integrated into a single package 1348. The package 1348 may have a cuboid shape. It may be made of ceramic or silicone. In the shown example, package 1348 has an outer rim 1350, which surrounds a recess 1352. The various functional elements of the projection module 1300 are arranged within this recess 1352. A lid 1354 may cover the recess 1352. FIG. 3a shows the projection module 1300 without the lid 1354. FIG. 3b shows the projection module 1300 including the lid 1354. The lid 1354 may be made of glass and include a metal mask. It may have a first transmission window 1356 for the light pixels P and a second transmission window 1358 for the infrared light from the ToF measurement assembly 1306.

An oblique support 1360 is arranged in the recess 1352. The oblique support 1360 has an inclined surface 1362. The photodetector chip 1330 is attached to the inclined surface 1362.

To have a better view of the photodetector chip 1330, the layer stack 1304 on top of that chip has been omitted from FIG. 3a. In reality, the beam splitting layer stack 1304 of course covers the entire photodetector chip 1330 and thus the entire inclined surface 1362.

The photodetector chip 1330 is divided into a first half H1, which includes the fourth silicon photomultiplier 1338 and the SPAD array 1324, and a second half H2, which includes the first three silicon photomultipliers 1332, 1334 and 1336.

The SPAD array 1324 may consist of a single row of single photon avalanche diodes, as indicated by the dashed rectangle in FIG. 3a. Preferably, all the SPADs of this single row are at the same vertical distance from the lower surface of the lid 1354. This can simplify the time-of-flight based 3D imaging since it minimises differences in path length from one SPAD pixel to the next.

The four laser diodes 1312, 1314, 1316 and 1322 are arranged in a facing relationship with respect to the inclined support 1360. The lasers are arranged in such a manner in the recess 1352 that the laser beams propagate in a direction parallel to the bottom of the recess 1352. In this configuration, the inclined surface 1362 of the support 1360 makes a 45° angle with the bottom of the recess 1352 such that the light beam components reflected off the beam splitting layer stack 1304 (not shown) can leave the projection module 1300 in a direction perpendicular to the plane defined by the lid 1354.

With reference to FIG. 4, we will now describe a second embodiment 2300 of the projection module according to the present disclosure. The second embodiment 2300 is in many ways similar to the first embodiment 1300 shown in FIGS. 1 to 3. In the following, we will only describe the differences with respect to the first embodiment. Regarding the similarities, reference is made to the previous description of the first embodiment.

The second embodiment 2300 is different from the first embodiment 1300 in that the TOF photodetector 2324 consists of a single SPAD. Furthermore, the TOF measurement assembly further comprises a TOF light scanning device 2500 for scanning the main beam component ILm over the external object O to obtain a 3D image thereof. The TOF light scanning device 2500 may be a micro-electromechanical system or MEMS with a scanning mirror. This variant has the advantage that it allows reducing the size of the TOF photodetector 2324.

Summarising, the light pixel projection modules of the present disclosure have in particular the following technical advantages:

• • an integrated time-of-flight measurement assembly for 3D imaging of external objects;
  • automatic correction of the brightness of projected images via real-time monitoring of the light intensity of the laser light sources;
  • infrared stray light monitoring for improved noise rejection during time-of-flight measurements;
  • simplified time-of-flight measurements by using a fraction of the ToF light beam as a trigger for time interval measurements;
  • very compact packaging concept with a multi-wavelength laser illumination block and a highly sensitive real-time multi-wavelength light monitoring block integrated into a single package in a face-to-face arrangement.

Claims

1. A light pixel projection module for integration into an electronic device, the light pixel projection module being adapted to emit pixels of light that are to be projected onto a surface, the light pixel projection module comprising: a light pixel generating assembly including a pixel light source;a light pixel projection assembly for projecting a light pixel generated by the light pixel generating assembly; andan optical time-of-flight (ToF) measurement assembly for measuring a distance between the projection module and an external object, the ToF measurement assembly including: a) a ToF light source for emitting light, which is to be directed onto the external object and reflected off the external object;b) a beam splitting optical device for splitting an incident light beam into a reflected main beam component and a transmitted and attenuated secondary beam component; andc) an APD-based ToF photodetector for light detection,wherein the beam splitting optical device is arranged in the optical path of light beams emitted by the ToF light source such that it splits each light beam emitted by the ToF light source into: i) a main beam component leaving the module and heading towards the external object; andii) a secondary beam component remaining within the module and hitting the ToF photodetector; andwherein the beam splitting optical device also acts as the light pixel projection assembly.

2. The projection module of claim 1, wherein the beam splitting optical device is a layer stack with an upper mirror section acting as a largely reflecting mirror, and a lower light attenuating section arranged underneath the upper mirror section.

3. The projection module of claim 2, wherein the layer stack is deposited over the active surface area of the ToF photodetector.

4. The projection module of claim 2, wherein the upper mirror section is an optical longpass filter with a cutoff wavelength that is higher than the maximum wavelength of the light beams emitted by the ToF light source such that the secondary beam component of a light beam split by the beam splitting optical device amounts to no more than 5% of the light beam's overall intensity.

5. The projection module of claim 2, wherein the lower light attenuating section is a neutral density filter with a fractional transmittance of no more than 0.01%.

6. (canceled)

7. The projection module of claim 1, further comprising a package, which houses the elements of the projection module, the ToF light source and the ToF photodetector thus sharing the same package.

8. The projection module of claim 1, further comprising a photodetector assembly for detecting a drift in the irradiance of the pixel light source and/or the ToF light source.

9. The projection module of claim 8, wherein the photodetector assembly includes one or more silicon photomultipliers.

10. The projection module of claim 8, wherein the beam splitting optical device covers both the photodetector assembly and the ToF photodetector.

11. The projection module of claim 1, wherein the ToF photodetector includes at least one single photon avalanche diode or SPAD.

12. The projection module of claim 11, wherein the ToF photodetector consists of a single SPAD and the ToF measurement assembly further comprises a ToF light scanning device for scanning the main beam component over the external object to obtain a 3D image thereof.

13. An image projection device for projecting a pixelated image onto a surface, the image projection device including the projection module of claim 1 and a light scanning module for distributing the light pixels projected by the projection module onto different parts of the surface, thus creating the pixelated image.