This disclosure generally relates to an optical sensor and, more particularly, to a TOF sensor that cancels interference from another TOF sensor, crosstalk from smudges on the TOF sensor and other objects to be detected by modulating a light source using a random code, updating a calibration value and changing a sampling period.
The time of flight (TOF) technique nowadays is mainly divided into direct TOF (dTOF) and indirect TOF (iTOF).
In the dTOF, a light emitter is used to emit a light pulse, and then a clock signal in a detector is used to count a flying time of the light pulse after the light pulse is reflected by an object to be detected and then reaches the detector. However, if it is desired to implement a high resolution sensing, the dTOF system needs to use a clock signal having an extremely high frequency such that the circuit design difficulty is increased.
In the iTOF, a light emitter is used to emit a continuous light wave, and then a detector is used to detect a reflected light wave reflected by an object to be detected and reaches the detector. By calculating a phase delay of the reflected light wave from the continuous light wave, it is able to obtain a flying time.
Although the iTOF needs not to use a clock signal with an extremely high frequency to obtain the required temporal resolution, the signal aliasing can easily occur such that it is not able to distinguish multiple objects to be detected and crosstalk. Furthermore, the iTOF sensing can be interfered by other TOF sensors nearby.
Accordingly, it is necessary to provide a time of flight sensor that can cancel the interference from other TOF sensors, crosstalk from smudges on the device itself and other objects to be detected.
The present disclosure provides an optical sensor that uses the reference pixel(s) to generate a random code for modulating a light source and a sampling signal to distinguish different optical sensors.
The present disclosure further provides an optical sensor that detects a current crosstalk for updating a recorded calibration value so as to adaptably cancel the crosstalk interference.
The present disclosure further provides an optical sensor that uses an adjustable sampling period to detect a phase delay of a reflected light wave for multiple times within a light-off interval to accomplish multiple objects detection.
The present disclosure provides an optical sensor including a light source, a light sensor and a processor. The light source is configured to illuminate light according to a light source driving signal. The light sensor is recorded with an event threshold corresponding to an exposure interval, and includes a first pixel and a second pixel. The first pixel is configured to sample according to a sampling signal. The second pixel is configured to respectively acquire reference photon events using multiple of the exposure intervals. The processor is configured to compare a number of the reference photon events of each of the multiple exposure intervals with the event threshold to generate a random code, and modulate the light source driving signal and the sampling signal using the random code.
The present disclosure further provides an optical sensor including a light source, a pixel and a processor. The light source is configured to illuminate light according to a light source driving signal to cause the light source to have a lighting interval and an extinction interval within one operation period. The pixel is configured to acquire photon events according to a sampling signal corresponding to the lighting interval and the extinction interval identical to each other. The processor is configured to calculate an object distance according to the photon events using indirect time-of-flight, and upon the object distance being larger than a predetermined distance, change the extinction interval to be longer than the lighting interval.
The present disclosure further provides an optical sensor including a light source, a pixel and a processor. The light source is configured to illuminate light according to a light source driving signal to cause the light source to have a lighting interval shorter than an extinction interval within one operation period. The pixel is configured to respectively acquire photon events within at least three sampling periods according to a sampling signal. The processor is configured to calculate at least two object distances respectively according to a ratio of numbers of the photon events of two adjacent sampling periods among the at least three sampling periods.
Other objects, advantages, and novel features of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.
It should be noted that, wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts.
The optical sensor, using a time of flight (TOF) sensor as an example, but not limited to, of the present disclosure is to distinguish different optical sensors by modulating a light source driving signal, and to cancel crosstalk interference by updating a stored calibration value, and further to distinguish multiple objects to be detected by adjusting lengths of multiple sampling periods within one extinction interval.
Please refer to
The TOF sensor 100 includes a light source 11 and a light sensor 12 arranged inside a package 13. The package 13 has a first opening O1 opposite to the light source 11 allowing light to penetrate therethrough.
The light source 11 is a coherent light source or a non-coherent light source, e.g., a light emitting diode or a laser diode. The light source 11 is lighted on or lighted off according to a light source driving signal Sd.
The light sensor (e.g., a sensor chip) 12 includes a first pixel 121 and a second pixel 122, wherein the first pixel 121 and the second pixel 122 respectively include a single photon avalanche diode (SPAD) or multiple SPADs (e.g., forming a pixel array). The operation of the SPAD for detecting photon events is known to the art, and thus is not described herein.
The first pixel 121 receives reflected light from an object outside the package 13 and illuminated by the light source 11. The package 13 has a second opening O2 opposite to the first pixel 121 allowing light to penetrate therethrough. The second pixel 122 receives reflected light directly from an inner surface of the package 13 illuminated by the light source 11, as shown in
It should be mentioned that although
The processor 123 is, for example, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a digital signal processor (DSP), which is manufactured inside the light sensor 12 to respectively control the first pixel 121 and the second pixel 122 (respectively shown as an array in
In another aspect, the processor 123 includes multiple counters to count PE1 and PE2. The method of counting photon events detected by SPAD using counters may be referred to U.S. patent application Ser. No. 16/258,673, entitled “IMAGE SENSOR EMPLOYING AVALANCHE DIODE” filed on Jan. 28, 2019, and assigned to the same assignee of the present application, and the full disclosure of which is incorporated herein by reference.
In the present disclosure, in order to distinguish emission light from different devices (e.g., other TOF sensors nearby), the light sensor 12 records (e.g., by eFuse or in OPT memory) an event threshold μ corresponding to an exposure interval. The event threshold μ is a highest number of photon events, e.g., a peak in
For example referring to
Because a number of photon events within an exposure interval is affected by a length of the exposure interval, the exposure interval is also previously determined. In the present disclosure, the exposure interval is selected based on a number of photon events at the peak minus a 3 times of standard deviation σ (=μ1/2) in the probability distribution (i.e. μ−3σ) being larger than 0, and a number of photon events at the peak plus a 3 times of standard deviation (i.e. μ+3σ) in the probability distribution being smaller than a saturation of the second pixel 122. The determined exposure interval is also recorded (e.g., by eFuse or in OPT memory) in the TOF sensor 100 (e.g., light sensor 12 thereof, but not limited to).
During actual operation, the processor 13 controls the second pixel 122 to respectively acquire reference photon events PE2 using multiple exposure intervals (identical to the recorded exposure interval). The processor 123 then compares a number of reference photon events PE2 of each of the multiple exposure intervals with the event threshold μ to generate a random code RNC, and uses the random code RNC to modulate the light source driving signal Sd and the sampling signal (including Sa1 and Sa2).
Please refer to
In this embodiment, if the second pixel 122 includes one SPAD, the random code RNC is one bit; and if the second pixel 122 includes two SPADs, the random code RNC is two bits. More specifically, if the second pixel 122 is an array, the array is selected to be divided into multiple regions to perform the coding with multiple bits, and each of the multiple regions includes one SPAD or multiple SPADs.
For example referring to
Please refer to
The processor 123 calculates the object distance using the iTOF according to the photon events PEA and PEB. e.g., object distance=(c/2)×(PEB/PEA+PEB)×Ton, where c is light speed. When the calculated object distance is larger than a predetermined distance, the processor 123 changes the extinction interval Toff to be larger than the lighting interval Ton, e.g., the extinction interval Toff larger than 2 times of the lighting interval Ton, and
In one aspect, the processor 123 does not change the lighting interval Ton, and only extends the extinction interval Toff, but not limited to. In another aspect, the processor 123 adjusts lengths of both the lighting interval Ton and the extinction interval Toff.
This embodiment is to cancel the crosstalk interference caused by reflected light from the protection cover 200 in front of the TOF sensor 100 and smudges (including dust and oil if there are) 90 on the protection cover 200.
Please refer to
This operating method is automatically performed every time the TOF sensor 100 is provided with electricity or instructed by a user command (e.g., pushing a button or clicking an icon).
Step S81: Before operation, the TOF sensor 100 (e.g., the light sensor 12) previously records a predetermined calibration value (e.g., including PEAr and PEBr, wherein PEAr is for calibrating PEA, and PEBr is for calibrating PEB), which is a number of photon events contributed by reflected light from the protection cover 200, e.g., measured before shipment. Accordingly, in calculating the object distance according to the photon events PEA and PEB detected by the first pixel 121, said photon events are subtracted by the predetermined calibration value to obtain a calibrated object distance, e.g., the calibrated object distance=
(c/2)×[(PEB−PEBr)/(PEA×PEAr)+(PEB−PEBr)]×Ton (1)
Step S82-S83: When the calculated distance according to equation (1) is smaller than a predetermined distance, it means that said photon events detected by the first pixel 121 also contains reflected light from the protection cover 200 and the smudge 90, non-distinguishable, and thus the processor 123 does not change any setting and uses the equation (1) to calculate the object distance.
Step S82 and S84: When the calculated distance according to equation (1) is larger than a predetermined distance, it means that said photon events detected by the first pixel 121 only contains reflected light from the protection cover 200 and the smudge 90. If the circuit design difficulty of generating the light source driving signal Sd is not considered, the predetermined distance (corresponding to a width of Sd) is selected to be as close to a distance of the protection cover 200 as possible. Because smudges 90 accumulated on the protection cover 200 may be increased after continuous usage of the TOF sensor 100, the variation of the smudges 90 are detected using the smudge detection in Step S84.
In the smudge detection of Step S84, the processor 123 changes the extinction interval Toff to be larger than the lighting interval Ton, as shown in
In another aspect, the processor 123 uses two sampling signals ChA and ChB similar to
Step S85: Finally, the processor 123 replaces (or updates) PEAr and PEBr using the PEA′ and PEB′, and then calculates an object distance using equation (2)
(c/2)×[(PEB−PEB′)/(PEA−PEA′)+(PEB−PEB′)]×Ton (2)
Accordingly, the crosstalk interference of the device itself can be cancelled, and the calibration value is continuously updated and recorded according to the usage conditions so as to increase the detection accuracy.
Please refer to
In this embodiment, a number of sampling periods within the extinction interval Toff is determined according to an expected distance detection range of the TOF sensor 100. When the expected distance detection range is larger, the extinction interval Toff is selected to be longer and the number of the sampling periods is selected to be larger; whereas, when the expected distance detection range is shorter, the extinction interval Toff is selected to be shorter and the number of the sampling periods is selected to be smaller. The sampling periods are not limited to be continuous to an end of the extinction interval Toff as long as a first sampling interval (e.g., an interval corresponding to photon events PE1) among the at least three sampling periods is corresponding to the lighting internal Ton and the rest sampling intervals are corresponding to or within the extinction interval Toff.
The processor 123 calculates at least two object distances respectively according to a ratio of numbers of photon events of two adjacent sampling periods. For example, a first object distance=(c/2)×(PE2/PE1+PE2)×Ton; a second object distance=(c/2)×(PE3/PE2+PE3)×Ton+delay (shown as delay1); a third object distance=(c/2)×(PE4/PE3+PE4)×Ton+2×delay (shown as delay2): and so on. That is, the processor 123 further adds a delay distance in calculating the object distance using two adjacent sampling periods behind a first sampling period among the at least three sampling periods.
The delay distance is determined according to lengths of the sampling periods and the lighting interval Ton. For example as shown in
Furthermore, the present disclosure further calibrates a distance resolution by changing lengths of the lighting interval Ton and the sampling periods. For example, in the first measurement (e.g., called rough measurement), 20 nm shown in
Moreover, after a distance is obtained in the first measurement, a length of the extinction interval Toff can be decreased so as to increase the detection speed.
It should be mentioned that values, including time intervals, code bits and a number of sampling periods, mentioned in the above embodiments are only intended to illustrate but not to limit the present disclosure.
Furthermore, the above embodiments are combinable to one another to form another embodiment, e.g., the random code RNC and the calibration value obtained using the second pixel 122 can be applied to all embodiments of the present disclosure.
In the present disclosure, in addition to be used as a random number generator, the second pixel 122 can also be used to compensate a temperature dependence of the detection result of the first pixel 121, e.g., referring to U.S. patent application Ser. No. 16/936,777, entitled “TIME OF FLIGHT SENSOR CAPABLE OF COMPENSATING TEMPERATURE DEVIATION AND TEMPERATURE COMPENSATION METHOD THEREOF” filed on Jul. 23, 2020, and assigned to the same assignee of the present application, and the full disclosure of which is incorporated herein by reference.
As mentioned above, it is difficult for the conventional iTOF technique to distinguish interference from multiple objects to be detected, crosstalk and other TOF devices. Therefore, the present disclosure further provides a time of flight sensor capable of cancelling interference (as shown in
Although the disclosure has been explained in relation to its preferred embodiment, it is not used to limit the disclosure. It is to be understood that many other possible modifications and variations can be made by those skilled in the art without departing from the spirit and scope of the disclosure as hereinafter claimed.