The present disclosure relates to an imaging device and a solid-state imaging element used in the imaging device.
Among methods for sensing an object or measuring a distance, the time-of-flight (TOF) method is known in which a distance is measured using flight time that light takes to travel to and return from a measurement object.
Unexamined Patent Application Publication (Translation of PCT Application) No. 2013-538342 discloses de-aliasing for disambiguating time of round-trip of detected light during repeated light emission and exposure without degradation in distance measurement accuracy by using different modulation frequencies in the phase-type TOF method. Specifically, Unexamined Patent Application Publication (Translation of PCT Application) No. 2013-538342 discloses a conventional technique for removing what is called aliasing in distance measurement by outputting a value of measured distance to an object for which the time of flight of reflected light (a delay dependent on an optical path) exceeds one cycle of the modulation frequency as a value of measured distance defined by the time of flight within one cycle of the modulation frequency.
In the conventional technique disclosed in Unexamined Patent Application Publication (Translation of PCT Application) No. 2013-538342, however, a low modulation frequency needs to be not greater than a half of a high modulation frequency in order to effectively realize de-aliasing. Accordingly, the above-mentioned conventional technique is problematic in that the frame rate is limited by low modulation frequencies and thus, speeding up is not possible.
In view of the above problem, an object of the present disclosure is to provide an imaging device which achieves high distance measurement accuracy and removal of what is called aliasing in distance measurement without lowering the frame rate.
In order to solve the aforementioned problem, an imaging device according to an aspect of the present disclosure is an imaging device which measures a distance to a subject by emitting light and receiving reflected light of the light emitted, the imaging device comprising: a controller which outputs a light emission signal instructing emission of the light and an exposure signal instructing exposure to the reflected light; a light source unit configured to emit the light according to a timing of the light emission signal; and a light receiver which performs exposure to the reflected light from the subject according to a timing of the exposure signal. The controller generates at least a first light emission/exposure period and a second light emission/exposure period and outputs the light emission signal and the exposure signal that differentiate a repetition cycle of each of the light emission signal and the exposure signal between the first light emission/exposure period and the second light emission/exposure period, the first light emission/exposure period being a period in which the light receiver receives the exposure signal and performs the exposure after a delay time with respect to a timing at which the light source unit receives the light emission signal and emits the light, the second light emission/exposure period being different in the delay time from the first light emission/exposure period.
Furthermore, an imaging device according to an aspect of the present disclosure is an imaging device which measures a distance to a subject by emitting light and receiving reflected light of the light emitted and includes: a light source unit configured to emit the light at a timing of a light emission signal instructing emission of the light; a light receiver which performs exposure at a plurality of different timings according to an exposure signal instructing exposure to the reflected light from the subject, and outputs a plurality of imaging signals; a calculator which receives the plurality of imaging signals and calculates the distance; and a controller which generates two or more patterns of varying phase relationships between the light emission signal and the exposure signal, and outputs the light emission signal and the exposure signal in a cycle that is different between at least two of the two or more patterns.
Furthermore, the controller may determine the cycle in each of the two or more patterns to output the light emission signal and the exposure signal in a cycle in which a phase relationship between the light emission signal and the exposure signal generated next to the exposure signal corresponding to the light emission signal in each of the two or more patterns is the same.
Furthermore, the controller may output the light emission signal and the exposure signal in the cycle that is arbitrary.
Furthermore, the controller may establish a period during which at least the light emission signal that is otherwise periodically generated is not generated.
Furthermore, the controller may establish a period during which at least the light emission signal that is otherwise periodically generated is not generated to make an average cycle in each of the two or more patterns the same.
Furthermore, the controller may establish a period during which the light emission signal that is otherwise periodically generated and the exposure signal corresponding to the light emission signal that is otherwise periodically generated are not generated.
Furthermore, the controller may establish a period during which the light emission signal that is otherwise periodically generated and the exposure signal corresponding to the light emission signal that is otherwise periodically generated are not generated to make an average cycle in each of the two or more patterns the same.
Furthermore, the light receiver may include: a solid-state imaging element which includes: a plurality of pixels; and a plurality of signal accumulators respectively associated with the plurality of pixels, and accumulates signals in the plurality of signal accumulators, and in the solid-state imaging element, each of the plurality of signal accumulators that accumulate signals detected by the same pixel may accumulate the signals resulting from exposure performed in an exposure period of which a timing of the exposure signal instructing the exposure to the reflected light from the subject is different from a timing of the light emission signal.
Furthermore, the controller may cause each of the plurality of signal accumulators that accumulate the signals detected by the same pixel of the light receiver to accumulate the signals through a plurality of light emissions and corresponding exposure, and may differentiate a repetition cycle of the light emission signal and a repetition cycle of the exposure signal of which a timing for instructing the exposure to the reflected light from the subject is different from the timing of the light emission signal.
Furthermore, the solid-state imaging element may be a charge coupled device (CCD) solid-state imaging element.
Furthermore, the imaging device may measure the distance to the subject using a time-of-flight (TOF) method.
Furthermore, a solid-state imaging element according to an aspect of the present disclosure is a solid-state imaging element used in an imaging device which includes: a light source unit which emits light at a timing of a light emission signal instructing emission of the light; a light receiver including the solid-state imaging element; and a calculator which receives an imaging signal from the light receiver, and calculates a distance, the imaging device measures a distance to a subject by generating two or more patterns of varying phase relationships between the light emission signal and an exposure signal, further generating the light emission signal and the exposure signal in a cycle that is different between at least two of the two or more patterns, emitting the light, and receiving reflected light of the light emitted, and the solid-state imaging element performs exposure at a plurality of different timings according to the exposure signal instructing exposure to the reflected light from the subject, and outputs a plurality of the imaging signals.
With the imaging device according to the present disclosure, it is possible to achieve high distance measurement accuracy and removal of aliasing in distance measurement without lowering the frame rate.
These and other objects, advantages and features of the disclosure will become apparent from the following description thereof taken in conjunction with the accompanying drawings that illustrate a specific embodiment of the present disclosure.
Hereinafter, an imaging device and a solid-state imaging element used in the imaging device according to embodiments of the present disclosure will be described with reference to the drawings. Note that each of the following embodiments shows one specific example of the present disclosure; the numerical values, shapes, materials, structural elements, the arrangement and connection of the structural elements, etc., shown in the following embodiments are mere examples, and are not intended to limit the present disclosure.
There are instances where overly detailed description is omitted. For example, detailed description of well-known matter, redundant description of substantially identical structural elements, etc., may be omitted. This is to prevent the subsequent description from becoming unnecessarily redundant, and thus facilitate understanding by a person having ordinary skill in the art.
Light source unit 11 includes a drive circuit, a capacitor, and a light-emitting element, and emits irradiation light by supplying a charge held in the capacitor to a light-emitting element. The light-emitting element may be a laser diode (LD), a light-emitting diode (LED), or the like. The irradiation light is, as an example, infrared light (including near-infrared light and far-infrared light).
Furthermore, controller 3 includes: exposure phase controller 35 which instructs a rising edge phase and a falling edge phase of an exposure signal; exposure repetition controller 36 which instructs the number of repetitions and the cycle of the exposure signal; and exposure signal compositing unit 34 which combines and outputs the exposure signal in accordance with the number of repetitions and the cycle instructed by exposure repetition controller 36.
With this configuration, controller 3 can independently determine the phase and the repetition cycle of each of the light emission signal and the exposure signal and can generate a plurality of patterns of varying phase relationships between the light emission signal and the exposure signal. Furthermore, the light emission signal and the exposure signal can be output in such a way that the repetition cycle is different between the patterns of varying phase relationships between the light emission signal and the exposure signal. Note that the repetition cycle instructed by light emission repetition controller 33 and exposure repetition controller 36 may be fixed or periodic or may change regularly or randomly.
Stated differently, controller 3 generates a light emission signal instructing light emission to a subject and an exposure signal instructing exposure to light reflected from the subject. More specifically, controller 3 according to the present embodiment causes each of a plurality of signal accumulators in which signals detected by the same pixel of light receiver 2 are accumulated to accumulate the signals through plural light emissions and corresponding exposure. Furthermore, controller 3 differentiates the repetition cycle of the light emission signal and the repetition cycle of the exposure signal of which the timing for instructing the exposure to the light reflected from the subject is different from the timing of the light emission signal.
Note that controller 3 may include separate controllers which generate the light emission signal and the exposure signal. For example, controller 3 includes an arithmetic processing unit such as a microcomputer. The microcomputer includes a processor (microprocessor) and memory and outputs a light emission control signal and an exposure control signal by the processor executing a drive program stored in the memory. Note that controller 3 may use a field-programmable gate array (FPGA), an Internet service provider (ISP), and the like, and may include one hardware or may include more than one hardware.
As illustrated in
Light source unit 1 blinks (emits pulsed light) according to the timing of the light emission signal generated by controller 3 and emits irradiation light (pulsed light) to the subject.
Light receiver 2 includes solid-state imaging element 20. Solid-state imaging element 20 receives reflected light (pulsed light) resulting from reflection of the irradiation light and background light (ambient light) such as sunlight.
Furthermore, solid-state imaging element 20 performs, for an area including the subject, exposure a plurality of times according to the timing indicated by the exposure signal generated by controller 3, includes a plurality of signal accumulators associated with respective pixels, and accumulates signals (accumulation signals) in the plurality of signal accumulators upon capturing an image, thereafter transfers the accumulated signals (accumulation signals), and outputs imaging signals corresponding the exposure amount. More specifically, in solid-state imaging element 20 according to the present embodiment, each of the plurality of signal accumulators that accumulate the signals detected by the same pixel accumulates signals resulting from exposure performed in an exposure period corresponding to the exposure signal having a timing different from the timing of the light emission signal.
Light receiver 2 further includes: a camera lens; an optical band-pass filter (BPF) which passes only light having a wavelength close to the wavelength of light irradiated by light source unit 1; and a circuit such as an A/D converter, as appropriate.
Calculator 4 outputs a distance signal (distance image) which is information of the distance to the subject obtained by calculation on the basis of the amounts of signals included in the imaging signal and a second imaging signal which have been received from light receiver 2.
The following describes the case where a charge coupled device (CCD) solid-state imaging element is used as solid-state imaging element 20 used in imaging device 10 according to the present embodiment.
As illustrated in this figure, solid-state imaging element 20 according to the present embodiment includes: photodiode (PD; a light-receiving element) 101; vertical transfer portion 102; horizontal transfer portion 103; signal charge detector 104; and semiconductor substrate voltage (SUB) terminal 105 which inputs signal φSUB for controlling SUB.
Photodiode 101 converts received light into a charge. Vertical transfer portion 102 includes a plurality of gates, and transfers charges read from photodiodes 101 sequentially in the vertical direction. Some of the plurality of gates are readout gates that read the charges from photodiodes 101.
Horizontal transfer portion 103 includes a plurality of gates, and transfers charges received from vertical transfer portions 102 by the plurality of gates as packets, sequentially in the horizontal direction. Signal charge detector 104 sequentially detects the charges received from the horizontal transfer portion, converts each charge into a voltage signal, and outputs the voltage signal.
Here, in the state where the readout gates are open, a substrate voltage (SUB) is controlled according to an exposure signal, and photodiode 101 is exposed to light in a period during which the exposure signal is Low. A charge generated by this exposure is accumulated in vertical transfer portion 102.
In other words, the exposure signal output from controller 3 and instructing an exposure timing is input to SUB terminal 105 and used to control the semiconductor substrate voltage (SUB).
In
Next, a method for driving imaging device 10 (operation timing) according to the present embodiment will be described with reference to
Note that (a) in
First, as illustrated in (a) and (b) in
The first light emission/exposure period is a period in which light receiver 2 receives the exposure signal and performs exposure after a first delay time with respect to the timing at which light source unit 1 receives the light emission signal and emits light. In the present embodiment, the length of the first exposure signal period is set to To which is the same as the length of the light emission signal period, and the first delay time is set to 0. Thus, the first exposure signal period is set to a period during which the light emission signal is transmitted (high level).
Next, as illustrated in (a) and (c) in
The second light emission/exposure period is a period in which light receiver 2 receives the exposure signal and performs exposure after a second delay time different from the first delay time with respect to the timing for receiving the light emission signal. In the present embodiment, the length of the second exposure signal period is set to To which is the same as the length of the light emission signal period and the length of the first exposure signal period, and the second delay time is set to To which is the sum of first delay time 0 and the first exposure signal period.
Next, as illustrated in (a) and (d) in
The third light emission/exposure period is a period in which light receiver 2 receives the exposure signal and performs exposure after a third delay time different from the first and second delay times with respect to the timing for receiving the light emission signal. In the present embodiment, the length of the third exposure signal period is set to To which is the same as the length of the light emission signal period and the length of each of the first and second exposure signal periods, and the third delay time is set to 2×To which is the sum of first delay time 0, first exposure signal period To, and second exposure signal period To.
This series of operations is repeatedly performed N times in the present embodiment. After this, the transfer of vertical transfer portion 102 and the transfer of horizontal transfer portion 103 are repeatedly performed sequentially, and the charge is converted into a voltage signal in signal charge detector 104 and output.
In this way, a plurality of packets already provided in vertical transfer portion 102 can be used as signal accumulators which accumulate signals (accumulation signals) obtained in a plurality of exposure periods that differ in the timing of the exposure signal for receiving reflected light from a subject with respect to the light emission signal. This makes it unnecessary to newly form signal accumulators. Photodiode 101 can be made larger with the same area, with it being possible to increase saturation sensitivity and increase the maximum light reception amount. Highly accurate distance measurement can thus be achieved.
The distance measuring operation by imaging device 10 according to the present embodiment will be described below in detail with reference to
First, controller 3 outputs the first exposure signal in the first light emission/exposure period, the second exposure signal in the second light emission/exposure period, and the third exposure signal in the third light emission/exposure period that differ in the timing at which light receiver 2 receives reflected light from a subject with respect to the light emission signal. In the present embodiment, the length of each of the first, second, and third exposure signal periods is set to Tb which is the same as the length of the light emission signal period, and the delay time of the first exposure signal with respect to the timing at which light source unit 1 receives the light emission signal and emits light is set to 0. Thus, the first exposure signal period is set to a period during which the light emission signal is transmitted (high level). The delay time of the second exposure signal is set to To which is the sum of first delay time 0 and first exposure signal period To. The delay time of the third exposure signal is set to 2×To which is the sum of second delay time To and second exposure signal period To. Accordingly, the exposure amount of background light is equal in the first, second, and third exposure signal periods.
Meanwhile, (a) in
In this case, calculator 4 compares total exposure amount S0 according to the first exposure signal and total exposure amount S2 according to the third exposure signal, and determines as indicated in the following Expression 1.
[Math. 1]
S0>S2 (Expression 1)
Let c be the speed of light (299,792,458 m/s). On the basis of the determination result of Expression 1, calculator 4 can calculate distance L according to the following Expression 2.
In this case, calculator 4 compares total exposure amount S0 according to the first exposure signal and total exposure amount S2 according to the third exposure signal, and determines as indicated in the following Expression 3.
[Math. 3]
S2≥S0 (Expression 3)
On the basis of the determination result of Expression 3, calculator 4 can calculate distance L according to the following Expression 4.
Here, in order to facilitate understanding of the imaging device and the solid-state imaging element used in the imaging device according to the present embodiment, a typical imaging device will be described with reference to (a) in
In the typical imaging device in (a) in
In (a) in
With the imaging device and the solid-state imaging element used in the imaging device according to the present embodiment, it is possible to solve the above-described problem. Details will be described with reference to (b) in
In (b) in
Next, with reference to
As described above, the present embodiment provides a plurality of signal accumulators (as an example, three signal accumulators) in which signals (accumulation signals) detected by the same pixel included in solid-state imaging element 20 are accumulated. The length of each of the first, second, and third exposure signal periods in the first, second, and third light emission/exposure periods in which the signals (accumulation signals) are accumulated in the plurality of signal accumulators is set to To which is the same as the length of the light emission signal period. The delay time of the first exposure signal with respect to the timing at which light source unit 1 receives the light emission signal and emits light is set to 0.
Thus, the first exposure signal period is set to a period during which the light emission signal is transmitted (high level). The delay time of the second exposure signal is set to To which is the sum of first delay time 0 and first exposure signal period To. The delay time of the third exposure signal is set to 2×To which is the sum of second delay time To and second exposure signal period To. Accordingly, the exposure amount of background light is equal in the first, second, and third exposure signal periods.
Furthermore, in the present embodiment, the repetition cycle of each of the light emission signal and the exposure signal in the first light emission/exposure period is set to six times the length To of the light emission signal period, the repetition cycle of each of the light emission signal and the exposure signal in the second light emission/exposure period is set to five times the length To of the light emission signal period, and the repetition cycle of each of the light emission signal and the exposure signal in the third light emission/exposure period is set to four times the length To of the light emission signal period.
In other words, the average of the cycles of repetition throughout the light emission/exposure period is no different from five times the length To of the light emission signal period which is one of the cycles of repetition for the light emission/exposure period generally set using the rectangular-wave TOF method (pulse TOF method) in which the repetition of light emission and exposure includes a phase in which no exposure is performed.
Furthermore,
In the first exposure period that is chronologically the first, exposure to the first half of the reflected light from the subject and the background light is performed. In the first exposure period that is chronologically the second and onward, exposure to the first half of the reflected light from the subject, the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) exceeds one cycle of repetition of the light emission/exposure period, that is, an object which may cause what is called aliasing, and the background light is performed.
In the second exposure period that is chronologically the first, exposure to the second half of the reflected light from the subject and the background light is performed. In the second exposure period that is chronologically the second and onward, exposure to the second half of the reflected light from the subject, the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) exceeds one cycle of repetition of the light emission/exposure period, that is, an object which may cause what is called aliasing, and the background light is performed.
In the third exposure period that is chronologically the first, exposure to only the background light is performed. In the third exposure period that is chronologically the second and onward, exposure to the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) exceeds one cycle of repetition of the light emission/exposure period, that is, an object which may cause what is called aliasing, and the background light is performed.
In other words, exposure is performed so as to include all reflected light from the subject in the period which is the sum of the first exposure signal period and the second exposure signal period. The exposure amount in the second exposure signal period is greater when delay Td of the reflected light from the subject with respect to the light emission signal timing is greater. Exposure to the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) is Td1 which exceeds 6×To which is the repetition cycle of each of the light emission signal and the exposure signal in the first light emission/exposure period (an object which may cause what is called aliasing) is performed equally in the first, second, and third exposure signal periods as with the background light.
This means that controller 3 generates two or more patterns of varying phase relationships between the light emission signal and the exposure signal, and furthermore, outputs the light emission signal and the exposure signal in a cycle that is different between at least two of the patterns.
Therefore, in Expression 2 for calculating distance L, the reflected light from the object which may cause what is called aliasing can be eliminated as with the background light.
As described above with reference to the drawings, the imaging device and the solid-state imaging element used in the imaging device according to the present embodiment eliminates the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) exceeds one cycle of repetition of the light emission/exposure period, that is, an object which may cause what is called aliasing, without adding special arithmetic processing, or increasing the light emission signal period, that is, pulse width To of the irradiation light, or increasing the average of the cycles of repetition throughout the light emission/exposure period. With this, it is possible to provide a high-speed, high-accuracy imaging device which can eliminate what is called aliasing in distance measurement.
Stated differently, (1) the width of the light emission signal which determines distance measurement accuracy is not changed, and furthermore (2) the average of the cycles of repetition of the light emission and the exposure is maintained, and thus the frame rate is not lowered, and (3) all exposure to the reflected light from an object of which the flight time (the delay dependent on the optical path) exceeds one cycle of repetition of the light emission and the exposure has the same signal amount, as with the background light, according to exposure signals which have different timings with respect to the timings of the light emission signals and instruct exposure for a signal to be accumulated in each of per-pixel different signal accumulators which accumulate signals (accumulation signals) detected by the same pixel of the light receiver. Thus, by removing the calculation process of subtracting the background light in the calculation for determining a distance signal from an imaging signal, it is possible to eliminate what is called aliasing in distance measurement at high speed with high accuracy.
Furthermore, in the present embodiment, the calculation for determining a distance signal from an imaging signal does not become complex including setting conditions for different cases, and the circuit size for signal processing is reduced, making it possible to downsize the imaging device and the solid-state imaging element used in the imaging device.
In a typical imaging device, S0 light emission/exposure signals and S2 light emission/exposure signals are different in the heat quantity per unit time because of a difference in the electric current value per unit time. Depending on the thermal characteristics of a component used in the imaging device, there is a problem of increased variations in the performance and characteristics of distance measurement. In particular, in the light source unit (a light source (such as a light-emitting element, an LED, and an LD) and a drive circuit (driver) for the light source), the effects of such thermal characteristics become noticeable.
However, the imaging device and the solid-state imaging element used in the imaging device according to the present variation can solve the above-described problem. Details will be described with reference to c and
Furthermore, since the S0 light emission/exposure signal and the S2 light emission/exposure signal have the same heat quantity per unit time, it is possible to reduce variations in the performance and characteristics of distance measurement that depend on the thermal characteristics of the light source unit (the light source (such as the light-emitting element, the LED, and the LD) and the drive circuit (driver) for the light source) and the light receiver (such as the solid-state imaging element).
Thus, in Variation 1 of the present embodiment, in addition to the same advantageous effects as in Embodiment 1, the advantageous effect that the distance measurement accuracy improves is further achieved.
With this, the S0 light emission signal and the S2 light emission signal have the same heat quantity per unit time; thus, in particular, it is possible to reduce variations in the performance and characteristics of distance measurement by preventing the effects of the thermal characteristics of the light source unit (the light source (such as the light-emitting element, the LED, and the LD)) and the drive circuit (driver) for the light source which significantly affect the characteristics of the imaging device.
Thus, in Variation 2 of the present embodiment, in addition to the same advantageous effects as in Embodiment 1, the advantageous effect that the distance measurement accuracy improves is further achieved.
The configurations and operations of an imaging device and a solid-state imaging element according to Embodiment 2 will be described below with reference to the drawings, focusing on differences from Embodiment 1.
The present embodiment provides, as described earlier, a plurality of signal accumulators (as an example, three signal accumulators) in which signals (accumulation signals) detected by the same pixel included in solid-state imaging element 20 are accumulated. The length of each of the first, second, and third exposure signal periods in the first, second, and third light emission/exposure periods in which the signals are accumulated in the plurality of signal accumulators is set to To which is the same as the length of the light emission signal period. The delay time of the first exposure signal with respect to the timing at which light source unit 1 receives the light emission signal and emits light is set to 0.
Thus, the first exposure signal period is set to a period during which the light emission signal is transmitted (high level). The delay time of the second exposure signal is set to To which is the sum of first delay time 0 and first exposure signal period To. The delay time of the third exposure signal is set to 2×To which is the sum of second delay time To and second exposure signal period To. Accordingly, the exposure amount of background light is equal in the first, second, and third exposure signal periods.
Furthermore, in the present embodiment, the cycle of repetition of the light emission signal and the exposure signal in the first light emission/exposure period is set to five and seven times the length To of the light emission signal period which are alternately repeated, the cycle of repetition of the light emission signal and the exposure signal in the second light emission/exposure period is set to four and six times the length To of the light emission signal period which are alternately repeated, and the cycle of repetition of the light emission signal and the exposure signal in the third light emission/exposure period is set to three and five times the length To of the light emission signal period which are alternately repeated. This means that controller 3 generates two or more patterns of varying phase relationships between the light emission signal and the exposure signal, and furthermore, outputs the light emission signal and the exposure signal in a cycle that is different between at least two of the patterns.
In other words, the average of the cycles of repetition throughout the light emission/exposure period is no different from five times the length To of the light emission signal period which is one cycle of repetition for the light emission/exposure period generally set using the rectangular-wave TOF method (pulse TOF method) in which the repetition of light emission and exposure includes a phase in which no exposure is performed.
Next,
In addition,
In the first exposure period that is chronologically the first, exposure to the first half of the reflected light from the subject and the background light is performed. In the first exposure period that is chronologically the second, exposure to the first half of the reflected light from the subject, the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) exceeds one cycle of repetition of the light emission/exposure period, that is, object 1-1 which may cause what is called aliasing, and the background light is performed. In the first exposure period that is chronologically the third, exposure to the first half of the reflected light from the subject, the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) exceeds one cycle of repetition of the light emission/exposure period, that is, object 1-2 which may cause what is called aliasing, and the background light is performed. From here on, the same exposures performed chronologically the second and the third are alternately repeated.
In the second exposure period that is chronologically the first, exposure to the second half of the reflected light from the subject and the background light is performed. In the second exposure period that is chronologically the second, exposure to the second half of the reflected light from the subject, the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) exceeds one cycle of repetition of the light emission/exposure period, that is, object 1-1 which may cause what is called aliasing, and the background light is performed. In the second exposure period that is chronologically the third, exposure to the second half of the reflected light from the subject, the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) exceeds one cycle of repetition of the light emission/exposure period, that is, object 1-2 which may cause what is called aliasing, and the background light is performed. From here on, the same exposures performed chronologically the second and the third are alternately repeated.
In the third exposure period that is chronologically the first, exposure to only the background light is performed. In the third exposure period that is chronologically the second, exposure to the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) exceeds one cycle of repetition of the light emission/exposure period, that is, object 1-1 which may cause what is called aliasing, and the background light is performed. In the third exposure period that is chronologically the third, exposure to the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) exceeds one cycle of repetition of the light emission/exposure period, that is, object 1-2 which may cause what is called aliasing, and the background light is performed. From here on, the same exposures performed chronologically the second and the third are alternately repeated.
In other words, exposure is performed so as to include all reflected light from the subject in the period which is the sum of the first exposure signal period and the second exposure signal period. The exposure amount in the second exposure signal period is greater when delay Td of the reflected light from the subject with respect to the light emission signal timing is greater. Exposure to the reflected light from objects 1-1 and 1-2 which may cause what is called aliasing is performed equally in the first, second, and third exposure signal periods as with the background light. Here, the reflected light from objects 1-1 and 1-2 is the reflected light of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) is Td1-1 and Td1-2 which are greater than 5×To, which is the smallest one of the cycle of repetition of the light emission signal and the exposure signal in the first light emission/exposure period, by a factor of more than one to less than 2.
Therefore, in Expression 2 for calculating distance L, the reflected light from the object which may cause what is called aliasing can be eliminated as with the background light.
As described above with reference to the drawings, the imaging device and the solid-state imaging element used in the imaging device according to the present embodiment eliminates the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) is greater than one cycle of repetition of the light emission/exposure period and less than two cycles thereof, that is, an object which may cause what is called aliasing, without adding special arithmetic processing, or increasing the light emission signal period, that is, pulse width To of the irradiation light, or increasing the average of the cycles of repetition throughout the light emission/exposure period. With this, it is possible to not only produce the same advantageous effects in Embodiment 1, but also eliminate what is called aliasing in distance measurement in a wider range.
Note that in the case of the average cycle (Duty, average Duty) indicated in Embodiment 2, controller 3 may set the cycle (duty) to be arbitrary (random). More specifically, the following is possible as illustrated in
(i) the S0 light emission signal and the S0 exposure signal are generated in cycles in which Duty=⅕ and Duty= 1/7 occur at the same rate as Duty=⅙;
(ii) the S1 light emission signal and the S1 exposure signal are generated in cycles in which Duty=¼ and Duty=⅙ occur at the same rate as Duty=⅕; and
(iii) the S2 light emission signal and the S2 exposure signal are generated in cycles in which Duty=⅓ and Duty=⅕ occur at the same rate as Duty=¼.
With this, a plurality of (as an example, three or more) aliasing phenomena can be reduced, and it is possible to not only produce the same advantageous effects in Embodiments 1 and 2, but also eliminate what is called aliasing in distance measurement in a wider range.
Note that the present variation can be used as a variation of Embodiment 1 described above.
The present embodiment provides, as described above, a plurality of signal accumulators (as an example, three signal accumulators) in which signals (accumulation signals) detected by the same pixel included in solid-state imaging element 20 are accumulated. The length of each of the first, second, and third exposure signal periods in the first, second, and third light emission/exposure periods in which the signals are accumulated in the plurality of signal accumulators is set to To which is the same as the length of the light emission signal period. The delay time of the first exposure signal with respect to the timing at which light source unit 1 receives the light emission signal and emits light is set to 0.
Thus, the first exposure signal period is set to a period during which the light emission signal is transmitted (high level). The delay time of the second exposure signal is set to To which is the sum of first delay time 0 and first exposure signal period To. The delay time of the third exposure signal is set to 2×To which is the sum of second delay time To and second exposure signal period To. Accordingly, the exposure amount of background light is equal in the first, second, and third exposure signal periods.
Furthermore, in the present embodiment, the cycle of repetition of the light emission signal and the exposure signal in the first light emission/exposure period is set to five and seven times the length To of the light emission signal period which are alternately repeated. The cycle of repetition of the light emission signal and the exposure signal in the second light emission/exposure period is set to four and seven times the length To of the light emission signal period which are alternately repeated. The cycle of repetition of the light emission signal and the exposure signal in the third light emission/exposure period is set to three and seven times the length To of the light emission signal period which are alternately repeated.
In other words, the average of the cycles of repetition throughout the light emission/exposure period is 5.5 times the length To of the light emission signal period, which is only 10% greater than five times the length To of the light emission signal period, which is one cycle of repetition for the light emission/exposure period generally set using the rectangular-wave TOF method (pulse TOF method) in which the repetition of light emission and exposure includes a phase in which no exposure is performed.
Next,
In the first exposure period that is chronologically the first, exposure to the first half of the reflected light from the subject and the background light is performed. In the first exposure period that is chronologically the second, exposure to the second half of the reflected light from the subject, the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) exceeds one cycle of repetition of the light emission/exposure period, that is, object 1 which may cause what is called aliasing, and the background light is performed.
In the first exposure period that is chronologically the third, exposure to the first half of the reflected light from the subject, the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) exceeds two cycles of repetition of the light emission/exposure period, that is, object 2 which may cause what is called aliasing, and the background light is performed. In the first exposure period that is chronologically the fourth, exposure to the first half of the reflected light from the subject, the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) exceeds one cycle of repetition of the light emission/exposure period, that is, object 1 which may cause what is called aliasing, the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) exceeds two cycles of repetition of the light emission/exposure period, that is, object 2 which may cause what is called aliasing, and the background light is performed. From here on, the same exposures performed chronologically the third and the fourth are alternately repeated.
In the second exposure period that is chronologically the first, exposure to the second half of the reflected light from the subject and the background light is performed. In the second exposure period that is chronologically the second, exposure to the second half of the reflected light from the subject, the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) exceeds one cycle of repetition of the light emission/exposure period, that is, object 1 which may cause what is called aliasing, and the background light is performed.
In the second exposure period that is chronologically the third, exposure to the second half of the reflected light from the subject, the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) exceeds two cycles of repetition of the light emission/exposure period, that is, object 2 which may cause what is called aliasing, and the background light is performed. In the second exposure period that is chronologically the fourth, exposure to the second half of the reflected light from the subject, the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) exceeds one cycle of repetition of the light emission/exposure period, that is, object 1 which may cause what is called aliasing, the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) exceeds two cycles of repetition of the light emission/exposure period, that is, object 2 which may cause what is called aliasing, and the background light is performed. From here on, the same exposures performed chronologically the third and the fourth are alternately repeated.
In the third exposure period that is chronologically the first, exposure to only the background light is performed. In the third exposure period that is chronologically the second, exposure to the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) exceeds one cycle of repetition of the light emission/exposure period, that is, object 1 which may cause what is called aliasing, and the background light is performed.
In the third exposure period that is chronologically the third, exposure to the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) exceeds two cycles of repetition of the light emission/exposure period, that is, object 2 which may cause what is called aliasing, and the background light is performed. In the third exposure period that is chronologically the fourth, exposure to the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) exceeds one cycle of repetition of the light emission/exposure period, that is, object 1 which may cause what is called aliasing, the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) exceeds two cycles of repetition of the light emission/exposure period, that is, object 2 which may cause what is called aliasing, and the background light is performed. From here on, the same exposures performed chronologically the third and the fourth are alternately repeated.
In other words, exposure is performed so as to include all reflected light from the subject in the period which is the sum of the first exposure signal period and the second exposure signal period. The exposure amount in the second exposure signal period is greater when delay Td of the reflected light from the subject with respect to the light emission signal timing is greater.
Exposure to the reflected light from objects 1 and 2 which may cause what is called aliasing is performed equally in the first, second, and third exposure signal periods as with the background light. Here, the reflected light from objects 1 and 2 is the reflected light of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) is Td1 which is greater than 5×To, which is the smallest one of the cycles of repetition of the light emission signal and the exposure signal in the first light emission/exposure period, by a factor of more than one to less than 2, and Td2 which is greater than 5×To by a factor of more than 2. Therefore, in Expression 2 for calculating distance L, the reflected light from the object which may cause what is called aliasing can be eliminated as with the background light.
As described above, in Variation 2 of the present embodiment, the reflected light from an object of which the delay dependent on the optical path with respect to the light emission signal timing (irradiation light) exceeds two cycles of repetition of the light emission/exposure period, that is, an object which may cause what is called aliasing, is eliminated without adding special arithmetic processing, or increasing the light emission signal period, that is, pulse width To of the irradiation light, or increasing the average of the cycles of repetition throughout the light emission/exposure period. With this, in addition to the same advantageous effects in Embodiments 1 and 2, the advantageous effect of allowing elimination of what is called aliasing in distance measurement in a wider range are achieved.
Note that out of the TOF methods, the rectangular-wave TOF method in which the repetition of light emission and exposure includes a phase in which no exposure is performed is exemplified as the method for obtaining a distance image in the above-described embodiments and variation thereof, but the method for obtaining a distance image is not limited to the rectangular-wave TOF method. The advantageous effects of the present disclosure can be obtained using other methods such as the modulation (phase) TOF method (irradiation light with sine waves) and the rectangular-wave modulation (phase) TOF method (irradiation light with rectangular waves) in which a distance image is obtained by calculation from signals obtained at four-phase exposure timings with different phases at every 90 degrees using a light source modulated for sine waves or rectangular waves.
Furthermore, although the above embodiments and variations thereof describe the case where exposure to the background light is performed (the background light is received) in order to improve the distance measurement accuracy, the advantageous effects of the present disclosure can be obtained even when exposure to only the irradiation light is performed (only the irradiation light is received) without performing exposure to the background light (without receiving the background light).
Furthermore, the above embodiments and variations thereof describe the imaging device, and the configuration of the imaging device according to the present disclosure is not limited to the imaging device which measures a distance according to distance information; it is possible to apply the present disclosure, for example, to a physical quantity detection device which accurately detects other physical quantity (for example, shape, temperature, and radiodensity) and an imaging device which accurately renders data of a captured image.
Although the imaging devices and the solid-state imaging elements according to the present disclosure have been described thus far based on the above embodiments and variations thereof, the imaging devices and the solid-state imaging elements according to the present disclosure are not limited to the embodiments and variations thereof described above. The present disclosure includes other embodiments implemented through a combination of arbitrary structural elements of the above embodiments and variations thereof, or variations obtained through the application of various modifications to the above embodiments and variations thereof that may be conceived by a person having ordinary skill in the art, without departing from the essence of the present disclosure, and various devices in which the imaging device and the solid-state imaging element according to the present disclosure are built-in.
The imaging device according to the present disclosure enables accurate three-dimensional detection and measurement of a subject without depending on the surrounding environment, and is thus useful, for example, for stereoscopic detection, display, and rendering of the forms of persons, buildings, and organs and tissues of human bodies, animals, and plants using a point cloud and the like, and eye gaze detection, gesture recognition, obstacle detection, road surface detection, and the like.
This application is a U.S. continuation application of PCT International Patent Application Number PCT/JP2016/004837 filed on Nov. 9, 2016, claiming the benefit of priority of U.S. Provisional Application No. 62/255,645 filed on Nov. 16, 2015, the entire contents of which are hereby incorporated by reference.
Number | Date | Country | |
---|---|---|---|
62255645 | Nov 2015 | US |
Number | Date | Country | |
---|---|---|---|
Parent | PCT/JP2016/004837 | Nov 2016 | US |
Child | 15975192 | US |