This application relates generally to imaging systems, and more specifically, to phase compensation in a time of flight system.
A time-of-flight camera (ToF camera) is a range imaging camera system that resolves distance based on the known speed of light, measuring the time-of-flight of a light signal between the camera and the subject for each point of the image. A time-of-flight camera generally includes an illumination unit that illuminates the subject with light modulated with frequencies up to 100 MHz. The illumination unit normally uses infrared light to make the illumination unobtrusive. A lens can be used to gather the reflected light and images the environment onto an image sensor, with an optical band-pass filter passing only the light with the same wavelength as the illumination unit. This helps suppress non-pertinent light and reduce noise. At the image sensor, each pixel measures the time the light has taken to travel from the illumination unit to the object and back to the sensor. From this time, a distance to the subject at that point can be determined.
In accordance with one example, a time of flight system is provided. An illumination system includes an illumination driver and an illumination source and is configured to project modulated electromagnetic radiation to a point on a surface of interest. A sensor system includes a sensor driver and is configured to receive and demodulate electromagnetic radiation reflected from the surface of interest. A temperature sensor is configured to provide a measured temperature representing a temperature at one of the illumination driver and the sensor driver and located at a position remote from the one of the illumination driver and the sensor driver. A compensation component is configured to calculate a phase offset between the illumination system and the sensor system from at least the measured temperature and a model representing transient heat flow within the system.
In accordance with another example, a method is provided for determining a distance to a surface of interest with a time of flight system. Modulated electromagnetic radiation is projected on a point on the surface of interest via an illumination system. Electromagnetic radiation reflected from the point on the surface of interest is received at a sensor system. The received electromagnetic radiation is demodulated at the sensor system. A temperature is measured at a position remote from respective drivers associated with each of the illumination system and the sensor system. A phase offset between the illumination system and the sensor system is calculated from at least the measured temperature and a model representing transient heat flow within the system. A location of the point on the surface of interest, relative to the time of flight system, is determined from the received electromagnetic radiation and the calculated phase offset.
In accordance with yet another example, a time of flight system is provided. An illumination system includes an illumination driver and an illumination source is configured to project modulated electromagnetic radiation to a point on a surface of interest. A sensor system includes a sensor driver and is configured to receive and demodulate electromagnetic radiation reflected from the surface of interest. A temperature sensor is configured to provide a measured temperature representing a temperature at one of the illumination driver and the sensor driver and located at a position remote from the one of the illumination driver and the sensor driver. A compensation component is configured to calculate a numerical representation of the time derivative of the measured temperature for at least one time as well as a phase offset between the illumination system and the sensor system. The phase offset is calculated from at least the measured temperature, the calculated time derivative of the measured temperature, and a model representing transient heat flow within the system. A time of flight calculation component is configured to determine a distance to the point of the surface of interest from the demodulated electromagnetic radiation and the phase offset.
The features, objects, and advantages of the hybrid qubit assembly will become more apparent from the detailed description set forth below when taken in conjunction with the drawings, wherein:
Described herein are systems and methods for maintaining an accurate phase offset in a time of flight imaging system. In a time of flight system, both the illumination unit and the image sensor have to be controlled by high-speed signals, provided by associated driver electronics, and synchronized with high accuracy to maintain to obtain a high resolution. For example, if the signals between the illumination unit and the sensor shift by only ten picoseconds, the measured distance changes by one and a half millimeters. At these levels of precision, changes in the temperature of the system components can affect this timing sufficiently to introduce significant inaccuracies. A correction for any differences in timing between the illumination unit and the image sensor, referred to as a phase offset, can be obtained via a calibration process for a range of expected temperatures.
It can be difficult to directly measure the temperature at the driver electronics, as there will be a finite distance between the driver and the sensor. Accordingly, in the systems described herein, it is assumed that the temperature information used for correction is provided from a location spatially remote from the drivers. When the system has been operating for some length of time, the temperature is relatively constant across the circuitry, and the remote measurement can be relied upon for calculating the phase offset. During a transition in the temperature of these components, as might occur during initiation of the system or when the power provided to the illumination is changed, the temperature at the remote location could vary significantly and nonlinearly from the temperature at the driver. As a result, a system relying solely on the measured temperature can experience significant inaccuracies in the phase offset and the corresponding measurement until a steady state temperature is reached.
Each of the illumination driver 14 and the sensor driver 22 can be provided with a timing reference from a common timing generator 11, although it will be appreciated that small differences in the provided reference can be introduced via differences in the path lengths between the drivers and the timing generator. The accuracy of the time of flight calculation can be affected by any difference in the timing of the illumination system 12 and the sensor 20, and more specifically by differences in the timing their respective drivers 14 and 22. Accordingly, a phase offset between the illumination driver 14 and the sensor driver 22 can be provided to the time of flight calculation component 24 from a compensation component 26 to account for any differences in timing. The phase offset is sensitive to a die temperature of the circuit board or boards containing the illumination driver 14 and the sensor driver 22. When the system 10 has reached a steady state temperature, the phase offset tends to stabilize to a standard value. During transitions, such as starting the system or changing the power of the illumination system 12, the offset value can vary significantly. It will be appreciated that it is undesirable to utilize the standard correction for the system during these times.
In the illustrated system 10, a direct measurement of the temperature at the illumination driver 14 or the sensor driver 22 is not available. Instead, one or more temperature sensors 28, each located remote from each of the illumination driver 14 or the sensor driver 22 but on a same circuit board as one of the drivers 14 and 22, each provide a measured temperature to the compensation component 26. From the measured temperature or temperatures and a transient heat flow model of the system 10, the phase offset can be corrected during transitions, maintaining an accurate time of flight measurement. In one implementation, the transient heat flow model can be a specific instance of a parameterized general model, with a set of parameters for a particular system configuration being determined via curve fitting of the results of a reference circuit board to the general model.
The illustrated system 10 takes into account the transient variation of temperatures rather than just the instantaneous values to calibrate the phase offset, providing faster convergence of the measured value of phase to the correct value. Current time of flight systems sometimes take several minutes to give stable value of phase as the thermal time constants are quite long. The illustrated system converges to the correct phase offset in only a few seconds, allowing the system to remain useful when the power provided to the illumination system 12 is altered or the temperature at the drivers 14 and 22 is otherwise changed.
The inventors have determined that, for the illustrated system 50 and similarly configured systems, the relationship between the phase offset and the temperature at the photosensor driver 52 is substantially linear. Specifically, the phase offset due to the temperature at the photosensor driver 52 at a given time, t, can be expressed as:
P0(t)=K(TS(t)−TSC) Eq. 1
where P0 is the phase offset, TS, is the photosensor temperature, TSC is the temperature for which the phase offset is calibrated to be zero, and K is a constant static coefficient determined for the system.
The temperature of the photosensor driver will vary from an initial temperature, TS0, to a final stable temperature, TSF, in an essentially exponential fashion, such that:
TS(t)TSF+(TS0−TSF)e−t/τ
where τS is a thermal time constant, measured for each system, associated with the temperature change at the photosensor.
From Eqs. 1 and 2, the change in the phase offset over time can be expressed, in terms of the photosensor temperature, as:
P0(t)=K(TSFTS0−TSF]e−t/τ
The measured temperature, T1, and the photosensor temperature, are also generally linearly related. In the illustrated system 50, however, during transitions, the presence of the capacitors CP and CS introduce nonlinearities into the relationship between the measured temperature and the temperature at the photosensor. Accordingly, while the measured temperature varies exponentially with time, it may due so with a different thermal time constant, τ1, such that:
T1(t)=T1F+(T10−T1F)e−t/τ
where T10 is an initial temperature at the temperature sensor, T1F is a final stable temperature, τ1 is a thermal time constant, measured for each system, associated with the temperature change at the temperature sensor.
From Eq. 4, we can determine a time derivative of the temperature at the temperature sensor as:
Since the temperature at the photosensor and the temperature at the temperature sensor are linearly related outside of temperature transients, Eq. 3 can be rewritten as:
P0(t)=K(T1FT10−T1F]e−t/τ
where T1C is the temperature for which the phase offset is calibrated to be zero.
Using Eq. 5, Eq. 6 can be rewritten as:
It will be appreciated that the time derivative of the measured temperature can be determined numerically from the measured temperature data. In many systems, it can be assumed that the time constants for the temperature at the photosensor driver 52 and the temperature sensor 56 are reasonably similar in value, such the term
is sufficiently close to zero to allow the exponential term to be linearly approximated near zero as
Accordingly, the model of Eq. 7 can be approximated as:
In other systems, it can be assumed that the time constants for the temperature at the photosensor driver 52 and the temperature sensor 56 are substantially similar in value, such that such the term
is sufficiently close to zero to allow the exponential term to be ignored. In this case, the model of Eq. 7 can be approximated as:
It will be appreciated that the time constants, τ1 and τs, as well as the constant K will differ among systems. To this end,
In view of the foregoing structural and functional features described above, methods in accordance with various aspects of the present invention will be better appreciated with reference to
At 180, a phase offset between the illumination system and the sensor system is calculated from at least the measured temperature and a model representing transient heat flow within the system. In one implementation, the model can include a time derivative of the measured temperature, such that calculating the phase offset includes calculating a numerical derivative of the measured temperature for at least one time. Depending on the specifics of the system, the model can include one or more of a nonlinear function of a time since a transition in temperature has begun, a product of the time derivative of the measured temperature and a linear term representing the time since the transition began, and a linear combination the time derivative of the measured temperature, a final stable temperature at the temperature sensor after transition in temperature, and a calibration temperature for the temperature sensor at which the phase offset is expected to be zero.
It will be appreciated that a given model can be parameterized for a general class of systems. To this end, the method can include an additional step of determining a plurality of system parameters for the model from a reference system via a curve fitting analysis. (not shown). In one implementation, the plurality of system parameters can include a calibration temperature for the measured temperature at which the phase offset is expected to be zero, a thermal time constant associated with the driver associated with the sensor system, a thermal time constant associated with the driver associated with the illumination system, a thermal time constant associated with the driver associated with each temperature sensor, and a static coefficient representing an expected amount of change in the phase offset associated with a given deviation of the measured temperature from the calibration value when the temperature of the system is not in transition. At 182, a location of the point on the surface of interest, relative to the time of flight system, is determined from the received electromagnetic radiation and the calculated phase offset.
It will be appreciated that the method, including 172, 174, 176, 178, 180, and 182, can be iteratively repeated to provide locations of multiple locations on the surface of interest. It will be appreciated that the calculated phase offset at 180 can be refined over time to reflective changes in temperature as the time of flight system is operating. In one implementation, measured temperatures from 178 can be retained across iterations and utilized to provide a more accurate calculation of the phase offset at 180. For example, multiple temperature readings over time can be used to calculate a numerical derivative as part of the phase offset calculation.
The system 200 can includes a system bus 202, a processing unit 204, a system memory 206, memory devices 208 and 210, a communication interface 212 (e.g., a network interface), a communication link 214, a display 216 (e.g., a video screen), and an input device 218 (e.g., a keyboard and/or a mouse). The system bus 202 can be in communication with the processing unit 204 and the system memory 206. The additional memory devices 208 and 210, such as a hard disk drive, server, stand-alone database, or other non-volatile memory, can also be in communication with the system bus 202. The system bus 202 interconnects the processing unit 204, the memory devices 206-210, the communication interface 212, the display 216, and the input device 218. In some examples, the system bus 202 also interconnects an additional port (not shown), such as a universal serial bus (USB) port.
The processing unit 204 can be a computing device and can include an application-specific integrated circuit (ASIC). The processing unit 204 executes a set of instructions to implement the operations of examples disclosed herein. The processing unit can include a processing core.
The additional memory devices 206, 208 and 210 can store data, programs, instructions, database queries in text or compiled form, and any other information that can be needed to operate a computer. The memories 206, 208 and 210 can be implemented as computer-readable media (integrated or removable) such as a memory card, disk drive, compact disk (CD), or server accessible over a network. In certain examples, the memories 206, 208 and 210 can comprise text, images, video, and/or audio, portions of which can be available in formats comprehensible to human beings.
Additionally or alternatively, the system 200 can access an external data source or query source through the communication interface 212, which can communicate with the system bus 202 and the communication link 214.
In operation, the system 200 can be used to implement one or more parts of a time of flight measurement system. Computer executable logic for implementing the system control 126 resides on one or more of the system memory 206, and the memory devices 208, 210 in accordance with certain examples. The processing unit 204 executes one or more computer executable instructions originating from the system memory 206 and the memory devices 208 and 210. The term “computer readable medium” as used herein refers to a medium that participates in providing instructions to the processing unit 204 for execution, and can include either a single medium or multiple non-transitory media operatively connected to the processing unit 204.
The invention has been disclosed illustratively. Accordingly, the terminology employed throughout the disclosure should be read in an exemplary rather than a limiting manner. Although minor modifications of the invention will occur to those well versed in the art, it shall be understood that what is intended to be circumscribed within the scope of the patent warranted hereon are all such embodiments that reasonably fall within the scope of the advancement to the art hereby contributed.
Number | Date | Country | Kind |
---|---|---|---|
1200/CHE/2014 | Mar 2014 | IN | national |
This continuation application claims priority to U.S. patent application Ser. No. 14/642,289, filed Mar. 9, 2015, which application claims priority to Indian provisional patent application No. 1200/CHE/2014, filed 10 Mar. 2014, both applications of which are incorporated herein in their entirety.
Number | Name | Date | Kind |
---|---|---|---|
6987823 | Stark | Jan 2006 | B1 |
20100123895 | Reiner | May 2010 | A1 |
20160116592 | Hiromi | Apr 2016 | A1 |
Number | Date | Country | |
---|---|---|---|
20180031682 A1 | Feb 2018 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 14642289 | Mar 2015 | US |
Child | 15729060 | US |