Patent Application
20230062290
This application claims priority to European Patent Application No. 21194148 filed on Aug. 31, 2021, the content of which is incorporated by reference herein in its entirety.
The present disclosure relates to pixel binning in time-of-flight sensors. In particular, implementations relate to a method for pixel binning in a time-of-flight sensor, a switched device for a time-of-flight sensor and a computer program.
Time-of-flight (ToF) cameras can generate three dimensional (3D) images comprising information on a distance of an object with respect to the ToF camera. The lateral resolution of the image depends on the size and number of pixels in the pixel array of the ToF sensor. The resolution of the pixel array can be reduced by binning. Binning is the combination of at least two pixels to obtain a larger pixel. Binning is usually applied to improve the signal-to-noise ratio (SNR) of the image since more photons can be collected on the combined pixel area. Signal noise can be further improved, e.g., by implementing correlated double sampling (CDS) determining a difference between signals related to illuminated and unilluminated pixels.
However, performance of current ToF sensors using pixel binning reaches usually its limits due to increased noise in the readout stage. Further, sensors with CDS stage and differential readout stage are complicated to implement. Currently, signal processing at the readout stage requires much power which is desired to be reduced.
Hence, there is a desire for improved pixel binning in time-of-flight sensors.
This desire is addressed by the subject matter of the disclosed implementations.
Some implementations relate to a method for pixel binning in a time-of-flight sensor. The method includes receiving a first readout signal from a first column of a pixel array of the time-of-flight sensor and receiving a second readout signal from a second column of the pixel array. The method further includes combining the first readout signal and the second readout signal to obtain a common analog signal useable to determine a distance based on at least two pixels of the time-of-flight sensor. The proposed method may enable a column-wise combination of readout signals in the analog domain. A number of analog-to-digital conversions, and hence power consumption, for digitizing the common analog signal can be reduced. Since less analog-to-digital conversions may be needed, noise may be reduced and less space may be required for implementing analog-to-digital conversion. According to the proposed method, signal processing in combination with CDS and/or differential signals can be enabled with less complexity.
Some implementations relate to a switched device for a time-of-flight sensor. The switched device includes first circuitry configured to receive a first readout signal from a first column of a pixel array of the time-of-flight sensor. Further, the switched device includes a second circuitry configured to receive a second readout signal from a second column of the pixel array. The switched device further includes a switching circuit including at least a first switch. The switching circuit connects a signal path of the first circuitry and a signal path of the second circuitry such that the first readout signal and the second readout signal are combinable to a common analog signal. The switched device may enable a column-wise combination of readout signals in the analog domain. A number of analog-to-digital converters (ADCs), and hence a power consumption, can be reduced for digitizing the common analog signal. Since less ADCs may be needed, less noise may appear and less space may be required for digitizing the common analog signal. According to the proposed architecture, the switched device is useable in combination with CDS and/or differential signals with less complexity.
Some examples of apparatuses and/or methods will be described in the following by way of example only, and with reference to the accompanying figures, in which
Some examples are now described in more detail with reference to the enclosed figures. However, other possible examples are not limited to the features of these implementations described in detail. Other examples may include modifications of the features as well as equivalents and alternatives to the features. Furthermore, the terminology used herein to describe certain examples should not be restrictive of further possible examples.
Throughout the description of the figures same or similar reference numerals refer to same or similar elements and/or features, which may be identical or implemented in a modified form while providing the same or a similar function. The thickness of lines, layers and/or areas in the figures may also be exaggerated for clarification.
When two elements A and B are combined using an ‘or’, this is to be understood as disclosing all possible combinations, e.g., only A, only B as well as A and B, unless expressly defined otherwise in the individual case. As an alternative wording for the same combinations, “at least one of A and B” or “A and/or B” may be used. This applies equivalently to combinations of more than two elements.
If a singular form, such as “a”, “an” and “the” is used and the use of only a single element is not defined as mandatory either explicitly or implicitly, further examples may also use several elements to implement the same function. If a function is described below as implemented using multiple elements, further examples may implement the same function using a single element or a single processing entity. It is further understood that the terms “include”, “including”, “comprise” and/or “comprising”, when used, describe the presence of the specified features, integers, steps, operations, processes, elements, components and/or a group thereof, but do not exclude the presence or addition of one or more other features, integers, steps, operations, processes, elements, components and/or a group thereof.
The time-of-flight sensor (or 3D image sensor) comprises a pixel array having a plurality of pixels. A (first) number of pixels of the pixel array may be assigned to the first column and a (second) number of pixels of the pixel array may be assigned to the second column. At least one pixel from the first column provides the first readout signal and at least one pixel from the second column provides the second readout signal. The first and second readout signals are analog signals. The proposed method enables a combination of the first readout signal (related to the first column) and the second readout signal (related to the second column) in the analog domain. Accordingly, column-wise pixel binning (or in-column-binning) is provided. The resulting common analog signal can be further used to determine a distance with respect to the time-of-flight sensor. To calculate a distance, a further independent measurement or an independently determined further analog readout signal may be required. The further analog readout signal may be determined by using the same pair of binned pixels in subsequent measurement. Alternatively, the further analog readout signal may be determined using another pair of binned pixels with different modulation. The proposed method may enable a better sensor performance since a higher SNR can be achieved due to pixel binning. Pixel binning is accomplished column-wise, prior to the analog-to-digital conversion. Since a single common analog signal is provided, only one analog-to-digital conversion may be needed (rather than at least two analog-to-digital conversions for the two readout signals). Since less analog-to-digital conversions may be needed, power consumption and/or the size of the time-of-flight sensor (or processing circuitry thereof, e.g., readout circuitry) can be reduced. Further, the time-of-flight sensor (or the corresponding processing circuitry) may be implementable with less complexity, with less effort or with less components (which in turn may reduce error sources). Further, the proposed technique may be useable with differential signals and in conjunction with CDS (e.g., being implemented prior to provision of the common analog signal). The proposed method is suitable for implementation on column level, e.g., close to the pixel outputs.
The first readout signal is an analog signal from the first column. For example, the first readout signal may be an analog signal from a single pixel of the first column. According to another example, the first readout signal may be an analog signal from a plurality (e.g., two, three or more) of pixels from the first column. The plurality of pixels of the first column may be combined to a first group (e.g., by in-pixel-binning within the first column) supplying the first readout signal to be further processed according to the proposed method.
The second readout signal may be similar to the first readout signal with respect to the second column. Hence, the second readout signal may be an analog signal from one pixel or a plurality of pixels (assigned to a group) of the second column. For example, the second column may be next to (or a direct neighbor of) the first column. Accordingly, the one or more pixels of each column may be located next to each other in the pixel array. Since column-wise binning can be also implemented for a group of pixels of each column, the proposed method can be useable to achieve horizontal and vertical (e.g., 2×2, 3×3, 2×3 related to the sub-array of the pixel array) pixel binning. In other words, the method 100 optionally further comprises combining signals from at least two neighboring pixels of the first column to generate the first readout signal and combining signals from at least two neighboring pixels of the second column to generate the second readout signal. Further details concerning this aspect are described in conjunction with
As described above, the proposed method may be useable in combination with CDS. For example, the method 100 further comprises using CDS for the first readout signal and the second readout signal before combining the first readout signal and the second readout signal. By this, each of the readout signals may comprise less noise (e.g., kTC noise) since pixels outputs can be corrected by using a reference signal (or reset signal, being provided if pixels are not illuminated). Hence, the proposed method applying CDS may allow to reduce errors in the first and second readout signal, and thus in the common analog signal and the digital representation thereof. Further details concerning this aspect are described in conjunction with
The common analog signal can be obtained by combining the first and second readout signal. For example, combining 103 the first readout signal and the second readout signal comprises averaging the first readout signal and second readout signal. Averaging in the analog domain may be achievable by a switching circuit. For example, the method 100 further comprises controlling a switching circuit comprising at least a first switch connecting a signal path of the first readout signal and a signal path of the second readout signal to obtain the common analog signal. Further details to this aspect are described in conjunction with
As described above, the proposed method may enable to reduce the number of analog-to-digital conversions. For example, if the time-of-flight sensor provides one ADC for each column, only one (of the two) ADC(s) may be needed for converting the common analog signal. For example, the method 100 may comprise activating an ADC corresponding to the first readout signal and deactivating an ADC corresponding to the second readout signal or vice versa to generate a digital signal based on the common analog signal. Due to deactivation of one ADC (which might be not needed since only the (single) common analog signal is to be converted) a power consumption and/or errors (e.g., due to crosstalk between ADCs) can be reduced. Further details to this aspect are described in conjunction with
The first and second readout signals may be differential signals each comprising a first signal portion and a second signal portion. For differential signals, controlling the switching circuit (as described above) may enable a combination of the first portions of the first and second readout signals and a combination of the second portions of the first and second readout signals. Hence, column-wise pixel binning can be provided for each of the respective signal pairs of the differential readout signals. Accordingly, a proper differential common analog signal can be provided and further processed, e.g., converted to digital. Further details concerning this aspect are described in conjunction with
Apart from the first switch, the method may optionally control a second switch of the switching circuit. For example, the method 100 comprises controlling the at least first switch and a second switch of the switching circuit. The second switch may connect the first switch and the signal path of the second readout signal. The first and second switch of the switching circuit may be activated and deactivated simultaneously such that proper combination of the first readout signal and the second readout signal can be achieved. By controlling two switches in the switching circuit rather than only one switch in the switching circuit, signal routing, e.g., to an ADC, may be reduced and/or one ADC can be shared for the first readout signal and the second readout signal. A deactivation of another ADC might not be necessary. For example, the method 100 comprises generating a digital signal based on the common analog signal using a common ADC coupled to the switching circuit. Further details concerning this aspect are described in conjunction with
The proposed method 100 may enable an improved implementation of or improved processing in time-of-flight sensors. The time-of-flight sensor can be used to supply a 3D image providing distance information of an object (relative to the time-of-flight sensor) to be illuminated. With the proposed method 100 the 3D image may be of higher quality since SNR due to pixel binning may be improved.
Although not explicitly illustrated in
The switched device 200 may enable an (structural) implementation of the method 100 for pixel binning in a time-of-flight sensor described in connection with
The switched device 200 may enable binning of different (e.g., neighboring) columns 208, 209 of the pixel array 212 of the time-of-flight sensor 210. The switched device 200 may enable a combination of the output of, e.g., two differential pixels, while preserving a CDS operation for removal of kTC noise. The proposed implementation may use a switched capacitor network. Hence, no extra current consumption branch may be needed in the readout path and thus a low power consuming implementation can be provided.
Although not explicitly illustrated in
Since (a single) common analog signal is provided by the switching circuit 303, the switched device 300 may (only) comprise either an ADC 314 coupled to the first circuitry to generate a digital signal based on the common analog signal or an ADC 316 coupled to the second circuitry 202 to generate a digital signal based on the common analog signal. Alternatively, the switched device 300 may comprise both ADCs 314, 316 wherein only one ADC (e.g., ADC 314) is activated (or actively operating e.g., sampling and converting) and the other ADC (e.g., ADC 316) is deactivated (e.g., turned off or in stand-by mode). The former implementation using only one ADC may be suitable, e.g., to reduce the number of components (and thus expenses or errors). The latter implementation using two ADCs may be suitable, e.g., to enable usage of the switched device 300 in combination with the time-of-flight sensor 210 also if column-wise binning is not applied (no binning operation if the switching circuit 303 is deactivated or decoupling the first and second signal path). Since only one ADC is actively operating in both implementations, power consumption and errors (e.g., due to crosstalk between the first and second ADC) can be reduced. The ADC may be, e.g., a flash ADC, successive approximation ADC, delta-sigma ADC or any type of ADC suitable to sample and convert common analog signals being based on time-of-flight readout signals.
As illustrated, the first readout signal 405a-b and second readout signal 406a-b are differential signals each comprising a first signal portion 405a, 406a and a second signal portion 405b, 406b. The at least first switch 403a connects the signal path of the first circuitry 201 and the signal path of the second circuitry 202 such that the first portions 405a, 406a of the first and second readout signals are combinable. The switching circuit further comprises a further switch 403b connecting the signal path of the first circuitry 201 and the signal path of the second circuitry 202 such that the second portions 405b, 406b of the first and second readout signals are combinable. Hence, column-wise binning can be enabled by implementing at least one switch for a respective portion of the differential signals. Using the at least first switch 403a the first portion 405a of the first differential readout signal and the first portion 406a of the second differential readout signal can be combined to obtain a first portion of the common analog signal to be passed to either the activated ADC 314 corresponding to the first circuitry 201 (while the ADC 316 corresponding to the second circuitry 202 is deactivated) or to the activated ADC 316 corresponding to the second circuitry 202 (while the ADC 314 corresponding to the first circuitry 201 is deactivated). Similarly, using the further switch 403b the second portion 405b of the first differential readout signal and the second portion 406b of the second differential readout signal can be combined to obtain a second portion of the common analog signal to be passed to either the activated ADC 314 corresponding to the first circuitry 201 or to the activated ADC 316 related to the second circuitry 202.
As shown in
More details and aspects are mentioned in connection with the examples described above or below. The example shown in
The proposed implementation with the switching circuit 503 comprising the first and second switch 503a-b may be suitable to achieve faster signal routing of the common analog signal 207. In other words, by the proposed implementation the common analog signal 207 can be passed to a processing circuitry, e.g., ADC, by a shorter signal path. Due to the shorter signal path, power dissipation can be reduced. The shorter signal path may be achievable since the common analog signal might not need to be processed (e.g., passed) by either the first circuitry 201 or the second circuitry 202 for the subsequent analog-to-digital conversion. The common analog signal 207 can be rather directed to a common circuit (e.g., common ADC 518) which can be shared (or used) by both the first circuitry 201 and second circuitry 202.
The common analog signal 207 can be generated at a common node connecting the signal path of the first circuitry 201 and the signal path of the second circuitry 202 using activated first and second switches 503a-b. As illustrated, the common node may be located between the first switch 503a and the second switch 503b. Deactivation of both the first switch 503a and the second switch 503b may turn off a binning operation (or mode) of the switched device 500. In this case, the switched device 500 may be useable in a standard mode, e.g., by successively converting the first readout signal 201 and the second readout signal 202 by the common ADC 518.
As described, the switched device 500 comprises a common ADC 518. The common ADC 518 is coupled to the first switch 503a and the second switch 503b and is configured to generate a digital signal based on the common analog signal 207. Similarly, as described in conjunction with proposed implementations described above, only one ADC may be needed for reading out two columns of the time-of-flight sensor. Compared to the switched device 300 described in connection with
More details and aspects are mentioned in connection with the examples described above or below. The example shown in
As described above, the proposed technique may be useable in combination with a CDS stage, e.g., being implemented in the first and second circuitry 201, 202 of the switched device. For example, each (circuitry) of the first and second circuitry 201, 202 comprises a switched capacitor circuit enabling CDS for the corresponding first or second readout signal 205, 206.
As described above, a readout signal from a column of the pixel array may relate to one or more pixels of the column.
In some aspects, the proposed technique can be used for processing the data in analog domain, e.g., without additional power consumption penalty (since collected charges is only shifted to a common collection point).
In some aspects, the proposed technique may enable a faster read-out time.
In some aspects, the proposed technique may improve SNR since no (or only reduced readout noise) may be present by using CDS and column-wise binning.
The aspects and features described in relation to a particular one of the previous examples may also be combined with one or more of the further examples to replace an identical or similar feature of that further example or to additionally introduce the features into the further example.
Examples may further be or relate to a (computer) program including a program code to execute one or more of the above methods when the program is executed on a computer, processor or other programmable hardware component. Thus, steps, operations or processes of different ones of the methods described above may also be executed by programmed computers, processors or other programmable hardware components. Examples may also cover program storage devices, such as digital data storage media, which are machine-, processor- or computer-readable and encode and/or contain machine-executable, processor-executable or computer-executable programs and instructions. Program storage devices may include or be digital storage devices, magnetic storage media such as magnetic disks and magnetic tapes, hard disk drives, or optically readable digital data storage media, for example. Other examples may also include computers, processors, control units, (field) programmable logic arrays ((F)PLAs), (field) programmable gate arrays ((F)PGAs), graphics processor units (GPU), application-specific integrated circuits (ASICs), integrated circuits (ICs) or system-on-a-chip (SoCs) systems programmed to execute the steps of the methods described above.
It is further understood that the disclosure of several steps, processes, operations or functions disclosed in the description or claims shall not be construed to imply that these operations are necessarily dependent on the order described, unless explicitly stated in the individual case or necessary for technical reasons. Therefore, the previous description does not limit the execution of several steps or functions to a certain order. Furthermore, in further examples, a single step, function, process or operation may include and/or be broken up into several sub-steps, -functions, -processes or -operations.
If some aspects have been described in relation to a device or system, these aspects should also be understood as a description of the corresponding method. For example, a block, device or functional aspect of the device or system may correspond to a feature, such as a method step, of the corresponding method. Accordingly, aspects described in relation to a method shall also be understood as a description of a corresponding block, a corresponding element, a property or a functional feature of a corresponding device or a corresponding system.
The following claims are hereby incorporated in the detailed description, wherein each claim may stand on its own as a separate example. It should also be noted that although in the claims a dependent claim refers to a particular combination with one or more other claims, other examples may also include a combination of the dependent claim with the subject matter of any other dependent or independent claim. Such combinations are hereby explicitly proposed, unless it is stated in the individual case that a particular combination is not intended. Furthermore, features of a claim should also be included for any other independent claim, even if that claim is not directly defined as dependent on that other independent claim.
| Number | Date | Country | Kind |
|---|---|---|---|
| 21194148 | Aug 2021 | EP | regional |
