Patent Grant
10847556
The present disclosure relates to a solid-state imaging apparatus and an imaging apparatus.
For conventional solid-state imaging apparatuses, a solid-state imaging apparatus as shown in, for example, Japanese Unexamined Patent Application Publication No. 2007-266556 has been proposed for increasing dynamic range. The solid-state imaging apparatus in Japanese Unexamined Patent Application Publication No. 2007-266556 includes high-sensitivity pixels and low-sensitivity pixels that are lower in sensitivity than the high-sensitivity pixels, and in, for example, paragraph 0059, “attaining a broad dynamic range from both the low sensitivity and high sensitivity by forming deep charge storage regions at the high-sensitivity pixels and shallow charge storage regions at the low-sensitivity pixels” has been proposed.
According to Japanese Unexamined Patent Application Publication No. 2007-266556, however, balancing dynamic range expansion and flicker reduction poses a problem.
Flicker reduction will be described first. In recent years, light-emitting diode (LED) light sources and laser diode (LD) light sources have become widespread. These light sources often use dynamic lighting in which LEDs are repeatedly turned on and off at a speed unperceivable by the human eye. In other words, flicker occurs at a speed unperceivable by the human eye. For example, LED light sources are used for, aside from luminaires, traffic lights, headlights and brake lights in vehicles, and the like. With dynamic lighting, light sources appear as static lighting to the human eye, but flicker causes interference when capturing these using a solid-state imaging apparatus. When a solid-state imaging apparatus captures such light sources or illumination environments using such light sources, there are cases where an image is obtained in which the light sources are on (or a bright image is obtained) and cases where an image is obtained in which the light sources are off (or a dark image is obtained). In other words, flicker occurs in the captured image itself. The latter, i.e., cases where an image is obtained in which the light sources are off (or a dark image is obtained) can be referred to as imaging faults. Hereinafter, controlling imaging faults that are the cause of such flicker is referred to as flicker reduction.
In Japanese Unexamined Patent Application Publication No. 2007-266556, high-sensitivity pixels and a long exposure time are used during low illuminance, and low-sensitivity pixels and a short exposure time are used during high illuminance in order to increase dynamic range.
Accordingly, flicker reduction is difficult since the exposure time during high illuminance is short. On the other hand, flicker reduction is possible by making the exposure time longer, but dynamic range will decrease due to pixel saturation.
However, flicker reduction can be anticipated since the exposure time during low illuminance is long, but dynamic range is low due to pixel saturation. On the other hand, the problem of dynamic range becoming low can be limited by making the exposure time shorter, but flicker reduction cannot be anticipated.
The present disclosure, which takes the above problem into account, provides a solid-state imaging apparatus and an imaging apparatus that balance flicker reduction and an increase in dynamic range.
In order to solve the above problem, a solid-state imaging apparatus in the present disclosure includes a plurality of high-sensitivity pixels that are arranged in a matrix, and perform a photoelectric conversion at a predetermined sensitivity; a plurality of low-sensitivity pixels that are arranged in a matrix in gaps between the plurality of high-sensitivity pixels, and perform a photoelectric conversion at a lower sensitivity than the predetermined sensitivity; and a signal processor that generates a pixel signal by (i) detecting a difference signal between a signal from the plurality of high-sensitivity pixels and a signal from the plurality of low-sensitivity pixels, and (ii) correcting the signal from the plurality of high-sensitivity pixels using the difference signal.
The solid-state imaging apparatus and the imaging apparatus in the present disclosure can balance flicker reduction and an increase in dynamic range.
These and other objects, advantages and features of the disclosure will become apparent from the following description thereof taken in conjunction with the accompanying drawings that illustrate a specific embodiment of the present disclosure.
Circumstances Leading Up to Present Disclosure
The problem stated in the description of the related art will first be described with reference to
Generally, as in Japanese Unexamined Patent Application Publication No. 2007-266556, pixels with a different sensitivity are disposed and a correct image is output by processing an output signal of these pixels in order to reduce LED flicker.
Here, the first problem is loss of LED light (light emitted by an LED being repeatedly on and off, pulsed light) (problem 1). The cause for this is an incongruity between exposure timing and LED lighting timing due to using rolling shutter, especially when the exposure time is short during in bright areas (cause 1). This can be solved by making the exposure time of low-sensitivity pixels with a high saturation-sensitivity ratio longer (for example, one vertical scanning period) (solution 1).
The second problem is pixel saturation (problem 2). The cause for this is that the pixels become saturated and a white balance is lost when the LED flicker duty is too high or when the exposure time is too long (cause 2). With regard to this, pixels with different saturation-sensitivity ratios are disposed, pixel saturation is made to occur less easily, and the dynamic range of the low-sensitivity pixels is increased (solution 2).
In other words, the low-sensitivity pixels that capture LED light may be disposed as pixels with static exposure (solution 1) and different saturation-sensitivity ratios (solution 2) with regard to the above problems 1 and 2.
The exposure time and gain parameters of the low-sensitivity pixels that capture this discontinuous LED light may be configured separately from the exposure time and gain parameters of the high-sensitivity pixels that capture continuous LED light.
This makes it possible to completely separately configure parameters of the flicker reduction and WDR functionality or balance these settings when capturing transportation machinery (to give an example, a vehicle) using the solid-state imaging apparatus.
In this manner, it is possible to balance dynamic range expansion and flicker reduction since the solid-state imaging apparatus includes high-sensitivity pixels and low-sensitivity pixels having different sensitivities, the high-sensitivity pixels are exposed in parallel with the low-sensitivity pixels, an exposure time of the low-sensitivity pixels is longer than an exposure time of the high-sensitivity pixels, and one pixel signal is generated by correcting the signal from the high-sensitivity pixels with the signal from the low-sensitivity pixels. As for the specifications of these low-sensitivity pixels, a photogate structure may be used for increasing the saturation level, a neutral-density (ND) filter (gray filter) may be used for decreasing the sensitivity, and the saturation-sensitivity ratio may also be increased.
According to this method, during low illuminance for example, the solid-state imaging apparatus increases a gain inside the pixels by using high-sensitivity pixels and make sure that a voltage (pixel signal) does not increase depending on an amount of light received with respect to noise produced by a pixel amplification transistor or analog circuit. In other words, it is possible to output a high-quality image with a high signal-to-noise ratio (S/N) with respect to dark subjects.
During high illuminance, however, it is possible to increase the dynamic range and have a long exposure time by using low-sensitivity pixels with a high saturation-sensitivity ratio and lowering the gain inside the pixels. In other words, it is possible to output an image accurately reproducing gradients of a subject without blown out highlights and further reducing flicker with respect to bright subjects.
Accordingly, the inventors have conceived a solid-state imaging apparatus that balances image quality improvement during low illuminance, dynamic range expansion during high illuminance, and flicker reduction by using low-sensitivity pixels with a high saturation-sensitivity ratio as the technique for increasing dynamic range and reducing flicker.
Hereinafter, the solid-state imaging apparatus according to embodiments in the present disclosure will be described with reference to the drawings.
However, unnecessarily detailed descriptions may be omitted.
For example, detailed descriptions of well-known matters or descriptions of components that are substantially the same as components described previous thereto may be omitted. This is to avoid redundancy and facilitate understanding of the descriptions for those skilled in the art. Note that the accompanying drawings and subsequent descriptions are provided to facilitate sufficient understanding of the present disclosure by those skilled in the art, and are thus not intended to limit the scope of the subject matter recited in the claims. In other words, the subsequent embodiments show a specific example in the present disclosure, and numerical values, shapes, components, placement and connection of the components, order of processes, and the like are mere examples and are not intended to limit the present disclosure.
A configuration example of a solid-state imaging apparatus according to Embodiment 1 will first be described.
Configuration Example of Solid-State Imaging Apparatus 1
Solid-state imaging apparatus 1 shown in the drawing includes pixel array 10, horizontal scanning circuit 12, vertical scanning circuit 14, vertical signal lines 19, timing controller 20, column processor 26, reference signal generator 27, output circuit 28, and signal processor 70. Solid-state imaging apparatus 1 further includes a master clock (MCLK) terminal that exteriorly receives a master clock input signal, a DATA terminal for exteriorly transceiving a command or data, a D1 terminal for exteriorly transmitting video data, and the like, and also includes various types of terminals to which a power-supply voltage and a ground voltage are supplied.
Pixel array 10 includes pixel circuits 3 arranged in a matrix. Pixel circuits 3 are arranged in n rows and m columns in
Horizontal scanning circuit 12 outputs an analog-to-digitally (A/D) converted pixel signal to horizontal signal lines 18 by scanning memory 256 in each of column A/D circuits 25 one by one. This scanning may be performed in the same order as column A/D circuits 25 are arranged.
Vertical scanning circuit 14 scans horizontal scan line group (also referred to as row control line group) 15 disposed per row of pixel circuits 3 in pixel array 10 in row units. With this, vertical scanning circuit 14 selects pixel circuits 3 per row unit and causes the pixel signal from pixel circuits 3 belonging to the selected row to be simultaneously output to m vertical signal lines 19. The same number of horizontal scan line groups 15 are disposed as there are rows of pixel circuits 3. In
Vertical signal line 19 is disposed per column of pixel circuits 3 in pixel array 10 and propagates the pixel signal from pixel circuits 3 belonging to the selected row to column A/D circuits 25. Vertical signal lines 19 include m vertical signal lines H1 to Hm. Analog-to-digital converter (ADC) input lines include m ADC input lines ADIN1 to ADINm.
Timing controller 20 controls the entirety of solid-state imaging apparatus 1 by generating various control signal groups. The various control signal groups include control signal groups CN1, CN2, CN4, CN5, CN8, and counter clock signal CK0. For example, timing controller 20 receives master clock MCLK via a terminal; generates various internal clocks; and controls horizontal scanning circuit 12, vertical scanning circuit 14, and the like.
Column processor 26 includes column A/D circuit 25 per column. Column A/D circuits 25 each A/D convert the pixel signal from vertical signal lines 19.
Column A/D circuits 25 each include voltage comparator 252, counter 254, and memory 256.
Voltage comparator 252 (i) compares the analog pixel signal from vertical signal lines 19 with reference signal RAMP that is generated by reference signal generator 27 and includes a ramp waveform (i.e., triangular wave), and, for example, (ii) inverts the output signal indicating the comparison result when the former is greater than the latter.
Counter 254 counts the time from when the triangular waves in reference signal RAMP start to change to when the output signal of voltage comparator 252 is inverted. Since the time up to the inverting is determined in accordance with the value of the analog pixel signal, a count value is digitalized pixel signal.
Memory 256 retains the count value of counter 254, i.e., the digital pixel signal.
Reference signal generator 27 generates reference signal RAMP including triangular waves, and outputs reference signal RAMP to a plus input terminal of voltage comparator 252 in each of column A/D circuits 25.
Output circuit 28 outputs the digital pixel signal to video data terminal D1.
Signal processor 70 includes corrector 75 and WDR synthesis circuit 76, and generates a pixel signal by (i) detecting a difference signal between a signal from the high-sensitivity pixels and a signal from the low-sensitivity pixels, and (ii) correcting the signal from the high-sensitivity pixels using the difference signal. This enables signal processor 70 to balance dynamic range expansion and flicker reduction.
Configuration Example of Pixel Circuit 3
A configuration example of pixel circuit 3 will be described next.
High-sensitivity pixel PD is a photoelectric conversion element, e.g. a photodiode, and performs the photoelectric conversion at the predetermined sensitivity, i.e., generates an electric charge depending on the amount of light received.
Low-sensitivity pixel K is a photoelectric conversion element, e.g. a photodiode, and performs the photoelectric conversion at a sensitivity lower than the predetermined sensitivity. For example, a light-receiving area of low-sensitivity pixel K is smaller than a light-receiving area of high-sensitivity pixel PD.
Low-sensitivity pixel K includes control electrode PG that covers a portion of low-sensitivity pixel K. Control electrode PG controls a potential of a surface of low-sensitivity pixel K in accordance with a voltage applied to control electrode PG (in the drawing, control voltage ΦPG). This enables control electrode PG to increase and decrease a saturation level of low-sensitivity pixel K.
Floating diffusion layer FD retains signal charge (e.g. an electron) forwarded from high-sensitivity pixel PD or low-sensitivity pixel K, converts the retained signal charge to a voltage, and supplies the converted voltage to a gate of amplification transistor SF. A substantive capacitance of floating diffusion layer FD does not only include a capacitance of floating diffusion layer FD itself, but also gate capacitance Cfd1 of amplification transistor SF, gate/drain capacitance Cp1 of amplification transistor SF, and stray capacitance Cfd2 of a drain wire of gain control transistor GC when gain control transistor GC is on.
First transfer transistor TR1 is a switch transistor that is turned on and off in accordance with readout control signal ΦTR1. First transfer transistor TR1 forwards (i.e., reads out) signal charge photoelectrically converted by high-sensitivity pixel PD to floating diffusion layer FD when readout control signal ΦTR1 is at a high level.
Second transfer transistor TR2 is a switch transistor that is turned on and off in accordance with readout control signal ΦTR2. Second transfer transistor TR2 forwards (i.e., reads out) signal charge photoelectrically converted by low-sensitivity pixel K to floating diffusion layer FD via gain control transistor GC when readout control signal ΦTR1 is at a high level.
Reset transistor RS is a switch transistor that is turned on and off in accordance with reset control signal ΦRS. Reset transistor RS sets floating diffusion layer FD to the power-supply voltage applied to the drain as a reset level via gain control transistor GC when reset control signal ΦRS is at a high level. In other words, reset transistor RS resets floating diffusion layer FD to the reset level when reset control signal (RS is at a high level.
Gain control transistor GC electrically disconnects or connects second transfer transistor TR2 and floating diffusion layer FD in accordance with gain control signal ΦGC. With this, a conversion gain that converts signal charge to a voltage in floating diffusion layer FD is changed. In other words, the conversion gain of floating diffusion layer FD increases upon turning off gain control transistor GC when the signal charge is being forwarded from high-sensitivity pixel PD to floating diffusion layer FD. The conversion gain of floating diffusion layer FD decreases upon turning on gain control transistor GC when the signal charge is being forwarded from low-sensitivity pixel K to floating diffusion layer FD.
Amplification transistor SF includes a source follower load that pairs current source 30 connected to vertical signal line 19 and amplification transistor SF, and outputs the voltage of the gate, i.e., the voltage of floating diffusion layer FD, to vertical signal line 19 as the analog pixel signal.
Selection transistor SEL is a switch transistor that is turned on and off in accordance with selection control signal (SEL. Selection transistor SEL electrically connects a source of amplification transistor SF and vertical signal line 19 when selection control signal (SEL is at a high level.
Note that pixel circuit 3 shown in
Readout Operation Example Using Correlated Double Sampling
An operation in which the reset level and signal level of the analog pixel signal are read out and measured using correlated double sampling (CDS) will be described next.
Reference signal RAMP has triangular waves at a count-down period (t10 to t14) and a count-up period (t20 to t24) in
The count-down period is a period for A/D converting reset component Vrst of the analog pixel signal output from amplification transistor SF. The time from when the count-down period starts (when the triangular wave starts changing) up to when the output of voltage comparator 252 is inverted is counted down by counter 254. This count value is an A/D conversion result of reset component Vrst of the analog pixel signal.
The count-up period is a period for A/D converting a level of a data component (signal component Vsig+reset component Vrst) of the analog pixel signal output from amplification transistor SF. The time from when the count-up period starts (when the triangular wave starts changing) up to when the output of voltage comparator 252 is inverted is counted up by counter 254. This count-up converts the data component (Vsig+Vrst) of the analog pixel signal to a digital value. Since this count-up sets a count-down value indicating reset component Vrst as an initial value, a count value when the count-up period has finished represents a result of the CDS which subtracts the reset component from the data component. In other words, the count value when the count-up period has finished is a digital value representing signal component Vsig. In this manner, column A/D circuits 25 eliminate any variations in each of the columns, e.g. clock skew or counter delay, leading to measurement errors, and extract only the proper signal component Vsig, i.e., perform a digital CDS.
Readout Operation with Gain Control
In image (a) of
In image (b) of
In this manner, when gain control transistor GC is at a low level during the readout, the conversion gain of floating diffusion layer FD increases due to gain control signal (GC. On the other hand, when gain control transistor GC is at a high level during the readout, the conversion gain of floating diffusion layer FD decreases.
Low-sensitivity pixel K can make the signal charge amount, which is accumulable proximate to the surface of low-sensitivity pixel K by control voltage (PG, and the saturation level variable. As shown in image (b) of
The saturation level and the sensitivity of low-sensitivity pixel K will be described next with reference to
The charge storage region of low-sensitivity pixel K includes (i) a deeply formed region similar to conventional photodiodes without control electrode PG and (ii) a new region formed proximate to a surface thereof by control electrode PG. When the voltage of control voltage ΦPG is low, an electric charge accumulates mainly in the deep region, and when the voltage of control voltage ΦPG increases, an electric charge accumulates in the shallow region proximate to the surface. In this manner, the saturation level of low-sensitivity pixel K is variable in accordance with the potential of control voltage ΦPG as shown in
The sensitivity of low-sensitivity pixel K can be adjusted by laminating a ND filter, e.g. gray filter, to an (R, Gr, B, Gb) on-chip filter and adjusting a thickness and laminated region of this filter. An alternative method to the ND filter is performing a metal shielding and adjusting the area of an aperture area and sensitivity. The shutter and readout use a rolling motion. As shown in
Since it is possible to adjust the saturation level and the sensitivity level separately, the slope of the charge storage level and the saturation level can be set to a suitable value with respect to the illuminance and the exposure time. As a result, the saturation-sensitivity ratio can be adjusted and the dynamic range of the pixels can be freely adjusted.
Pixel Arrangement Example
A pixel arrangement of solid-state imaging apparatus 1 will be described next.
The four high-sensitivity pixels PD (R, Gr, B, Gb in the drawing) and the four low-sensitivity pixels K corresponding thereto form a basic Bayer arrangement. The sensitivity can be decreased by laminating the gray filter to the (R, Gr, B, Gb) on-chip filter and causing the incident light to attenuate.
This makes it possible to make the saturation level of low-sensitivity pixel K higher than that of high-sensitivity pixel PD and the sensitivity of low-sensitivity pixel K lower than that of high-sensitivity pixel PD.
This makes it possible to increase the dynamic range of a light signal of the low-sensitivity pixel, and, for example, to receive a signal transmitted from a signaler with a comparatively low illuminance to a headlight with a high illuminance in the same frame with respect to a light-emitting source repeatedly turned on and off (pulsed light-emitting source, LED light source).
Note that the two pixels per cell pixel circuit 3 shown in
Exposure Operation of High-Sensitivity Pixel PD and Low-Sensitivity Pixel K
An exposure operation of high-sensitivity pixel PD and low-sensitivity pixel K in solid-state imaging apparatus 1 will be described next.
Solid-state imaging apparatus 1 in the present embodiment generates, within one vertical scanning period, high-sensitivity frames by performing an exposure operation a plurality of times, each having a different exposure time, on high-sensitivity pixels PD, and low-sensitivity frames by performing an exposure operation a plurality of times, each having a different exposure time, on low-sensitivity pixels K Signal processor 70 corrects a signal of each of the high-sensitivity frames using the signal from low-sensitivity pixels K, and synthesizes the corrected high-sensitivity frames.
Solid-state imaging apparatus 1, as illustrated in
In this manner, solid-state imaging apparatus 1 according to the present embodiment exposes high-sensitivity pixel PD three times, exposure times tL, tM, and tS, which are split up with different durations, for increasing the dynamic range of high-sensitivity pixel PD. Low-sensitivity pixel K is exposed for the longer exposure time tK in parallel with high-sensitivity pixel PD.
The LED light in
Note that
By making exposure time tK of low-sensitivity pixel K roughly the same duration as one vertical scanning period, the lighting signal can reliably be captured with low-sensitivity pixel K when capturing a light-emitting source repeatedly and cyclically turned on and off (pulsed light-emitting source, LED light source).
With regard to the problem of the pixel saturation, external light changing within a range from low illuminance to high illuminance (pulsed light, LED light, LD light) can be captured in photodiode K of the low-sensitivity pixel having a high saturation-sensitivity ratio compared to high-sensitivity pixel PD. In other words, with regard to the lighting signal problem, light can be continuously received along the time axis, and with regard to the dynamic range problem, low-sensitivity pixel K can receive light more broadly.
Note that high saturation-sensitivity ratio means the following. Low-sensitivity pixel K has a higher saturation level and/or lower sensitivity than high-sensitivity pixel PD.
Configuration Example of Signal Processor 70
A configuration of signal processor 70 will be described next.
As illustrated in
Low-sensitivity signal generating circuit 743 generates a signal of a new third low-sensitivity frame by multiplying pixel signal (K) of low-sensitivity pixel K by a third coefficient. The third coefficient is for scale aligning pixel signal (K) of low-sensitivity pixel K to pixel signal (S) of the third high-sensitivity frame. For example, the third coefficient is determined depending on exposure time tK, second exposure time tM, the gain and sensitivity of high-sensitivity pixel PD, and the gain and sensitivity of low-sensitivity pixel K.
Correction circuit 752 generates a second correction signal by detecting a difference signal between pixel signal (M) from the second high-sensitivity frame and a signal from the second low-sensitivity frame generated by low-sensitivity signal generating circuit 742, and correcting pixel signal (M) from the second high-sensitivity frame using the difference signal.
As illustrated in
Difference detection circuit 757 detects the difference signal between pixel signal (M) from the second high-sensitivity frame and the signal from the second low-sensitivity frame.
Addition circuit 758 adds the difference signal from difference detection circuit 757 to pixel signal (M) of the second high-sensitivity frame, and outputs the addition result as the second correction signal.
Correction circuit 753 generates a third correction signal by detecting a difference signal between pixel signal (S) from the third high-sensitivity frame and a signal from the third low-sensitivity frame, and correcting pixel signal (S) from the third high-sensitivity frame using the difference signal. Correction circuit 753 detects a difference between pixel signal (S) of second high-sensitivity frame and the signal of the third low-sensitivity frame, adds the difference to pixel signal (S), and outputs the addition result as the third correction signal.
WDR synthesis circuit 76 generates a synthesized pixel signal by synthesizing pixel signal (L) from the first high-sensitivity frame, the second correction signal from correction circuit 752, and the third correction signal from correction circuit 753.
As illustrated in
However, first high-sensitivity pixel frame (L), first high-sensitivity frame (M), and third high-sensitivity frame (S) are generated from high-sensitivity pixel PD by implementing the first exposure operation, the second exposure operation, and the third exposure operation, each having a different exposure time.
Subsequently, corrector 75 (i) corrects second high-sensitivity frame (M) and third high-sensitivity frame (S) with, respectively, the second low-sensitivity frame and the third low-sensitivity frame, and (ii) generates a corrected second high-sensitivity frame (M) and a corrected third high-sensitivity frame (S).
Subsequently, WDR synthesis circuit 76 synthesizes pixel signal (L) from the first high-sensitivity frame that is not corrected, corrected pixel signal (M) from the second high-sensitivity frame that is corrected, and corrected pixel signal (S) from the third high-sensitivity frame that is corrected. This synthesis makes it possible to implement the WDR functionality, i.e., increase the dynamic range.
A characteristics example and the dynamic range of the synthesized pixel signal of high-sensitivity pixel PD and low-sensitivity pixel K will be described next.
In
In this manner, with regard to the exposure time, pixel signal (L), pixel signal (M), pixel signal (S), which are split up with different exposure times, are generated from high-sensitivity pixel PD for balancing a high S/N and the WDR. For example, exposure time tL:tM:tS may be approximately 256:16:1. Low-sensitivity pixel K may be exposed in parallel with high-sensitivity pixel PD, and the exposure time may be tK>tL+tM+tS. tK may have, for example, roughly the same duration as one vertical synchronization period, and may be between approximately 16.6 ms (= 1/60 fps) of one vertical synchronization period and approximately 11 ms (= 1/90 Hz) corresponding to a minimum frequency of LEDs.
This makes it possible to obtain a pixel signal that balances flicker reduction and dynamic range expansion.
Solid-state imaging apparatus 1 in the above-described Embodiment 1 includes a plurality of high-sensitivity pixels that are arranged in a matrix, and perform a photoelectric conversion at a predetermined sensitivity; a plurality of low-sensitivity pixels that are arranged in a matrix in gaps between the plurality of high-sensitivity pixels, and perform a photoelectric conversion at a lower sensitivity than the predetermined sensitivity; and signal processor 70 that generates a pixel signal by (i) detecting a difference signal between a signal from the plurality of high-sensitivity pixels and a signal from the plurality of low-sensitivity pixels, and (ii) correcting the signal from the plurality of high-sensitivity pixels using the difference signal.
This makes it possible to easily reduce flicker and increase the dynamic range.
The high-sensitivity pixels may be exposed in parallel with the low-sensitivity pixels. An exposure time of the low-sensitivity pixels may be longer than an exposure time of the high-sensitivity pixels, shorter than one vertical synchronization period, and longer than a predetermined flicker cycle.
This makes it possible to sufficiently reduce flicker with respect to flicker cycles that are shorter than a predetermined flicker cycle. The predetermined flicker cycle may be the longest flicker cycle of a normal LED light source, e.g. an LED traffic light.
Signal processor 70 may add the difference signal to the signal from the high-sensitivity pixels as the correcting of the signal from the high-sensitivity pixels.
Since this enables the correcting to be a simple addition and subtraction without conditional determination, the correcting is suitable for simplifying and speeding up the configuration of the correction circuit.
Solid-state imaging apparatus 1 may generate, within one vertical scanning period, high-sensitivity frames by performing an exposure operation a plurality of times, each having a different exposure time, on the high-sensitivity pixels. Signal processor 70 may correct a signal of each of the high-sensitivity frames using the signal from the low-sensitivity pixels, and synthesize the corrected high-sensitivity frames.
With this, the dynamic range can easily be increased using the signals, each having a different exposure time in one vertical synchronization period, from the high-sensitivity pixels. Flicker can also be reduced thanks to the correcting.
Solid-state imaging apparatus 1 may generate, within one vertical scanning period, a first high-sensitivity frame, a second high-sensitivity frame, and a third high-sensitivity frame by respectively performing a first exposure operation that exposes the high-sensitivity pixels for a first exposure time, a second exposure operation that exposes the high-sensitivity pixels for a second exposure time, and a third exposure operation that exposes the high-sensitivity pixels for a third exposure time. The first exposure time may be longer than the second exposure time and the second exposure time may be longer than the third exposure time. Signal processor 70 may generate (i) a second low-sensitivity frame and a third low-sensitivity frame by multiplying a frame including the signal from the low-sensitivity pixels by, respectively, a second coefficient corresponding to the second exposure time and a third coefficient corresponding to the third exposure time, (ii) a second correction signal by detecting a difference signal between a signal from the second high-sensitivity frame and a signal from the second low-sensitivity frame, and correcting the signal from the second high-sensitivity frame using the difference signal, (iii) a third correction signal by detecting a difference signal between a signal from the third high-sensitivity frame and a signal from the third low-sensitivity frame, and correcting the signal from the third high-sensitivity frame using the difference signal, and (iv) the pixel signal by synthesizing a signal from the first high-sensitivity frame, the second correction signal and the third correction signal.
With this, the dynamic range can easily be increased using the three signals, each having a different exposure time in one vertical synchronization period, from the high-sensitivity pixels. Flicker can also be reduced thanks to the correcting.
Solid-state imaging apparatus 1 may include pixel circuits 3 arranged in a matrix. Pixel circuits 3 may each include at least one high-sensitivity pixel; floating diffusion layer FD that retains signal charge; a first transfer transistor that is disposed corresponding to the at least one high-sensitivity pixel, and transfers signal charge of the at least one high-sensitivity pixel to floating diffusion layer FD; at least one low-sensitivity pixel; a second transfer transistor that is disposed corresponding to the at least one low-sensitivity pixel, and transfers signal charge of the at least one low-sensitivity pixel to floating diffusion layer FD; and transistor GC that electrically disconnects or connects the second transfer transistor and floating diffusion layer FD.
Solid-state imaging apparatus 1 may further include control electrode PG that is disposed at each low-sensitivity pixel, and covers a portion of the low-sensitivity pixels. Control electrode PG may control a potential of a surface of each of the low-sensitivity pixels in accordance with a voltage applied to control electrode PG.
Another configuration example of signal processor 70 in Embodiment 1 will be described next.
Low-sensitivity signal generating circuit 741 generates the first low-sensitivity frame from pixel signal (K) of low-sensitivity pixel K. To be specific, signal processor 70 generates a first low-sensitivity frame, a second low-sensitivity frame and a third low-sensitivity frame by multiplying a frame including pixel signal (K) from low-sensitivity pixel K by, respectively, a first coefficient corresponding to first exposure time tL, a second coefficient corresponding to second exposure time tM, and a third coefficient corresponding to third exposure time tS. The first coefficient is for scale aligning pixel signal (K) of low-sensitivity pixel K to pixel signal (L) of the first high-sensitivity frame. For example, the first coefficient is determined depending on exposure time tK, first exposure time tL, the gain and sensitivity of high-sensitivity pixel PD, and the gain and sensitivity of low-sensitivity pixel K.
Correction circuit 751 has the same internal configuration as correction circuit 752, and generates a first correction signal by detecting a difference signal between pixel signal (L) from the first high-sensitivity frame and a signal from the first low-sensitivity frame generated by low-sensitivity signal generating circuit 741, and correcting pixel signal (L) from the first high-sensitivity frame using the difference signal.
WDR synthesis circuit 76 generates a synthesized pixel signal by synthesizing the first correction signal from correction circuit 751, the second correction signal from correction circuit 752, and the third correction signal from correction circuit 753.
As illustrated in
However, the first high-sensitivity frame, the second high-sensitivity frame, and the third high-sensitivity frame, whose exposure time are changed, are generated from high-sensitivity pixel PD.
Subsequently, corrector 75 (i) corrects first high-sensitivity frame (L), second high-sensitivity frame (M), and third high-sensitivity frame (S) with, respectively, the first low-sensitivity frame, the second low-sensitivity frame, and the third low-sensitivity frame, and (ii) generates a corrected first high-sensitivity frame (L), a corrected second high-sensitivity frame (M) and a corrected third high-sensitivity frame (S).
Subsequently, WDR synthesis circuit 76 synthesizes corrected pixel signal (L) from the corrected first high-sensitivity frame, corrected pixel signal (M) from the second high-sensitivity frame that is corrected, and corrected pixel signal (S) from the third high-sensitivity frame that is corrected. This synthesis makes it possible to implement the WDR functionality, i.e., increase the dynamic range.
This makes it possible to obtain a pixel signal that balances flicker reduction and dynamic range expansion. The synthesized pixel signal in
Solid-state imaging apparatus 1 in another configuration example in the above Embodiment 1 generates, within one vertical scanning period, a first high-sensitivity frame, a second high-sensitivity frame, and a third high-sensitivity frame by respectively performing a first exposure operation that exposes the high-sensitivity pixels for a first exposure time, a second exposure operation that exposes the high-sensitivity pixels for a second exposure time, and a third exposure operation that exposes the high-sensitivity pixels for a third exposure time. The first exposure time is longer than the second exposure time and the second exposure time is longer than the third exposure time. Signal processor 70 generates (i) a second low-sensitivity frame and a third low-sensitivity frame by multiplying a frame including the signal from the low-sensitivity pixels by, respectively, a second coefficient corresponding to the second exposure time and a third coefficient corresponding to the third exposure time, (ii) a second correction signal by detecting a difference signal between a signal from the second high-sensitivity frame and a signal from the second low-sensitivity frame, and correcting the signal from the second high-sensitivity frame using the difference signal, (iii) a third correction signal by detecting a difference signal between a signal from the third high-sensitivity frame and a signal from the third low-sensitivity frame, and correcting the signal from the third high-sensitivity frame using the difference signal, and (iv) the pixel signal by synthesizing a signal from the first high-sensitivity frame, the second correction signal and the third correction signal.
Solid-state imaging apparatus 1 in Embodiment 2 will be described next. A configuration example of solid-state imaging apparatus 1 including low-sensitivity pixels K1 and K2 instead of low-sensitivity pixel K will be described in Embodiment 2.
Pixel circuit 3 in
High-sensitivity pixels PD1 and PD2 and low-sensitivity pixels K1 and K2 are each light-receiving elements that perform a photoelectric conversion, and generate an electric charge depending on the amount of light received. High-sensitivity pixel PD1 and high-sensitivity pixel PD2 have the same sensitivity. Low-sensitivity pixel K1 and low-sensitivity pixel K2 each have a different sensitivity. High-sensitivity pixel PD1 and low-sensitivity pixel K1 form a pair, and high-sensitivity pixel PD2 and low-sensitivity pixel K2 form a pair.
Floating diffusion layer FD temporarily retains each of the electric charges read out by readout transistors TR11, TR12, TR21, and TR22 from high-sensitivity pixels PD1 and PD2 and low-sensitivity pixels K1 and K2.
In image (a) of
In image (b) of
In this manner, when gain control transistor GC is at a low level during the readout, the conversion gain of floating diffusion layer FD increases due to gain control signal (GC. On the other hand, when gain control transistor GC is at a high level during the readout, the conversion gain of floating diffusion layer FD decreases.
Low-sensitivity pixels K1 and K2 can make the signal charge amount, which is accumulable proximate to the surface of low-sensitivity pixel K by control voltage (PG, and the saturation level variable. As shown in image (b) of
Note that in
The sensitivity of low-sensitivity pixels K1 and K2 can be adjusted by laminating a ND filter, e.g. gray filter, to an (R, Gr, B, Gb) on-chip filter and adjusting a thickness and laminated region of this filter. An alternative method to the ND filter is performing a metal shielding and adjusting the area of an aperture area and sensitivity. The shutter and readout use a rolling motion.
In
As illustrated in
In other words, first pixel groups G1 and G2 each have a basic configuration of four high-sensitivity pixels and four low-sensitivity pixels.
The four high-sensitivity pixel PD1 or PD2 are indicated by (R, Gr, B, and Gb). Low-sensitivity pixels with a low saturation-sensitivity ratio are collectively called K1 and low-sensitivity pixels with a high saturation-sensitivity ratio are collectively called K2. First pixel groups G1 include four high-sensitivity pixels (R, Gr, B, and Gb) and four respectively corresponding low-sensitivity pixels K1. Second pixel groups G2 include four high-sensitivity pixels (R, Gr, B, and Gb) and four respectively corresponding low-sensitivity pixels K2. Four high-sensitivity pixels PD1 or PD2 in first pixel groups G1 or second pixel groups form a Bayer arrangement. Similarly, four low-sensitivity pixels K1 also form a Bayer arrangement. Similarly, four low-sensitivity pixels K2 also form a Bayer arrangement.
Low-sensitivity pixel K1 and low-sensitivity pixel K2 each have a different saturation-sensitivity ratio. For example, low-sensitivity pixel K1 has a lower saturation-sensitivity ratio than low-sensitivity pixel K2.
This makes it possible to increase the dynamic range of a light signal of low-sensitivity pixels K1 and K2, and, for example, to receive a signal transmitted from a signaler with a comparatively low illuminance to a headlight with a high illuminance in the same frame with respect to a light-emitting source repeatedly turned on and off (pulsed light-emitting source, LED light source). Note that the configuration example of pixel circuit 3 shown in
Exposure operations of high-sensitivity pixels PD1 and PD2 and low-sensitivity pixels K1 and K2 in solid-state imaging apparatus 1 will be described next.
High-sensitivity pixel PD1 has the same sensitivity and saturation level as high-sensitivity pixel PD2. Similar to high-sensitivity pixel PD shown in
Solid-state imaging apparatus 1, generates, within vertical scanning period 1V, low-sensitivity frame (K1) by performing an exposure operation that exposes low-sensitivity pixel K1 for exposure time (tK1) corresponding to one vertical scanning period. Similarly, solid-state imaging apparatus 1, generates, within vertical scanning period 1V, low-sensitivity frame (K2) by performing an exposure operation that exposes low-sensitivity pixel K2 for exposure time (tK2) corresponding to one vertical scanning period.
Note that in
Solid-state imaging apparatus 3 according to the present embodiment exposes the high-sensitivity pixels three times, exposure times tL, tM, and tS, which are split up with different durations, for increasing the dynamic range of the high-sensitivity pixels. Low-sensitivity pixels K1 and K2 are exposed for a longer exposure time in parallel with the high-sensitivity pixels. In other words, tK1>tL+tM+tS and tK2>tL+tM+tS.
The LED light in the drawing cannot be exposed with exposure operations (Long, Middle, and Short) of the high-sensitivity pixel (×mark), but can be exposed in low-sensitivity pixels K1 and K2 (O mark), and flicker reduction is possible upon correcting the signal of the high-sensitivity pixel with the signal of the low-sensitivity pixels.
Note that in
By making exposure times tK1 and tK2 of low-sensitivity pixels K1 and K2 roughly the same duration as one vertical scanning period, the lighting signal can reliably be captured with the low-sensitivity pixels when capturing a light-emitting source repeatedly and cyclically turned on and off (pulsed light-emitting source, LED light source).
For example, low-sensitivity pixel K1 has a lower saturation-sensitivity ratio than low-sensitivity pixel K2. With regard to the problem of the pixel saturation, external light with low illuminance (pulsed light, LED light, LD light) can be captured in low-sensitivity pixel K1 (saturation-sensitivity ratio: low). However, external light with high illuminance (pulsed light, LED light, LD light) can be captured in low-sensitivity pixel K2 (saturation-sensitivity ratio: high). In other words, with regard to the lighting signal problem, light can be continuously received along the time axis, and with regard to the dynamic range problem, low-sensitivity pixels K1 and K2 can receive light more broadly.
Configuration Example of Signal Processor 70
A configuration of signal processor 70 will be described next.
As illustrated in
Interpolation circuit 731 interpolates the pixel signal missing in low-sensitivity frame (K1) and generates interpolated low-sensitivity frame (K1). In other words, interpolation circuit 731 generates interpolated low-sensitivity frame (K1) including pixel signals (K1) obtained from low-sensitivity pixels K1, interpolated signals (P1) obtained through interpolation using pixel signals (K1). Interpolated signal (P1) uses at least four surrounding pixel signals (K1) and is generated by weighted interpolation depending on a center of gravity.
Interpolation circuit 732 interpolates the pixel signal missing in low-sensitivity frame (K2) and generates interpolated low-sensitivity frame (K2). In other words, interpolation circuit 732 generates interpolated low-sensitivity frame (K2) including pixel signals (K2) obtained from low-sensitivity pixels K2, interpolated signals (P2) obtained through interpolation using pixel signals (K2). Interpolated signal (P2) uses at least four surrounding pixel signals (K2) and is generated by weighted interpolation depending on a position of gravity.
Low-sensitivity signal generating circuit 741 generates first low-sensitivity frame (L) for correcting first high-sensitivity frame (L) by synthesizing the pixel signal (K1 or P1) of interpolated low-sensitivity frame (K1) and the pixel signal (P2 or K2) of interpolated low-sensitivity frame (K2). To be specific, low-sensitivity signal generating circuit 741 multiplies the pixel signal (K1 or P1) of interpolated low-sensitivity frame (K1) by a first type first coefficient and the pixel signal (P2 or K2) of interpolated low-sensitivity frame (K2) by a second type first coefficient.
First type first coefficient here depends on first exposure time tL and exposure time tK1 of low-sensitivity pixel K1, and is a coefficient for scale aligning the pixel signal (K1 or P1) of interpolated low-sensitivity frame (K1) to pixel signal (L) of the first high-sensitivity frame. For example, the first type first coefficient is determined depending on exposure time tK1, first exposure time tL, the gain and sensitivity of high-sensitivity pixel PD, and the gain and sensitivity of low-sensitivity pixel K1. Similarly, the second type first coefficient depends on first exposure time tL and exposure time tK2 of low-sensitivity pixel K2, and is a coefficient for scale aligning the pixel signal (K2 or P2) of interpolated low-sensitivity frame (K2) to pixel signal (L) of the first high-sensitivity frame. For example, the second type first coefficient is determined depending on exposure time tK2, first exposure time tL, the gain and sensitivity of high-sensitivity pixel PD, and the gain and sensitivity of low-sensitivity pixel K2.
Low-sensitivity signal generating circuit 741 generates first low-sensitivity frame (L) by synthesizing (i) interpolated low-sensitivity frame (K1) multiplied by the first type first coefficient and (ii) interpolated low-sensitivity frame (K2) multiplied by the second type first coefficient.
Low-sensitivity signal generating circuit 742 generates second low-sensitivity frame (M) for correcting second high-sensitivity frame (M) by synthesizing the pixel signal (K1 or P1) of interpolated low-sensitivity frame (K1) and the pixel signal (P2 or K2) of interpolated low-sensitivity frame (K2). To be specific, low-sensitivity signal generating circuit 742 multiplies the pixel signal (K1 or P1) of interpolated low-sensitivity frame (K1) by a first type second coefficient and the pixel signal (P2 or K2) of interpolated low-sensitivity frame (K2) by a second type second coefficient.
First type second coefficient here depends on second exposure time tM and exposure time tK1 of low-sensitivity pixel K1, and is a coefficient for scale aligning the pixel signal (K1 or P1) of interpolated low-sensitivity frame (K1) to pixel signal (M) of the second high-sensitivity frame. For example, the first type second coefficient is determined depending on exposure time tK1, second exposure time tM, the gain and sensitivity of high-sensitivity pixel PD, and the gain and sensitivity of low-sensitivity pixel K1. Similarly, the second type second coefficient depends on second exposure time tM and exposure time tK2 of low-sensitivity pixel K2, and is a coefficient for scale aligning the pixel signal (K2 or P2) of interpolated low-sensitivity frame (K2) to pixel signal (M) of the second high-sensitivity frame. For example, the second type second coefficient is determined depending on exposure time tK2, second exposure time tM, the gain and sensitivity of high-sensitivity pixel PD, and the gain and sensitivity of low-sensitivity pixel K2.
Low-sensitivity signal generating circuit 742 generates second low-sensitivity frame (M) by synthesizing (i) interpolated low-sensitivity frame (K1) multiplied by the first type second coefficient and (ii) interpolated low-sensitivity frame (K2) multiplied by the second type second coefficient.
Low-sensitivity signal generating circuit 743 generates third low-sensitivity frame (S) for correcting third high-sensitivity frame (S) by synthesizing the pixel signal (K1 or P1) of interpolated low-sensitivity frame (K1) and the pixel signal (P2 or K2) of interpolated low-sensitivity frame (K2). To be specific, low-sensitivity signal generating circuit 743 multiplies the pixel signal (K1 or P1) of interpolated low-sensitivity frame (K1) by a first type third coefficient and the pixel signal (P2 or K2) of interpolated low-sensitivity frame (K2) by a second type third coefficient.
First type third coefficient here depends on third exposure time tS and exposure time tK1 of low-sensitivity pixel K1, and is a coefficient for scale aligning the pixel signal (K1 or P1) of interpolated low-sensitivity frame (K1) to pixel signal (S) of the third high-sensitivity frame. For example, the first type third coefficient is determined depending on exposure time tK1, third exposure time tS, the gain and sensitivity of high-sensitivity pixel PD, and the gain and sensitivity of low-sensitivity pixel K1. Similarly, the second type third coefficient depends on third exposure time tS and exposure time tK2 of low-sensitivity pixel K2, and is a coefficient for scale aligning the pixel signal (K2 or P2) of interpolated low-sensitivity frame (K2) to pixel signal (S) of the third high-sensitivity frame. For example, the second type third coefficient is determined depending on exposure time tK2, third exposure time tS, the gain and sensitivity of high-sensitivity pixel PD, and the gain and sensitivity of low-sensitivity pixel K2.
Low-sensitivity signal generating circuit 743 generates third low-sensitivity frame (S) by synthesizing (i) interpolated low-sensitivity frame (K1) multiplied by the first type third coefficient and (ii) interpolated low-sensitivity frame (K2) multiplied by the second type third coefficient.
Corrector 75 and WDR synthesis circuit 76 are the same as in
Note that the exposure times of low-sensitivity pixels K1 and K2 may be tK1>tL+tM+tS and tK2>tL+tM+tS. tK1 and tK2 may be between approximately 16.6 ms (= 1/60 fps) of one vertical synchronization period and approximately 11 ms (= 1/90 Hz) corresponding to a minimum frequency of LEDs.
The saturation-sensitivity ratio of low-sensitivity pixel K1 to low-sensitivity pixel K2 may, for example, be 1:20, and low-sensitivity pixels K1 and K2 may be configured as to not saturate even when receiving an object's maximum illuminance (illuminance of sunlight) of 100,000 lux.
A simplified calculation of the dynamic range (DR) of the high-sensitivity pixels is as follows.
Note that with regard to the dynamic range of one frame, the dynamic range (=saturation/noise ratio) of the pixels are limited by a precision of A/D bit width, assuming that the dynamic range of the pixels exceeds the precision of A/D bit width.
A simplified calculation of the dynamic range of low-sensitivity pixels K1 and K2 is similarly as follows.
The dynamic range of low-sensitivity pixels K1 and K2 is configured to form a S/N curve from 100,000 lux.
The solid-state imaging apparatus according to the present embodiment adjusts control voltage ΦPG of control electrode PG so that the low-sensitivity pixels do not saturate and that the saturation level is not too low during the signal processing of the low-sensitivity pixels, and can also implement automatic exposure (AE) control and a white balance.
The solid-state imaging apparatus implements a gravity correction so that a gravity of each pixel does not shift during the interpolation by interpolation circuits 731 and 732.
This enables the low-sensitivity pixels to be in a correct exposure state.
Control electrode PG has a dark current problem due to the electric charge accumulating proximate to the surface, and there are, therefore, cases where image quality (S/N) is too low, especially for the image quality during low illuminance that emphasizes S/N.
In Embodiments 1 and 2, however, the S/N of the low-sensitivity pixels including control electrode PG does not deteriorate since the low-sensitivity pixels are used for the correction of the high-sensitivity pixel signal for the flicker reduction.
Note that an example has been described of low-sensitivity pixels K, K1, and K2 having control electrode PG with a photodiode structure as a means to adjust the saturation level, but as long as the saturation-sensitivity ratio can be adjusted at will by adjusting only the sensitivity, low-sensitivity pixels K, K1, and K2 may also not have a photodiode structure.
An example has been described of the high-sensitivity pixels generating three high-sensitivity frames (L, M, and S), but are not limited thereto.
An example has been described of the high-sensitivity pixels being disposed in a Bayer arrangement (R, Gr, B, and Gb), but are not limited thereto, and may also be disposed as (R, C, C, and B), (R, C, C, and C), or (R, G, B, and Ir).
In solid-state imaging apparatus 1 in the above-described Embodiment 2, the low-sensitivity pixels include first low-sensitivity pixels K1 and second low-sensitivity pixels K2. At least one of a signal saturation level and a sensitivity of first low-sensitivity pixels K1 is different from the at least one of a signal saturation level and a sensitivity of second low-sensitivity pixels K2.
Solid-state imaging apparatus 1 may includes first pixel groups G1 and second pixel groups G2. First pixel groups G1 may each include four high-sensitivity pixels disposed in a square region, and four first low-sensitivity pixels disposed in the square region. Second pixel groups G2 may each include four high-sensitivity pixels disposed in a square region, and four second low-sensitivity pixels disposed in the square region. First pixel groups G1 and second pixel groups G2 may be alternately disposed in rows and columns.
Signal processor 70 may generate (i) a first interpolated frame including first pixel signals obtained from the first low-sensitivity pixels, and first interpolated signals obtained through interpolation using the first pixel signals, (ii) a second interpolated frame including second pixel signals obtained from the second low-sensitivity pixels, and second interpolated signals obtained through interpolation using the second pixel signals, and (iii) the pixel signal by detecting a difference signal between the signal from the high-sensitivity pixels and a signal obtained by synthesizing the first interpolated frame and the second interpolated frame, and correcting the signal from the high-sensitivity pixels using the difference signal.
Solid-state imaging apparatus 1 may generate, within one vertical scanning period, a first high-sensitivity frame, a second high-sensitivity frame, and a third high-sensitivity frame by respectively performing a first exposure operation that exposes the high-sensitivity pixels for a first exposure time, a second exposure operation that exposes the high-sensitivity pixels for a second exposure time, and a third exposure operation that exposes the high-sensitivity pixels for a third exposure time. The first exposure time may be longer than the second exposure time and the second exposure time may be longer than the third exposure time. Signal processor 70 may generate (i) a second low-sensitivity frame and a third low-sensitivity frame by multiplying a frame including the signal from the low-sensitivity pixels by, respectively, a second coefficient corresponding to the second exposure time and a third coefficient corresponding to the third exposure time, (ii) a second correction signal by detecting a difference signal between a signal from the second high-sensitivity frame and a signal from the second low-sensitivity frame, and correcting the signal from the second high-sensitivity frame using the difference signal, (iii) a third correction signal by detecting a difference signal between a signal from the third high-sensitivity frame and a signal from the third low-sensitivity frame, and correcting the signal from the third high-sensitivity frame using the difference signal, and (iv) the pixel signal by synthesizing a signal from the first high-sensitivity frame, the second correction signal and the third correction signal.
Solid-state imaging apparatus 1 that can adjust the sensitivity of low-sensitivity pixel K by distributing the time during which the normal exposure and the intermediate exposure are repeated is described in Embodiment 3 as a variation of the solid-state imaging apparatus according to the above Embodiment 1. Note that hereinafter, differences with Embodiment 1 will mainly be described and overlapping descriptions are omitted.
As illustrated in
In other words, the sensitivity of low-sensitivity pixel K is made variable by (i) adjusting control voltage ΦPG of the above photodiode and a pulse duty of transfer voltage ΦTR is adjusted, and (ii) and causing the normal exposure and the intermediate exposure (electric charge emitting) to repeat.
As illustrated in Embodiment 2, for example, when having low-sensitivity pixels K1 and K2, a magnitude relationship thereof is sensitivity of K1>sensitivity of K2 when normal exposure time of K1>normal exposure time of K2, and saturation-sensitivity ratio of K1<saturation-sensitivity ratio of K2 can be set.
This makes it possible to adjust the saturation level with the voltage level of ΦPG, and to adjust the sensitivity level with a distribution of the normal exposure time and the intermediate exposure time. This makes it possible to freely determine the dynamic range by freely configuring the saturation/sensitivity ratio.
The sensitivity of low-sensitivity pixel K during this multiple exposure can be expressed with the following formula.
sensitivity of multiple exposure=(sensitivity 1×normal exposure time+sensitivity 2×intermediate exposure time)/1V
Sensitivity 1 is the sensitivity of the normal exposure per 1V. Sensitivity 2 is the sensitivity of the intermediate exposure per 1V.
The normal exposure and the intermediate exposure (electric charge emission) may be performed simultaneously for every row. This is because the layout of an address decoder of vertical scanning circuit 14 is more complicated when using the rolling shutter motion.
As illustrated in
Note that in
This makes it possible to broadly adjust the sensitivity of low-sensitivity pixel K through the multiple exposure thereof with regard to the problem of the dynamic range.
With regard to the multiple exposure, a ratio of the exposure time of photodiode K of the low-sensitivity pixel to the intermediate exposure time can be adjusted. To give an example, one H exposure time and three H intermediate exposure times, one H exposure time and seven H intermediate exposure times, one H exposure time and 15 H intermediate exposure times, or one H exposure time and 31 H intermediate exposure times can be set. When the signal readout motion and the intermediate exposure motion are simultaneous, the signal readout motion has priority.
Solid-state imaging apparatus 1 in the above-described Embodiment 3 includes pixel circuits 3 arranged in a matrix. Pixel circuits 3 each include at least one high-sensitivity pixel; floating diffusion layer FD that retains signal charge; a first transfer transistor that is disposed corresponding to the at least one high-sensitivity pixel, and transfers signal charge of the at least one high-sensitivity pixel to floating diffusion layer FD; at least one low-sensitivity pixel; a second transfer transistor that is disposed corresponding to the at least one low-sensitivity pixel, and transfers signal charge of the at least one low-sensitivity pixel to the floating diffusion layer. Solid-state imaging apparatus 1 repeatedly performs a normal exposure operation and an intermediate exposure operation on the low-sensitivity pixels within one vertical scanning period; during the normal exposure operation, a gate electrode of the second transfer transistor is at a low level; during the intermediate exposure operation, the gate electrode of the second transfer transistor is at a high level or a medium level; and simultaneously switches all rows of the low-sensitivity pixels from the normal exposure operation to the intermediate exposure operation, and simultaneously switches all rows of the low-sensitivity pixels from the intermediate exposure operation to the normal exposure operation.
This makes it possible to broadly adjust the sensitivity of low-sensitivity pixel K through the multiple exposure thereof.
Hereinafter, an imaging apparatus according to Embodiment 4 will be described with reference to the drawings. Note that the imaging apparatus in the present embodiment includes at least one solid-state imaging apparatus 1 according to the above Embodiments 1 to 3. This will be described in detail hereinafter.
CIS 91 is solid-state imaging apparatus 1 shown in each of the embodiments.
ISP 92 receives a pixel signal from CIS 91 and performs image processing, e.g., enlargement, shrinking, compressing coding and the like.
Monitor 93 serves as confirmation for the user during capturing.
Note that CIS 91 and ISP 92 may be a single system on chip (SoC) 90, and may also be separate chips. When CIS 91 and ISP 92 are separate chips, signal processor 70 may be included in CIS 91 or in ISP 92.
A portion of signal processor 70 may be realized using software and not a circuit.
CIS 91 and ISP 92 are the same as in
Sensor 94 is, for example, a radar sensor for ranging or a light detection and ranging (lidar) sensor for ranging.
Sensor ECU 95 receives a signal from ISP 92 and sensor 94 and controls alarm 96 and controller 97.
Alarm 96 is, for example, a variety of status display lights or warning lights inside an instrument panel of a vehicle.
Controller 97 controls, for example, an actuator and the like that drives steering, breaking, and the like of a vehicle.
Note that the imaging apparatus in
In
Capture ranges S1 to S9 shown in
As illustrated in
As described above, the imaging apparatus in Embodiment 4 includes the above solid-state imaging apparatus 1, and any one of a view system, an advanced driver-assistance sensing system, and an automated driving sensing system.
The imaging apparatus is installed on at least one of a front-view mirror, a left side mirror, a right side mirror, and a rear-view mirror of transportation machinery.
A solid-state imaging apparatus and an imaging apparatus using the solid-state imaging apparatus in the present disclosure has been described based on the above embodiments, but are not limited thereto. Forms realized by optionally combining components and functions in the embodiments, forms obtained by various combinations of the components in the different embodiments that can be conceived by a person skilled in the art which are within the scope of the essence of the present disclosure, and various devices built in the solid-state imaging apparatus and the imaging apparatus using the solid-state imaging apparatus in the present disclosure are also be included in the scope of the one or more aspects of the present disclosure.
Although only some exemplary embodiments of the present disclosure have been described in detail above, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiments without materially departing from the novel teachings and advantages of the present disclosure. Accordingly, all such modifications are intended to be included within the scope of the present disclosure.
The present disclosure is applicable to a solid-state imaging apparatus and an imaging apparatus.
This application is a U.S. continuation application of PCT International Patent Application Number PCT/JP2017/042988 filed on Nov. 30, 2017, claiming the benefit of priority of U.S. Provisional Patent Application No. 62/431,603 filed on Dec. 8, 2016, the entire contents of which are hereby incorporated by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 8823850 | Rysinski et al. | Sep 2014 | B2 |
| 20040223065 | Takayanagi | Nov 2004 | A1 |
| 20070080838 | Asayama et al. | Apr 2007 | A1 |
| 20100245647 | Honda et al. | Sep 2010 | A1 |
| 20160173796 | Takado | Jun 2016 | A1 |
| 20160255289 | Johnson et al. | Sep 2016 | A1 |
| 20170104942 | Hirota | Apr 2017 | A1 |
| 20170155858 | Mabuchi | Jun 2017 | A1 |
| 20190289238 | Ikuma et al. | Sep 2019 | A1 |
| Number | Date | Country |
|---|---|---|
| 0757476 | Feb 1997 | EP |
| 1 819 151 | Aug 2007 | EP |
| 2007-266556 | Oct 2007 | JP |
| Entry |
|---|
| Extended European Search Report dated Dec. 2, 2019 issued in corresponding European Patent Application No. 17878149.8. |
| Partial Supplementary European Search Report dated Sep. 19, 2019 issued in corresponding European Patent Application No. 17878149.8. |
| International Search Report and Written Opinion dated Feb. 20, 2018 in International Application No. PCT/JP2017/042988; with partial English translation. |
| International Search Report and Written Opinion dated Jan. 23, 2018 in International Application No. PCT/JP2017/042989; with partial English translation. |
| Notice of Allowance issued in U.S. Appl. No. 16/431,302, dated Mar. 20, 2020. |
| Number | Date | Country | |
|---|---|---|---|
| 20190288020 A1 | Sep 2019 | US |
| Number | Date | Country | |
|---|---|---|---|
| 62431603 | Dec 2016 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/JP2017/042988 | Nov 2017 | US |
| Child | 16431277 | US |
