The present disclosure relates to a photoelectric conversion apparatus, a photoelectric conversion system, and a mobile object.
U.S. Patent Application Publication No. 2020/0152807 describes a single photon avalanche photodiode (SPAD) having a protective film consisting of an oxide film, a nitride film, or a combination thereof on a silicon substrate surface.
According to an aspect of the present disclosure, a photoelectric conversion apparatus having an avalanche photodiode is provided. The photoelectric conversion apparatus includes a semiconductor layer having a first surface and a second surface, a first semiconductor region of a first conductivity type disposed at a first depth, a second semiconductor region of a second conductivity type disposed at a second depth that is greater than the first depth from the first surface, wherein the second semiconductor region forms the avalanche photodiode together with the first semiconductor region, a third semiconductor region of the first conductivity type provided in contact with an end portion of the first semiconductor region, an oxide film disposed on the first surface of the semiconductor layer, a member provided on an opposite side of the oxide film from the semiconductor layer in cross-sectional view, wiring, and a contact plug configured to connect the wiring to the first surface of the semiconductor layer, wherein the member is disposed to overlap at least a boundary between the first semiconductor region and the third semiconductor region in plan view, wherein the member is disposed between the wiring and the semiconductor layer, and wherein a work function of the member differs from a work function of each of the first semiconductor region and the third semiconductor region so that a potential gradient for signal charge is generated in a depth direction of the semiconductor layer at at least the boundary.
According to another aspect of the present disclosure, a photoelectric conversion apparatus having an avalanche photodiode is provided. The photoelectric conversion apparatus includes a semiconductor layer having a first surface and a second surface, a first semiconductor region of a first conductivity type disposed at a first depth, a second semiconductor region of a second conductivity type disposed at a second depth that is greater than the first depth from the first surface, wherein the second semiconductor region forms the avalanche photodiode together with the first semiconductor region, a third semiconductor region of the first conductivity type provided in contact with an end portion of the first semiconductor region, an oxide film disposed on the first surface of the semiconductor layer, a protective film disposed on an opposite side of the oxide film from the semiconductor layer, wherein the protective film is made of a material that differs from the material used for the oxide film, an interlayer film disposed on an opposite side of the protective film from the oxide film, and a member provided between the oxide film and the protective film in cross-sectional view, wherein the member is disposed to overlap at least a boundary between the first semiconductor region and the third semiconductor region in plan view.
Further features of the present disclosure will become apparent from the following description of exemplary embodiments with reference to the attached drawings.
In an SPAD, an avalanche multiplication is initiated by applying a strong electric field to a PN junction diode provided in a semiconductor layer to detect photons. However, when the electric field applied to the PN junction diode becomes intense, hot carriers accelerated by the electric field are generated. In the structure described in U.S. Patent Application Publication No. 2020/0152807, hot carriers are trapped near the cathode region, causing the potential to change and, thus, the breakdown voltage to change over time.
Accordingly, the present disclosure provides a structure of a photoelectric conversion apparatus to reduce a temporal change in the breakdown voltage due to an increase in hot carriers over time.
Embodiments described below are for embodying the technical concept of the present disclosure and are not intended to limit the present disclosure. The sizes and positional relationships of members illustrated in the drawings may be exaggerated for clarity of description. In the following description, the same configuration may be identified by the same reference numeral, and description may be omitted.
The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. In the following description, the terms indicating specific directions and positions (for example, “upper”, “lower”, “right”, “left”, and other terms including these terms) are used as necessary. These terms are used to facilitate understanding of the embodiments with reference to the drawings, and the technical scope of the present disclosure is not limited by the meanings of the terms.
As used herein, the term “plan view” is used to refer to a view in a direction perpendicular to the light incident surface of a semiconductor layer. The term “cross-sectional view” refers to a plane in a direction perpendicular to the light incident surface of the semiconductor layer. When the light incident surface of the semiconductor layer is microscopically rough, the plan view is defined on the basis of the light incident surface of the semiconductor layer when viewed macroscopically.
In the following description, the anode of an avalanche photodiode (APD) is set at a fixed electric potential, and a signal is taken from the cathode side. Therefore, a semiconductor region of a first conductivity type in which the charges of a polarity the same as that of a signal charge are majority carriers is an N-type semiconductor region, and a semiconductor region of a second conductivity type in which the charges of the polarity different from that of a signal charge are majority carriers is a P-type semiconductor region.
The present disclosure can also be applied when the cathode of the APD is set at a fixed electric potential and the signal is taken from the anode side. In this case, the semiconductor region of the first conductivity type in which charges of a polarity the same as that of the signal charge are majority carriers is a P-type semiconductor region, and the semiconductor region of the second conductivity type in which charges of a polarity different from that of the signal charge are majority carriers is an N-type semiconductor region. Hereinafter, the case where one of the nodes of the APD is set to a fixed electric potential is described below. However, the potentials of both nodes may vary.
The term “impurity concentration” as simply used herein refers to the net impurity concentration obtained after subtracting the amount compensated by the impurity of an opposite conductivity type. That is, the term “impurity concentration” refers to the NET doping concentration. A region in which the P-type additive impurity concentration is higher than the N-type additive impurity concentration is a P-type semiconductor region. In contrast, a region in which the N-type additive impurity concentration is higher than the P-type additive impurity concentration is an N-type semiconductor region.
A configuration common to all embodiments of a photoelectric conversion apparatus and a method for driving the photoelectric conversion apparatus according to the present disclosure are described below with reference to
The photoelectric conversion apparatus 100 is configured by stacking and electrically connecting a sensor substrate 11 with a circuit substrate 21. The sensor substrate 11 has a first semiconductor layer including photoelectric conversion elements 102 (described below) and a first wiring structure. The circuit substrate 21 has a second semiconductor layer including circuits, such as a signal processing unit 103 (described below), and a second wiring structure. The photoelectric conversion apparatus 100 is configured by stacking the second semiconductor layer, the second wiring structure, the first wiring structure, and the first semiconductor layer in this order. The photoelectric conversion apparatus described in each of the embodiments is a back-illuminated photoelectric conversion apparatus in which light is incident on a first surface and a circuit substrate is disposed on a second surface.
Although the sensor substrate 11 and the circuit substrate 21 in the form of diced chips are described below, the forms are not limited to chips. For example, the substrates may be wafers. In addition, the substrates in the form of wafers may be stacked and then diced or may be made into chips and then stacked and bonded.
The sensor substrate 11 has a pixel region 12 disposed therein, and the circuit substrate 21 has, disposed therein, a circuit region 22 for processing signals detected by the pixel region 12.
The pixels 101 are typically pixels for forming an image. However, when used for TOF (Time of Flight), the pixels 101 do not necessarily form an image. That is, the pixels 101 may be pixels for measuring the time and the amount of light when the light reaches the pixels 101.
The photoelectric conversion element 102 illustrated in
The vertical scanning circuit unit 110 receives a control pulse supplied from the control pulse generation unit 115 and supplies the control pulse to each of the pixels. Logic circuits, such as a shift register and an address decoder, are used in the vertical scanning circuit unit 110.
A signal output from the photoelectric conversion element 102 of the pixel is processed by the signal processing unit 103. The signal processing unit 103 includes a counter, a memory, and the like, and a digital value is held in the memory.
The horizontal scanning circuit unit 111 inputs a control pulse for sequentially selecting each of columns to the signal processing unit 103 to read a signal from the memory of each of the pixels that holds the digital signal.
The signal is output to the signal line 113 from the signal processing unit 103 of the pixel selected by the vertical scanning circuit unit 110 for the selected column.
The signal output to the signal line 113 is output to an external recording unit or the signal processing unit of the photoelectric conversion apparatus 100 via an output circuit 114.
In
As illustrated in
In
The APD 201 generates charge pairs in accordance with incident light by photoelectric conversion. A voltage VL (a first voltage) is supplied to the anode of the APD 201. In addition, a voltage VH (a second voltage) that is higher than the voltage VL supplied to the anode is supplied to the cathode of the APD 201. A reverse bias voltage is supplied to the anode and cathode so that the APD 201 performs an avalanche multiplication operation. By supplying such voltages, charges generated by the incident light undergo avalanche multiplication, which generates an avalanche current.
When a reverse bias voltage is supplied, an APD has two modes of operation: the Geiger mode in which the potential difference between the anode and cathode is greater than the breakdown voltage and the linear mode in which the potential difference between the anode and cathode is less than or equal to the breakdown voltage.
An APD operated in the Geiger mode is referred to as an SPAD. For example, the voltage VL (the first voltage) is −30 V, and the voltage VH (the second voltage) is 1 V. The APD 201 may be operated in either the linear mode or the Geiger mode.
A quenching element 202 is connected to a power source that supplies the voltage VH and the APD 201. The quenching element 202 functions as a load circuit (a quenching circuit) during signal multiplication by avalanche multiplication, reduces the voltage supplied to the APD 201, and has a function of reducing avalanche multiplication (a quenching operation). In addition, the quenching element 202 has a function of causing a current corresponding to the voltage drop due to the quenching operation to flow and returning the voltage supplied to the APD 201 to the voltage VH (a recharge operation).
The signal processing unit 103 includes a waveform shaping unit 210, a counter circuit 211, and a selection circuit 212. Herein, the signal processing unit 103 can include any one of the waveform shaping unit 210, the counter circuit 211, and the selection circuit 212.
The waveform shaping unit 210 shapes a change in the potential of the cathode of the APD 201 obtained during photon detection and outputs a pulse signal. For example, an inverter circuit is used as the waveform shaping unit 210. Although
The counter circuit 211 counts the pulse signals output from the waveform shaping unit 210 and holds a count value. Furthermore, when a control pulse pRES is supplied via a drive line 213, a count value held in the counter circuit 211 is reset.
The selection circuit 212 is supplied with a control pulse pSEL from the vertical scanning circuit unit 110 illustrated in
A switch, such as a transistor, may be provided between the quenching element 202 and the APD 201 or between the photoelectric conversion element 102 and the signal processing unit 103 to switch the electrical connection. Similarly, the voltage VH and the voltage VL supplied to the photoelectric conversion element 102 may be electrically switched using a switch, such as a transistor.
According to the present embodiment, the configuration using the counter circuit 211 is described. However, the photoelectric conversion apparatus 100 that acquires the pulse detection timing may be achieved by using a time-to-digital converter (hereinafter referred to as a TDC) and a memory instead of the counter circuit 211. At this time, the generation timing of the pulse signal output from the waveform shaping unit 210 is converted into a digital signal by the TDC. To measure the timing of the pulse signal, the TDC receives a control pulse pREF (a reference signal) supplied thereto from the vertical scanning circuit unit 110 illustrated in
Between time t0 and time t1, a potential difference of (VH−VL) is applied to the APD 201 illustrated in
It should be noted that the arrangement of the signal lines 113 and the arrangement of the column circuit 112 and the output circuit 114 are not limited to those illustrated in
A photoelectric conversion apparatus according to each of the embodiments is described below.
A photoelectric conversion apparatus according to the first embodiment is described below with reference to
The structure and function of the photoelectric conversion element 102 are described below. The photoelectric conversion element 102 has an N-type (first conductivity type) first semiconductor region 311, a third semiconductor region 313, a fifth semiconductor region 315, and a sixth semiconductor region 316. Furthermore, the photoelectric conversion element 102 has a P-type (second conductivity type) second semiconductor region 312, a fourth semiconductor region 314, a seventh semiconductor region 317, and a ninth semiconductor region 319.
According to the present embodiment, in the cross-sectional view illustrated in
The impurity concentration of the first semiconductor region 311 is higher than that of each of the third semiconductor region 313 and the fifth semiconductor region 315. A PN junction is formed between the P-type second semiconductor region 312 and the N-type first semiconductor region 311, and an APD is formed. By making the impurity concentration of the second semiconductor region 312 lower than that of the first semiconductor region 311, the entire portion of the second semiconductor region 312 that overlaps the center of the first semiconductor region in plan view is turned into a depletion layer region. At this time, the potential difference between the first semiconductor region 311 and the second semiconductor region 312 is greater than the potential difference between the second semiconductor region 312 and the fifth semiconductor region 315. The depletion layer region extends to a part of the first semiconductor region 311, and a strong electric field is induced in the extended depletion layer region. The strong electric field causes avalanche multiplication in the depletion layer region extending to the part of the first semiconductor region 311, and an electric current based on the amplified charge is output as signal charge. When light incident on the photoelectric conversion element 102 is photoelectrically converted and avalanche multiplication occurs in the depletion layer region (an avalanche multiplication region), the generated charge of the first conductivity type is collected in the first semiconductor region 311.
Although in
The pixels are separated by a pixel separation portion 324 having a trench structure. In addition, the P-type seventh semiconductor region 317 is provided around the pixel separation portion 324, and adjacent photoelectric conversion elements are separated from each other by a potential gradient. Since the photoelectric conversion elements are also separated by the potential of the seventh semiconductor region 317, a trench structure such as the pixel separation portion 324 is not essential as a pixel separation portion. Even when the pixel separation portion 324 having the trench structure is provided, the depth and position of the trench structure are not limited to those in the configuration illustrated in
A pinning film 321, a planarization film 322, and a microlens 323 are formed adjacent to the light incident surface of the semiconductor layer 302.
A concavo-convex structure (not illustrated) by a trench may be provided adjacent to the light incident surface. The concavo-convex structure is surrounded by a P-type fourth semiconductor region 314 and scatters light incident on the photoelectric conversion element 102. Since the incident light travels diagonally inside the photoelectric conversion element, an optical path length that is greater than or equal to the thickness of the semiconductor layer 302 can be ensured, enabling photoelectric conversion of light with a longer wavelength than in the case without the concavo-convex structure. In addition, the concavo-convex structure prevents the reflection of incident light within the substrate, which has the effect of improving the photoelectric conversion efficiency for the incident light.
The fifth semiconductor region 315 and the concavo-convex structure are formed so as to overlap in plan view. The area of a portion of the fifth semiconductor region 315 that overlaps the concavo-convex structure in plan view is greater than the area of a portion of the fifth semiconductor region 315 that does not overlap the concavo-convex structure. Charges generated at a position away from the avalanche multiplication region formed between the first semiconductor region 311 and the fifth semiconductor region 315 have a longer travel time to reach the avalanche multiplication region than charges generated at a position closer to the avalanche multiplication region. Therefore, timing jitter may increase. By placing the fifth semiconductor region 315 and the concavo-convex structure so as to overlap in plan view, the collection time for charges generated at positions away from the avalanche multiplication region can be reduced, which reduces the timing jitter. The fourth semiconductor region 314 three-dimensionally covers the concavo-convex structure, which reduces the generation of thermally excited charge at the interface portion of the concavo-convex structure. As a result, the DCR of the photoelectric conversion element is reduced.
Furthermore, a filter layer (not illustrated) or the like may be disposed adjacent to the light incident surface. As the filter layer, an optical filter, such as a color filter, an infrared cut filter, or a monochrome filter, can be used. As the color filter, an RGB color filter, an RGBW color filter, or the like can be used.
The wiring structure including a conductor and an insulating film is provided on the surface opposite the light incident surface of the semiconductor layer. An interlayer film (an interlayer insulating film) 343 is provided between the wiring provided in the wiring layer and the semiconductor layer. Between the interlayer film 343 and the semiconductor layer, an oxide film 341 and a protective film 342 are provided from the side adjacent to the semiconductor layer. The oxide film 341 and the protective film 342 are films provided over the plurality of pixels. The oxide film 341 is made of, for example, silicon oxide (SiO). However, SiON and the like may be used. The protective film 342 is a dielectric film and is made of a different material than the oxide film 341. The protective film 342 functions as a film to protect the avalanche photodiode from plasma damage and metal contamination during etching. A silicon nitride (SiN) film, which is a nitride film, is widely used as the protective film 342. However, a silicon oxynitride (SiON) film, a silicon carbide (SiC) film, a silicon carbonitride (SiCN) film, or another film may be used. If nitrogen is contained in both the oxide film 341 and the protective film 342, the film having higher nitrogen content is used as the protective film.
As used herein, the term “silicon nitride” refers to a compound of nitrogen (N) and silicon (Si) in which nitrogen (N) and silicon (Si) are elements other than light elements that make up the top two composition ratios of the constituent elements of the compound. Silicon nitride can contain light elements, such as hydrogen (H) and helium (He), the amount (atomic percent) of which may be more or less than each of nitrogen (N) and silicon (Si). Silicon nitride can contain elements other than nitrogen (N), silicon (Si), and light elements in lower concentrations than each of nitrogen (N) and silicon (Si). Typical elements that can be contained in silicon nitride are boron (B), carbon (C), oxygen (O), fluorine (F), phosphorus (P), chlorine (Cl), and argon (Ar). If, among the constituent elements of silicon nitride, the third most abundant element other than a light element is oxygen, the silicon nitride can be referred to as “silicon oxide nitride” or “oxygen-containing silicon nitride”.
Similarly, the term “silicon oxide” refers to a compound of oxygen (O) and silicon (Si) in which oxygen (O) and silicon (Si) are elements other than light elements that make up the top two composition ratios of the constituent elements of the compound. Typical elements that can be contained in silicon oxide are hydrogen (H), helium (He), boron (B), carbon (C), nitrogen (N), fluorine (F), phosphorus (P), chlorine (Cl), and argon (Ar). If, among the constituent elements of silicon oxide, the third most abundant element other than a light element is nitrogen, the silicon oxide can be referred to as “silicon nitride oxide” or “nitrogen-containing silicon oxide”. The elements contained in each of the constituent members of the photoelectric conversion apparatus can be analyzed by, for example, the energy dispersive X-ray spectrometry (EDX). The hydrogen content can be analyzed by, for example, the elastic recoil detection analysis (ERDA).
Cathode wiring 331A is electrically connected to the first semiconductor region 311 and supplies potential to the first semiconductor region 311. Anode wiring 331B supplies potential to the seventh semiconductor region 317 via the ninth semiconductor region 319. The wiring portions that constitute the cathode wiring 331A and the anode wiring 331B are composed of conductors including metals such as Cu and Al, for example. The anode wiring 331B is referred to as a “first wiring portion”, and the cathode wiring 331A is referred to as a “second wiring portion”. Depending on context, the wiring located in the first layer of the wiring layer is also referred to as “cathode wiring 331A” or “anode wiring 331B”. In this case, the contact portion that is in direct contact with the semiconductor layer, the cathode wiring 331A, and anode wiring 331B may be distinguished from one another, and description may be made. In another case, the entire wiring portion that supplies a voltage to the semiconductor region is also referred to as “cathode wiring 331A and anode wiring 331B”. In this case, the contact portion is included in the cathode wiring 331A and anode wiring 331B.
A member 332 is electrically connected to the cathode wiring 331A and is supplied with the same potential as the cathode wiring 331A. The member 332 is located between the cathode wiring 331A provided in a first wiring layer (M1) and the semiconductor layer 302 in cross-sectional view. In
The member 332 is, for example, a member having a work function greater than the work function of each of the first semiconductor region 311 and the third semiconductor region 313 of the first conductivity type when the first conductivity type is N-type. For example, the member 332 is P-type polysilicon.
When the first conductivity type is P-type, the work function of the member 332 is required to be less than that of each of the first semiconductor region 311 of the first conductivity type and the third semiconductor region 313 of the first conductivity type. In this case, a potential gradient is formed in the depth direction for holes, which are the signal charges. For example, N-type polysilicon can be used as the member 332.
That is, the work function of each of the first and third semiconductor regions 311 and 313 differs from the work function of the member 332, so that a potential gradient for the signal charge is formed in the depth direction of the semiconductor layer at least at the boundary.
To avoid hot carrier trapping, the anode wiring can be extended so as to be disposed in a region where the anode wiring overlaps the first semiconductor region 311 and the third semiconductor region 313 in plan view. However, if the distance between the anode wiring and cathode wiring is too short, breakdown voltage concerns may arise. According to the configuration of the present embodiment, since the member 332 is disposed, the distance between the cathode wiring 331A and the anode wiring 331B can be sufficiently maintained to reduce hot carrier trapping, and the breakdown voltage concerns can also be reduced.
While the above description has been made with reference to an example of electrical connection of the member 332 with the cathode wiring 331A, the above-described potential gradient is generated without electrical connection of the member 332 with the cathode wiring 331 under some work function conditions of the member 332 and semiconductor region. Accordingly, a configuration in which a predetermined voltage is not applied to the member 332 may be employed.
While the above description has been made with reference to an example in which the member 332 is provided between the protective film 342 and the interlayer film 343, the member 332 may be provided between the oxide film 341 and the protective film 342. In this case, the contact that electrically connects the cathode wiring 331A to the member 332 penetrates the protective film 342. Since as described above, a predetermined voltage need not be provided to the member 332, the contact wiring is not absolutely necessary.
A photoelectric conversion apparatus according to the second embodiment is described below with reference to
According to the present embodiment, the member 332 is formed between the oxide film 341 and the protective film 342 in cross-sectional view. In addition, in cross-sectional view, the protective film 342 is formed between two members 332. Furthermore, in plan view, the member 332 covers the boundary portion between the first semiconductor region 311 and the third semiconductor region 313. Unlike the first embodiment, the member 332 is isolated from the cathode wiring 331A and is not supplied with an electric potential.
The member 332 is made of a material that can make the interface state density between the member 332 and the oxide film 341 less than the interface state density between the oxide film 341 and the protective film 342. The interface state density is synonymous with the interface defect density. The interface state density (the interface defect density) is the density of defects caused by the incommensurability that occurs at the hetero interface of the members.
Hot carriers are trapped in the interface between the oxide film 341 and the protective film 342. This affects the potential distribution in the avalanche region in the semiconductor layer 302, and the breakdown voltage may change. Then, the hot carrier trapping can be reduced by providing the member 332 that satisfies the condition that the interface state density between the oxide film 341 and the member 332 is less than the interface state density between the oxide film 341 and the protective film 342. By forming the member 332 between the oxide film 341 and the protective film 342, the temporal change in breakdown voltage can be reduced.
In addition, like the first embodiment, a sufficient distance between the cathode wiring 331A and the anode wiring 331B can be maintained, thus reducing the breakdown voltage concerns.
According to the present embodiment, as illustrated in
A photoelectric conversion apparatus according to the present embodiment is described below with reference to
In the configuration illustrated in
However, according to the configuration illustrated in
The contact plug 331C may be made of the same material as the contact plug that electrically connects the cathode wiring 331A to the member 332. Alternatively, the contact plug 331C may be made of a different material than the member 332.
However, like the configuration illustrated in
According to the configuration of the present embodiment, the pixel size can be reduced as compared with the configuration of the first embodiment. In addition, like the configuration of the first embodiment, a temporal change in breakdown voltage can be reduced. Furthermore, breakdown voltage concerns can be reduced.
A photoelectric conversion system according to the present embodiment is described below with reference to
The photoelectric conversion apparatus described in each of the first to third embodiments can be applied to a variety of photoelectric conversion systems. Examples of an applicable photoelectric conversion system include a digital still camera, a digital camcorder, a surveillance camera, a copying machine, a facsimile, a mobile phone, on-vehicle camera, and an observation satellite. In addition, a camera module including an optical system, such as a lens, and an image pickup apparatus is included in the photoelectric conversion systems.
The image pickup apparatus 1004 is the photoelectric conversion apparatus according to any one of the above-described embodiments. The image pickup apparatus 1004 converts an optical image formed by the lens 1002 into an electrical signal. The photoelectric conversion system further includes a signal processing unit 1007 that serves as an image generation unit that generates an image by processing an output signal output from the image pickup apparatus 1004. The signal processing unit 1007 performs various corrections and compressions as necessary and outputs image data. The signal processing unit 1007 may be formed in a semiconductor substrate having the image pickup apparatus 1004 provided therein or may be formed in a semiconductor substrate other than the semiconductor substrate having the image pickup apparatus 1004 therein.
The photoelectric conversion system further includes a memory unit 1010 for temporarily storing image data and an external interface unit (external I/F unit) 1013 for communicating with an external computer or the like. Still furthermore, the photoelectric conversion system includes a recording medium 1012, such as a semiconductor memory, for recording and reading image data therein and therefrom, and a recording medium control interface unit (recording medium control I/F unit) 1011 for recording or reading data in and from the recording medium 1012.
The recording medium 1012 may be built in the photoelectric conversion system or may be removable. Furthermore, the photoelectric conversion system includes an overall control/calculation unit 1009 that performs control of various calculations and overall control of the digital still camera and a timing generation unit 1008 that outputs various timing signals to the image pickup apparatus 1004 and the signal processing unit 1007.
The timing signal and the like may be input from the outside, and the photoelectric conversion system can include at least the image pickup apparatus 1004 and the signal processing unit 1007 that processes the output signal output from the image pickup apparatus 1004.
The image pickup apparatus 1004 outputs an image pickup signal to the signal processing unit 1007. The signal processing unit 1007 performs predetermined signal processing on the image pickup signal output from the image pickup apparatus 1004 and outputs image data. The signal processing unit 1007 generates an image using the image pickup signal. As described above, according to the present embodiment, a photoelectric conversion system that employs the photoelectric conversion apparatus (the image pickup apparatus) of any one of the above-described embodiments can be achieved.
A photoelectric conversion system and a mobile object according to the present embodiment are described below with reference to
The photoelectric conversion system 1300 includes an image processing unit 1312 that performs image processing on a plurality of image data acquired by the image pickup apparatus 1310. Furthermore, the photoelectric conversion system 1300 includes a parallax acquisition unit 1314 that calculates a parallax (the phase difference of a parallax image) from the plurality of image data acquired by the photoelectric conversion system 1300. Still furthermore, the photoelectric conversion system 1300 further includes a distance acquisition unit 1316 that calculates the distance to the physical object on the basis of the calculated parallax and a collision determination unit 1318 that determines the collision probability on the basis of the calculated distance.
The parallax acquisition unit 1314 and the distance acquisition unit 1316 are examples of distance information acquisition units for acquiring information regarding the distance to a physical object. That is, the distance information is information related to a parallax, a defocus amount, the distance to a physical object, and the like. The collision determination unit 1318 may use any one of these pieces of distance information to determine the collision probability. The distance information acquisition unit may be implemented by dedicatedly designed hardware or may be implemented by a software module. Alternatively, the distance information acquisition unit may be implemented by a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or combinations thereof. Still alternatively, the distance acquisition unit may use a ToF range sensor, which is described in the following embodiment.
The photoelectric conversion system 1300 is connected to a vehicle information acquisition apparatus 1320. Thus, the photoelectric conversion system 1300 can acquire vehicle information, such as a vehicle speed, a yaw rate, and a steering angle. In addition, the photoelectric conversion system 1300 is connected to a control ECU 1330 which is a control unit that outputs a control signal for generating a braking force to the vehicle on the basis of the determination result of the collision determination unit 1318. Furthermore, the photoelectric conversion system 1300 is connected to an alarm device 1340 that emits an alarm to a driver on the basis of the determination result of the collision determination unit 1318. For example, if the collision determination unit 1318 determines that the collision probability is high, the control ECU 1330 performs vehicle control to avoid collisions or reduce damage by braking, releasing the accelerator pedal, or reducing the engine output. The alarm device 1340 emits an alarm to a user by, for example, sounding the alarm, displaying alarm information on a screen of a car navigation system, or vibrating a seat belt or steering wheel.
According to the present embodiment, the photoelectric conversion system 1300 captures the image of the surroundings of the vehicle, for example, the front view or rear view of the vehicle.
While an example of performing control so as not to collide with another vehicle has been described, the configuration can also be applied to control of self-driving vehicles to follow another vehicle or control of self-driving vehicles to keep the lane. Furthermore, the photoelectric conversion system can be applied not only to a vehicle, such as an automobile, but also to a mobile object (a moving apparatus), such as a boat, an aircraft, or an industrial robot. Still furthermore, the photoelectric conversion system can be applied not only to a mobile object but also to equipment that uses object recognition over a wide area, such as an intelligent transportation system (ITS).
A photoelectric conversion system according to the present embodiment is described with reference to
As illustrated in
The optical system 407 includes one or more lenses. The optical system 407 guides image light (incident light) from the object to the photoelectric conversion apparatus 408 and forms an image on the light receiving surface (a sensor unit) of the photoelectric conversion apparatus 408.
As the photoelectric conversion apparatus 408, the photoelectric conversion apparatus of any one of the embodiments described above is applied, and a distance signal indicating the distance obtained from the received light signal output from the photoelectric conversion apparatus 408 is supplied to the image processing circuit 404.
The image processing circuit 404 performs image processing to construct a range image based on the distance signal supplied from the photoelectric conversion apparatus 408. The range image (image data) obtained through the image processing is supplied to the monitor 405 and is displayed. In addition, the range image is supplied to the memory 406 and is stored (recorded).
In the range image sensor 401 configured in this way, by applying the above-described photoelectric conversion apparatus, it is possible to obtain, for example, a more accurate range image in accordance with improvement of the characteristics of the pixels. The photoelectric conversion system may also be a system that outputs the results of calculations using distance information, but not the image information used in the calculations. The photoelectric conversion system may set a predetermined exposure time from the light emission time and acquire distance information using light reception information during the predetermined exposure time. In this case, the distance information can be acquired without using the time period information from light emission to light reception.
A photoelectric conversion system according to the present embodiment is described below with reference to
The endoscope 1100 is composed of a lens barrel 1101, a predetermined length of the front end of which is to be inserted into the body cavity of the patient 1132, and a camera head 1102, which is connected to the base end of the lens barrel 1101. In the example of
An opening having an objective lens fitted thereinto is provided at the front end of the lens barrel 1101. A light source device 1203 is connected to the endoscope 1100, and light generated by the light source device 1203 is guided to the front end of the lens barrel 1101 by a light guide extending inside the lens barrel 1101. The light is emitted to an observation object in the body cavity of the patient 1132 through the objective lens. The endoscope 1100 may be a straight-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.
An optical system and a photoelectric conversion apparatus are provided inside the camera head 1102, and the reflected light (observation light) from the observation object is focused on the photoelectric conversion apparatus by the optical system. The photoelectric conversion apparatus photoelectrically converts the observation light to generate an electrical signal corresponding to the observation light. That is, an image signal corresponding to the observation image is generated. As the photoelectric conversion apparatus, the photoelectric conversion apparatus described in any one of the above embodiments can be used. The image signal is transmitted to a camera control unit (CCU) 1135 in the form of RAW data.
The CCU 1135 includes a central processing unit (CPU), a graphics processing unit (GPU), and the like. The CCU 1135 comprehensively controls the operations performed by the endoscope 1100 and a display device 1136. Furthermore, the CCU 1135 receives an image signal from the camera head 1102 and performs various image processing, such as development processing (demosaicing), for displaying an image based on the image signal.
Under the control of the CCU 1135, the display device 1136 displays an image based on the image signal subjected to image processing performed by the CCU 1135.
The light source device 1203 includes a light source, such as a light emitting diode (LED), and supplies the endoscope 1100 with irradiation light for capturing the image of a surgical site or the like.
An input device 1137 is an input interface to the endoscopic surgery system 1150. A user can input a variety of information and instructions to the endoscopic surgery system 1150 via the input device 1137.
A treatment tool control device 1138 controls driving of an energy treatment tool 1112 for tissue cauterization, incision, blood vessel sealing, or the like.
The light source device 1203 that supplies irradiation light to the endoscope 1100 when the image of a surgical site is captured can include, for example, a white light source, such as an LED, a laser light source, or combinations thereof. When the white light source is configured by a combination of R, G, and B laser light sources, the output intensity and output timing of each of the colors (each of the wavelengths) can be controlled with high accuracy. Thus, white balance of a captured image can be adjusted in the light source device 1203. In this case, the observation target is irradiated with laser light from each of the R, G, and B laser light sources in a time-division manner, and driving of an image pickup element of the camera head 1102 is controlled in synchronization with the irradiation timing. In this way, an image corresponding to each of the RGB colors can be captured in a time-division manner. According to the technique, a color image can be obtained without providing a color filter on the image pickup element.
In addition, the driving of the light source device 1203 may be controlled such that the intensity of the output light is changed at predetermined time intervals. A high dynamic range without so-called crushed shadows and blown out highlights can be generated by controlling the driving of the image pickup elements of the camera head 1102 in synchronization with the timing of the change in the intensity of the light, acquiring images in a time-division manner, and combining the images.
In addition, the light source device 1203 may be configured so as to be able to supply light in a predetermined wavelength band corresponding to special light observation. In special light observation, the wavelength dependency of light absorption by a body tissue is used, for example. More specifically, a high contrast image of a predetermined tissue, such as a blood vessel on the surface of the mucous membrane, is captured by irradiating the tissue with light in a narrower band than the irradiation light used during normal observation (that is, white light).
Alternatively, in special light observation, fluorescence observation may be performed in which an image is captured using fluorescence generated by irradiation with excitation light. In fluorescence observation, a body tissue is irradiated with excitation light, and fluorescence from the body tissue can be observed. Alternatively, a reagent, such as indocyanine green (ICG), is locally injected into the body tissue, and the body tissue is irradiated with excitation light corresponding to the fluorescence wavelength of the reagent. Thus, a fluorescent image can be obtained. The light source device 1203 can be configured so as to supply narrowband light and/or excitation light corresponding to the special light observation.
A photoelectric conversion system according to the present embodiment is described below with reference to
The glasses 1600 further include a control device 1603. The control device 1603 functions as a power source that supplies electric power to the photoelectric conversion apparatus 1602 and the display device. In addition, the control device 1603 controls the operations performed by the photoelectric conversion apparatus 1602 and the display device. The lens 1601 has an optical system formed therein to focus light onto the photoelectric conversion apparatus 1602.
The user's line of sight to the displayed image is detected from the captured infrared light images of the eyeball. Any known technique can be applied to line-of-sight detection using captured images of eyeballs. As an example, an eye gaze detection technique based on a Purkinje image obtained using reflection of irradiation light on the cornea.
More specifically, line-of-sight detection processing is performed on the basis of the pupillary-corneal reflection technique. The user's line of sight is detected by calculating a line of sight vector representing the orientation (the rotational angle) of the eyeball on the basis of the pupil image and the Purkinje image included in the captured eyeball image by using the pupillary-corneal reflection technique.
The display device according to the present embodiment may include a photoelectric conversion apparatus including a light receiving element and may control the display image of the display device on the basis of the user's line-of-sight information obtained from the photoelectric conversion apparatus.
More specifically, the display device determines a first field of view region that the user gazes at and a second field of view region other than the first field of view region on the basis of the line-of-sight information. The first field of view region and the second field of view region may be determined by a control unit of the display device. Alternatively, a first field of view region and a second field of view region determined by an external control device may be received. In the display area of the display device, the display resolution of the first field of view region may be controlled to be higher than the display resolution of the second field of view region. That is, the resolution of the second field of view region may be set to lower than that of the first field of view region.
Furthermore, the display area may have a first display area and a second display area different from the first display area, and a higher priority one of the first display area and the second display area may be determined on the basis of the line of sight information. The first field of view region and the second field of view region may be determined by the control unit of the display device or an external control device. Alternatively, a first field of view region and a second field of view region determined by an external control device may be received. The resolution of a high priority area may be set higher than the resolution of an area other than the high priority area. That is, the resolution of a relatively low priority area may be decreased.
Artificial intelligence (AI) may be used to determine the first field of view region and a high priority area. AI model may be a model configured to estimate the angle of the line of sight and the distance to an object in the line of sight from the eyeball image by using, as training data, eyeball images and the directions in which the eyeballs in the images are actually looking. The AI program may be stored in the display device, the photoelectric conversion apparatus, or an external device. When stored in the external device, the AI program is transmitted to the display device via communication.
In the case of display control based on visual recognition detection, the display control can be applied to smart glasses that further include a photoelectric conversion apparatus that captures the image of the outside. The smart glasses can display captured external information in real time.
The present disclosure is not limited to the above embodiments, and various modifications can be made. For example, an example in which part of the configuration of any one of the embodiments is added to another embodiment and an example in which part of the configuration of any one of the embodiments is replaced by part of another embodiment are also included in embodiments of the present disclosure.
In addition, the photoelectric conversion systems according to the fourth embodiment and the fifth embodiment are examples of photoelectric conversion systems to which the photoelectric conversion apparatus can be applied, and a photoelectric conversion system to which the photoelectric conversion apparatus according to the present disclosure can be applied is not limited to the configurations illustrated in
It should be noted that the above-described embodiments merely illustrate specific examples for carrying out the present disclosure, and the technical scope of the present disclosure should not be construed to be limited by the embodiments. That is, the present disclosure can be carried out in various forms without departing from its technical concept or main features.
The present disclosure can provide a photoelectric conversion apparatus that can reduce a temporal change in breakdown voltage due to an increase in hot carriers over time.
While the present disclosure has been described with reference to exemplary embodiments, it is to be understood that the present disclosure is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Application No. 2022-190921 filed Nov. 30, 2022, which is hereby incorporated by reference herein in its entirety.
Number | Date | Country | Kind |
---|---|---|---|
2022-190921 | Nov 2022 | JP | national |