Patent Application
20230395637
The aspect of the embodiment relates to a photoelectric conversion apparatus and a photoelectric conversion system using the photoelectric conversion apparatus.
International Publication No. 2017/169882 describes a photoelectric conversion apparatus provided with a through electrode for supplying voltage to a semiconductor substrate having a photoelectric conversion element formed therein.
Japanese Patent Laid-Open No. 2018-0201005 describes a photoelectric conversion apparatus provided with an electrode for supplying voltage to a semiconductor substrate having an avalanche photodiode formed therein.
According to the structure described in International Publication No. 2017/169882, the degree of freedom of arrangement of transistors is limited due to the presence of a through electrode. In addition, according to the structure described in Japanese Patent Laid-Open No. 2018-0201005, the degree of design freedom of the configurations of a portion in which an anode voltage is supplied to a substrate to maintain the DCR (Dark Count Rate) and a portion in which a cathode voltage is supplied to the substrate are limited.
According to an aspect of the embodiments, an apparatus includes a photodiode disposed in a first substrate having a first surface and a second surface opposite the first surface. The photodiode includes a first region of a first conductivity type disposed at a first depth, a second region of a second conductivity type disposed at a second depth that is deeper than the first depth with respect to the second surface, a third region disposed at a third depth that is deeper than the second depth with respect to the second surface, a fourth region in contact with the third region, a first conductive line connected to the first region and disposed adjacent to the second surface, where the first conductive line is used to read a signal from the first region, and a second conductive line disposed adjacent to the first surface. The difference between a potential applied to the first conductive line and a potential applied to the second conductive line is greater than or equal to a breakdown voltage.
According to another aspect of the embodiments, an apparatus includes a first substrate having a first surface on which light is incident and the second surface opposite the first surface and a second substrate stacked adjacent to the second surface of the first substrate. The first substrate includes an element including a first region of a first conductivity type and a second region of a second conductivity type, a third region in contact with the second region, a first conductive line connected to the first region and used to read out a signal from the first region, and a second conductive line provided adjacent to the first surface and used to supply a potential to the first substrate. The first substrate includes a third surface facing the second surface, a fourth surface opposite the third surface, and a transistor formed on the fourth surface. The transistor is a part of a pixel circuit that processes a signal output from the element.
Further features of the disclosure will become apparent from the following description of exemplary embodiments with reference to the attached drawings.
Embodiments described below are for embodying the technical concept of the disclosure and are not intended to limit the disclosure. The sizes and positional relationships of members illustrated in the drawings may be exaggerated for clarity of description. In the following description, the same configuration may be identified by the same reference numeral, and description may be omitted.
The embodiments of the disclosure are described in detail below with reference to the accompanying drawings. In the following description, the terms indicating specific directions and positions (for example, “upper”, “lower”, “right”, “left”, and other terms including these terms) are used as necessary. These terms are used to facilitate understanding of the embodiments with reference to the drawings, and the technical scope of the disclosure is not limited by the meanings of the terms.
As used herein, the term “plan view” is used to refer to a view in a direction perpendicular to the light incident surface of a semiconductor layer. The term “cross-sectional view” refers to a plane in a direction perpendicular to the light incident surface of the semiconductor layer. When the light incident surface of the semiconductor layer is microscopically rough, the plan view is defined on the basis of the light incident surface of the semiconductor layer when viewed macroscopically.
In the following description, the anode of an avalanche photodiode (APD) is set at a fixed electric potential, and a signal is taken from the cathode side. Therefore, a semiconductor region of a first conductivity type in which the charges of a polarity the same as that of a signal charge are majority carriers is an N-type semiconductor region, and a semiconductor region of a second conductivity type in which the charges of the polarity different from that of a signal charge are majority carriers is a P-type semiconductor region.
The disclosure can also be applied when the cathode of the APD is set at a fixed electric potential and the signal is taken from the anode side. In this case, the semiconductor region of the first conductivity type in which charges of a polarity the same as that of the signal charge are majority carriers is a P-type semiconductor region, and the semiconductor region of the second conductivity type in which charges of a polarity different from that of the signal charge are majority carriers is an N-type semiconductor region. Hereinafter, the case where one of the nodes of the APD is set to a fixed electric potential is described below. However, the potentials of both nodes may vary.
The term “impurity concentration” as simply used herein refers to the net impurity concentration obtained after subtracting the amount compensated by the impurity of an opposite conductivity type. That is, the term “impurity concentration” refers to the NET doping concentration. A region in which the P-type additive impurity concentration is higher than the N-type additive impurity concentration is a P-type semiconductor region. In contrast, a region in which the N-type additive impurity concentration is higher than the P-type additive impurity concentration is an N-type semiconductor region.
A configuration common to all embodiments of a photoelectric conversion apparatus and a method for driving the photoelectric conversion apparatus according to the disclosure are described below with reference to
The photoelectric conversion apparatus 100 is configured by stacking and electrically connecting a sensor substrate 11 with a circuit substrate 21. The sensor substrate 11 has a first semiconductor layer including photoelectric conversion elements 102 (described below) and a first wiring structure. The circuit substrate 21 has a second semiconductor layer including circuits, such as a signal processing unit 103 (described below), and a second wiring structure. The photoelectric conversion apparatus 100 is configured by stacking the second semiconductor layer, the second wiring structure, the first wiring structure, and the first semiconductor layer in this order. The photoelectric conversion apparatus described in each of the embodiments is a back-illuminated photoelectric conversion apparatus in which light is incident on a first surface and a circuit substrate is disposed on a second surface.
Although the sensor substrate 11 and the circuit substrate 21 in the form of diced chips are described below, the forms are not limited to chips. For example, the substrates may be wafers. In addition, the substrates in the form of wafers may be stacked and then diced or may be made into chips and then stacked and bonded.
The sensor substrate 11 has a pixel region 12 disposed therein, and the circuit substrate 21 has, disposed therein, a circuit region 22 for processing signals detected by the pixel region 12.
The pixels 101 are typically pixels for forming an image. However, when used for TOF (Time of Flight), the pixels 101 do not necessarily form an image. That is, the pixels 101 may be pixels for measuring the time and the amount of light when the light reaches the pixels 101.
The photoelectric conversion element 102 illustrated in
The vertical scanning circuit unit 110 receives a control pulse supplied from the control pulse generation unit 115 and supplies the control pulse to each of the pixels. Logic circuits, such as a shift register and an address decoder, are used in the vertical scanning circuit unit 110.
A signal output from the photoelectric conversion element 102 of the pixel is processed by the signal processing unit 103. The signal processing unit 103 includes a counter, a memory, and the like, and a digital value is held in the memory.
The horizontal scanning circuit unit 111 inputs a control pulse for sequentially selecting each of columns to the signal processing unit 103 to read a signal from the memory of each of the pixels that holds the digital signal.
The signal is output to the signal line 113 from the signal processing unit 103 of the pixel selected by the vertical scanning circuit unit 110 for the selected column.
The signal output to the signal line 113 is output to an external recording unit or the signal processing unit of the photoelectric conversion apparatus 100 via an output circuit 114.
In
As illustrated in
In
The APD 201 generates charge pairs in accordance with incident light by photoelectric conversion. A voltage VL (a first voltage) is supplied to the anode of the APD 201. In addition, a voltage VH (a second voltage) that is higher than the voltage VL supplied to the anode is supplied to the cathode of the APD 201. A reverse bias voltage (a voltage equal to or higher than the breakdown voltage) is supplied to the anode and cathode so that the APD 201 performs an avalanche multiplication operation. By supplying such voltages, charges generated by the incident light undergo avalanche multiplication, which generates an avalanche current.
When a reverse bias voltage is supplied, an APD has two modes of operation: the Geiger mode in which the potential difference between the anode and cathode is greater than the breakdown voltage and the linear mode in which the potential difference between the anode and cathode is less than or equal to the breakdown voltage.
An APD operated in the Geiger mode is referred to as an SPAD. For example, the voltage VL (the first voltage) is −30 V, and the voltage VH (the second voltage) is 1 V. The APD 201 may be operated in either the linear mode or the Geiger mode.
A quenching element 202 is connected to a power source that supplies the voltage VH and the APD 201. The quenching element 202 functions as a load circuit (a quenching circuit) during signal multiplication by avalanche multiplication, reduces the voltage supplied to the APD 201, and has a function of reducing avalanche multiplication (a quenching operation). In addition, the quenching element 202 has a function of causing a current corresponding to the voltage drop due to the quenching operation to flow and returning the voltage supplied to the APD 201 to the voltage VH (a recharge operation).
The signal processing unit 103 includes a waveform shaping unit 210, a counter circuit 211, and a selection circuit 212. Herein, the signal processing unit 103 can include one of the waveform shaping unit 210, the counter circuit 211, and the selection circuit 212. The signal processing unit 103 is also referred to as a “pixel circuit” that processes a signal output from the photoelectric conversion element.
The waveform shaping unit 210 shapes a change in the potential of the cathode of the APD 201 obtained during photon detection and outputs a pulse signal. For example, an inverter circuit is used as the waveform shaping unit 210. Although
The counter circuit 211 counts the pulse signals output from the waveform shaping unit 210 and holds a count value. Furthermore, when a control pulse pRES is supplied via a drive line 213, a count value held in the counter circuit 211 is reset.
The selection circuit 212 is supplied with a control pulse pSEL from the vertical scanning circuit unit 110 illustrated in
A switch, such as a transistor, may be provided between the quenching element 202 and the APD 201 or between the photoelectric conversion element 102 and the signal processing unit 103 to switch the electrical connection. Similarly, the voltage VH and the voltage VL supplied to the photoelectric conversion element 102 may be electrically switched using a switch, such as a transistor.
According to the present embodiment, the configuration using the counter circuit 211 is described. However, the photoelectric conversion apparatus 100 that acquires the pulse detection timing may be achieved by using a time-to-digital converter (hereinafter referred to as a TDC) and a memory instead of the counter circuit 211. At this time, the generation timing of the pulse signal output from the waveform shaping unit 210 is converted into a digital signal by the TDC. To measure the timing of the pulse signal, the TDC receives a control pulse pREF (a reference signal) supplied thereto from the vertical scanning circuit unit 110 illustrated in
Between time t0 and time t1, a potential difference of (VH−VL) is applied to the APD 201 illustrated in
It should be noted that the arrangement of the signal lines 113 and the arrangement of the column circuit 112 and the output circuit 114 are not limited to those illustrated in
While the description above has been made with reference to a photoelectric conversion element that is a pixel including an APD, the photoelectric conversion element may be a CMOS image sensor.
In
The signal processing unit 103 includes a transfer transistor TX, a floating diffusion FD, a reset transistor RES, a source follower transistor SF, and a selection transistor SEL. A control signal for controlling each of the transistors is input from the vertical scanning circuit unit 110 illustrated in
Furthermore, while the disclosure has been described with reference to the photoelectric conversion apparatus in which a plurality of semiconductor substrates are stacked, the photoelectric conversion apparatus may be a photoelectric conversion apparatus in which pixels and circuits are formed on the same semiconductor substrate.
A photoelectric conversion apparatus according to each of the embodiments is described below.
A photoelectric conversion apparatus according to the first embodiment is described below with reference to
The structure and function of the photoelectric conversion element 102 are described below. The photoelectric conversion element 102 includes an N-type first semiconductor region 311 and an N-type third semiconductor region 313. The photoelectric conversion elements 102 further includes a P-type fourth semiconductor region 314 and a P-type fifth semiconductor region 315.
According to the present embodiment, in the cross section illustrated in
Pixels are separated by a pixel separation portion 325 having a trench structure, and the P-type fourth semiconductor region 314 formed around the pixel separation portion 325 separates adjacent photoelectric conversion elements from each other by a potential barrier. Since the photoelectric conversion elements are also separated by the potential of the fourth semiconductor region 314, the trench structure, such as the pixel separation portion 325, is not essential for the pixel separation portion. Thus, when the pixel separation portion 325 having a trench structure is provided, the depth and position are not limited to those in the configuration illustrated in
A pinning film and a planarization film (neither is illustrated) are formed adjacent to the light incident surface of the semiconductor layer. Furthermore, microlenses ML are formed adjacent to the light incident surface. Still furthermore, a filter layer (not illustrated) or the like may be disposed adjacent to the light incident surface. As the filter layer, an optical filter, such as a color filter, an infrared cut filter, or a monochrome filter, can be used. As the color filter, an RGB color filter, an RGBW color filter, or the like can be used.
The wiring structure including a conductor and an insulating film is provided on the surface opposite the light incident surface of the semiconductor layer. An interlayer film, which is an insulating film, is provided between a conductive line and the semiconductor layer and between wiring layers.
According to the layout illustrated in
In addition, as illustrated in
In one embodiment, the contact electrode 324 and the semiconductor substrate is joined by low resistance ohmic contact. For this purpose, for example, the contact electrode 324 may be formed by implanting ions into the contact region from the light incident surface side and activating the contact region by laser annealing.
By employing such a structure, the width of the conductive line that covers the pixel separation portion 325 can be reduced than in the structure in which the pixel separation portions 325 surround the entire periphery of a pixel. Thus, incident light blocked by the conductive line can be reduced. Light loss due to a conductive line can be reduced particularly when pixels are tiny.
When a gap is provided between the pixel separation portions 325, a voltage can be supplied to the pixel by supplying a voltage to the fifth semiconductor region 315 in the outer peripheral region without forming the contact electrode 324 in the pixel region. However, a voltage drop occurs between the edge portion of the pixel region near the outer peripheral region and the central portion of the pixel region distant from the outer peripheral region, which may cause a difference between the voltages applied to the pixels.
Therefore, by providing the contact electrode 324 in the pixel region, a constant voltage can be provided to the pixels throughout the pixel region. Note that the density of the contact electrodes 324 may be reduced as compared with the density of the pixels. For example, one contact electrode 324 may be provided for a plurality of pixels (for example, four pixels).
In the cross-sectional view illustrated in
According to the configuration, the process of forming a pad opening in the substrate can be reduced, as compared with the configuration illustrated in
In the configuration illustrated in
A photoelectric conversion apparatus according to the second embodiment is described below with reference to
The first substrate and the second substrate are connected by a contact electrode (an inter-substrate contact), and a signal for driving the gate of a transfer transistor and a signal for outputting the voltage of the FD are transmitted across the substrates.
The photoelectric conversion apparatus illustrated in
Different voltages may be supplied to the wells of the first substrate and the wells of the second substrate.
By making the supplied voltages independent, for example, it is possible to separate the influences of instability of power source on the two substrates.
In the configuration illustrated in
A voltage supplied from the Pad electrode provided adjacent to the second substrate is supplied to the well of the first substrate via a through electrode or the like. Therefore, a signal for driving the gate of the transfer transistor, a signal for outputting the voltage of the FD, and the well potential are transmitted between the first substrate and the second substrate. The through electrode for transmitting the well potential between the first substrate and the second substrate is required to be formed in the peripheral portion of the pixel, for example, and each of the pixels need not have a conductive line. One through electrode may be provided for a plurality of pixels.
According to the configuration, the man-hours for the Pad opening process can be reduced as compared with that required for the configuration illustrated in
A photoelectric conversion apparatus according to the third embodiment is described below with reference to
The first substrate and the second substrate are connected by bonding metal conductive lines. A first bonding portion included in a first wiring structure of the first substrate is bonded to a second bonding portion included in a second wiring structure of the second substrate at a bonding plane between the first wiring structure and the second wiring structure. In addition, a first insulating member included in the first wiring structure is bonded to a second insulating member included in the second wiring structure at the bonding plane between the first wiring structure and the second wiring structure. An example of the metal conductive line is a Cu-contained conductive line.
As in
As illustrated in
A photoelectric conversion apparatus according to the fourth embodiment is described below with reference to
The structure and function of the photoelectric conversion element 102 are described below. The photoelectric conversion element 102 includes a first semiconductor region 311, a third semiconductor region 313, and a sixth semiconductor region 316 of N-type. The photoelectric conversion element 102 further includes a second semiconductor region 312, a fourth semiconductor region 314, and a fifth semiconductor region 315 of P-type.
According to the present embodiment, as illustrated in the cross section illustrated in
The first semiconductor region 311 has a higher N-type impurity concentration than the third semiconductor region 313.
A PN junction is formed between the P-type second semiconductor region 312 and the N-type first semiconductor region 311. By making the impurity concentration of the second semiconductor region 312 lower than that of the first semiconductor region 311, the entire portion of the second semiconductor region 312 that overlaps the center of the first semiconductor region in plan view is turned into a depletion layer region. At this time, the potential difference between the first semiconductor region 311 and the second semiconductor region 312 is greater than the potential difference between the second semiconductor region 312 and the third semiconductor region 313. In addition, the depletion layer region extends to a part of the first semiconductor region 311, and a strong electric field is induced in the extended depletion layer region. The strong electric field causes avalanche multiplication in the depletion layer region extending to the part of the first semiconductor region 311, and an electric current based on the amplified charge is output as signal charge. When light incident on the photoelectric conversion element 102 is photoelectrically converted and avalanche multiplication occurs in the depletion layer region (an avalanche multiplication region), the generated charge of the first conductivity type is collected in the first semiconductor region 311.
Although in
In addition, the sixth semiconductor region 316 may be a P-type semiconductor region instead of an N-type semiconductor region.
In this case, the P-type impurity concentration of the sixth semiconductor region 316 is set lower than the P-type impurity concentration of the second semiconductor region 312. This is because if the impurity concentration of the sixth semiconductor region 316 is too high, an avalanche multiplication region is formed between the sixth semiconductor region 316 and the first semiconductor region 311, increasing the dark count rate (DCR).
The pixels are separated by a pixel separation portion 325 having a trench structure, and the P-type fourth semiconductor region 314 formed around the pixel separates adjacent photoelectric conversion elements by a potential barrier. Since the photoelectric conversion elements are also separated by the potential of the fourth semiconductor region 314, a trench structure such as the pixel separation portion 325 is not essential as a pixel separation portion. When the pixel separation portion 325 having the trench structure is provided, the depth and position of the trench structure are not limited to those in the configuration illustrated in
A pinning film and a planarization film (not illustrated) are further formed adjacent to the light incident surface of the semiconductor layer. Furthermore, a microlenses ML are formed adjacent to the light incident surface. Still furthermore, a filter layer (not illustrated) or the like may be disposed adjacent to the light incident surface. As the filter layer, an optical filter, such as a color filter, an infrared cut filter, or a monochrome filter, can be used. As the color filter, an RGB color filter, an RGBW color filter, or the like can be used.
A wiring structure including a conductor and an insulating film is provided on the surface opposite the light incident surface of the semiconductor layer. An interlayer film, which is the insulating film, is provided between a conductive line and the semiconductor layer and between wiring layers.
In the photoelectric conversion apparatus according to the present embodiment, an anode line is provided adjacent to the light incident surface of the semiconductor substrate, and a cathode line is provided adjacent to the surface opposite the light incident surface. The cathode line supplies a first voltage (a cathode voltage) to the first semiconductor region 311, and the anode line supplies a second voltage (an anode voltage) to the fourth semiconductor region 314 via the contact electrode 324.
When the anode line and the cathode line are provided in the same wiring layer, the wiring layout is limited in order to maintain the withstand voltage between the conductive lines. According to the present configuration, since the anode line and the cathode line are provided in different wiring layers with the substrate therebetween, the wiring layout is highly flexible, which is advantageous in reducing the pixel size, for example.
As illustrated in
In the case of an SPAD, a potential difference of nearly 30 V is formed between the first semiconductor region 311 to which the cathode voltage is supplied and the fourth semiconductor region 314 to which the anode voltage is supplied in order to cause avalanche multiplication to occur. For this reason, in one embodiment, the first semiconductor region 311 and the fourth semiconductor region 314 are located as far apart as possible. However, the pixel size can be reduced with decreasing distance between the first semiconductor region 311 and the fourth semiconductor region 314.
When the anode line is disposed on the first surface and the anode potential is supplied from the light incident surface side as in the photoelectric conversion apparatus according to the present embodiment, formation of the fourth semiconductor region 314 on the second surface is not needed. This configuration enables further reduction in the size of the pixel.
To reduce the size of the pixel, the fourth semiconductor region 314 is not provided in the vicinity of the second surface in the pixel region. The fourth semiconductor region 314 may be formed in the outer peripheral region.
In
For the purpose of light reflection, the same effect can be obtained by extending the cathode line. However, to speed up the SPAD operation, it is advantageous to keep the parasitic capacitance on the cathode electrode small. Therefore, in one embodiment, it is to make the size of the cathode electrode minimized. For this reason, it is effective to use, as a dummy line, a line other than the cathode line.
A photoelectric conversion apparatus according to the fifth embodiment is described below with reference to
The first substrate and the second substrate are connected by contact electrodes 324.
A photoelectric conversion apparatus according to the sixth embodiment is described below with reference to
The first substrate and the second substrate are connected by bonding metal conductive lines. In a bonding plane between the first substrate and the second substrate, a metal conductive line exposed from the surface of the first substrate is bonded to a metal conductive line exposed from the surface of the second substrate. Similarly, in the bonding plane, an insulating member on the surface of the first substrate is bonded to an insulating member on the surface of the second substrate. An example of the metal conductive line is a Cu-contained conductive line.
A photoelectric conversion system according to the present embodiment is described below with reference to
The photoelectric conversion apparatuses described in each of the first to sixth embodiments can be applied to a variety of photoelectric conversion systems. Examples of an applicable photoelectric conversion system include a digital still camera, a digital camcorder, a surveillance camera, a copying machine, a facsimile, a mobile phone, on-vehicle camera, and an observation satellite. In addition, a camera module including an optical system, such as a lens, and an image pickup apparatus is included in the photoelectric conversion systems.
The photoelectric conversion system illustrated in
The photoelectric conversion system further includes a signal processing unit 1007 that serves as an image generation unit that generates an image by processing an output signal output from the image pickup apparatus 1004. The signal processing unit 1007 performs various corrections and compressions as necessary and outputs image data. The signal processing unit 1007 may be formed in a semiconductor substrate having the image pickup apparatus 1004 provided therein or may be formed in a semiconductor substrate other than the semiconductor substrate having the image pickup apparatus 1004 therein.
The photoelectric conversion system further includes a memory unit 1010 for temporarily storing image data and an external interface unit (external I/F unit) 1013 for communicating with an external computer or the like. Still furthermore, the photoelectric conversion system includes a recording medium 1012, such as a semiconductor memory, for recording and reading image data therein and therefrom, and a recording medium control interface unit (recording medium control I/F unit) 1011 for recording or reading data in and from the recording medium 1012. The recording medium 1012 may be built in the photoelectric conversion system or may be removable.
Furthermore, the photoelectric conversion system includes an overall control/calculation unit 1009 that performs control of various calculations and overall control of the digital still camera and a timing generation unit 1008 that outputs various timing signals to the image pickup apparatus 1004 and the signal processing unit 1007. The timing signal and the like may be input from the outside, and the photoelectric conversion system can include at least the image pickup apparatus 1004 and the signal processing unit 1007 that processes the output signal output from the image pickup apparatus 1004.
The image pickup apparatus 1004 outputs an image pickup signal to the signal processing unit 1007. The signal processing unit 1007 performs predetermined signal processing on the image pickup signal output from the image pickup apparatus 1004 and outputs image data. The signal processing unit 1007 generates an image using the image pickup signal.
As described above, according to the present embodiment, a photoelectric conversion system that employs the photoelectric conversion apparatus (the image pickup apparatus) of any one of the above-described embodiments can be achieved.
A photoelectric conversion system and a mobile object according to the present embodiment are described below with reference to
Alternatively, the distance information acquisition unit may be implemented by a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or combinations thereof.
The photoelectric conversion system 2300 is connected to a vehicle information acquisition apparatus 2320. Thus, the photoelectric conversion system 2300 can acquire vehicle information, such as a vehicle speed, a yaw rate, and a steering angle. In addition, the photoelectric conversion system 2300 is connected to a control ECU 2330 which is a control unit that outputs a control signal for generating a braking force to the vehicle on the basis of the determination result of the collision determination unit 2318. Furthermore, the photoelectric conversion system 2300 is connected to an alarm device 2340 that emits an alarm to a driver on the basis of the determination result of the collision determination unit 2318. For example, if the collision determination unit 2318 determines that the collision probability is high, the control ECU 2330 performs vehicle control to avoid collisions or reduce damage by braking, releasing the accelerator pedal, or reducing the engine output. The alarm device 2340 emits an alarm by, for example, sounding the alarm, displaying alarm information on a screen of a car navigation system, or vibrating a seat belt or steering wheel.
According to the present embodiment, the photoelectric conversion system 2300 captures the image of the surroundings of the vehicle, for example, the front view or rear view of the vehicle.
While an example of performing control so as not to collide with another vehicle has been described, the configuration can also be applied to control of self-driving vehicles to follow another vehicle or control of self-driving vehicles to keep the lane. Furthermore, the photoelectric conversion system can be applied to autonomous driving control for, for example, following another vehicle and keeping the lane. Furthermore, the photoelectric conversion system can be applied not only to a vehicle, such as one's own vehicle, but also to a mobile object (a moving apparatus), such as a boat, an aircraft, or an industrial robot. Still furthermore, the photoelectric conversion system can be applied not only to a mobile object but also to equipment that uses object recognition over a wide area, such as an intelligent transportation system (ITS).
A photoelectric conversion system according to the present embodiment is described with reference to
As illustrated in
The optical system 407 includes one or more lenses. The optical system 407 guides image light (incident light) from the object to the photoelectric conversion apparatus 403 and forms an image on the light receiving surface (a sensor unit) of the photoelectric conversion apparatus 403.
As the photoelectric conversion apparatus 403, the photoelectric conversion apparatus of any one of the embodiments described above is applied, and a distance signal indicating the distance obtained from the received light signal output from the photoelectric conversion apparatus 403 is supplied to the image processing circuit 404.
The image processing circuit 404 performs image processing to construct a range image based on the distance signal supplied from the photoelectric conversion apparatus 403. The range image (image data) obtained through the image processing is supplied to the monitor 405 and is displayed. In addition, the range image is supplied to the memory 406 and is stored (recorded).
In the range image sensor 401 configured in this way, by applying the above-described photoelectric conversion apparatus, it is possible to obtain, for example, a more accurate range image in accordance with improvement of the characteristics of the pixels.
The photoelectric conversion system according to the present embodiment is described below with reference to
The endoscope 1100 is composed of a lens barrel 1101, a predetermined length of the front end of which is to be inserted into the body cavity of the patient 1132, and a camera head 1102, which is connected to the base end of the lens barrel 1101. In
An opening having an objective lens fitted thereinto is provided at the front end of the lens barrel 1101. A light source device 1203 is connected to the endoscope 1100, and light generated by the light source device 1203 is guided to the front end of the lens barrel 1101 by a light guide extending inside the lens barrel 1101. The light is emitted to an observation object in the body cavity of the patient 1132 through the objective lens. The endoscope 1100 may be a straight-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.
An optical system and a photoelectric conversion apparatus are provided inside the camera head 1102, and the reflected light (observation light) from the observation object is focused on the photoelectric conversion apparatus by the optical system. The photoelectric conversion apparatus photoelectrically converts the observation light to generate an electrical signal corresponding to the observation light. That is, an image signal corresponding to the observation image is generated. As the photoelectric conversion apparatus, the photoelectric conversion apparatus described in any one of the above embodiments can be used. The image signal is transmitted to a camera control unit (CCU) 1135 in the form of RAW data.
The CCU 1135 includes a central processing unit (CPU), a graphics processing unit (GPU), and the like. The CCU 1135 comprehensively controls the operations performed by the endoscope 1100 and a display device 1136. Furthermore, the CCU 1135 receives an image signal from the camera head 1102 and performs various image processing, such as development processing (demosaicing), for displaying an image based on the image signal.
Under the control of the CCU 1135, the display device 1136 displays an image based on the image signal subjected to image processing performed by the CCU 1135.
The light source device 1203 includes a light source, such as a light emitting diode (LED), and supplies the endoscope 1100 with irradiation light for capturing the image of a surgical site or the like.
An input device 1137 is an input interface to the endoscopic surgery system 1150. A user can input a variety of information and instructions to the endoscopic surgery system 1150 via the input device 1137.
A treatment tool control device 1138 controls driving of an energy treatment tool 1112 for tissue cauterization, incision, blood vessel sealing, or the like.
The light source device 1203 that supplies irradiation light to the endoscope 1100 when the image of a surgical site is captured can include, for example, a white light source, such as an LED, a laser light source, or combinations thereof. When the white light source is configured by a combination of R, G, and B laser light sources, the output intensity and output timing of each of the colors (each of the wavelengths) can be controlled with high accuracy. Thus, white balance of a captured image can be adjusted in the light source device 1203. In this case, the observation target is irradiated with laser light from each of the R, G, and B laser light sources in a time-division manner, and driving of an image pickup element of the camera head 1102 is controlled in synchronization with the irradiation timing. In this way, an image corresponding to each of the RGB colors can be captured in a time-division manner. According to the technique, a color image can be obtained without providing a color filter on the image pickup element.
In addition, the driving of the light source device 1203 may be controlled such that the intensity of the output light is changed at predetermined time intervals. A high dynamic range without so-called crushed shadows and blown out highlights can be generated by controlling the driving of the image pickup elements of the camera head 1102 in synchronization with the timing of the change in the intensity of the light, acquiring images in a time-division manner, and combining the images.
In addition, the light source device 1203 may be configured so as to be able to supply light in a predetermined wavelength band range corresponding to special light observation. In special light observation, the wavelength dependency of light absorption by a body tissue is used, for example. More specifically, a high contrast image of a predetermined tissue, such as a blood vessel on the surface of the mucous membrane, is captured by irradiating the tissue with light in a narrower band than the irradiation light used during normal observation (that is, white light).
Alternatively, in special light observation, fluorescence observation may be performed in which an image is captured using fluorescence generated by irradiation with excitation light. In fluorescence observation, a body tissue is irradiated with excitation light, and fluorescence from the body tissue can be observed. Alternatively, a reagent, such as indocyanine green (ICG), is locally injected into the body tissue, and the body tissue is irradiated with excitation light corresponding to the fluorescence wavelength of the reagent. Thus, a fluorescent image can be obtained. The light source device 1203 can be configured so as to supply narrowband light and/or excitation light corresponding to the special light observation.
A photoelectric conversion system according to the present embodiment is described below with reference to
The glasses 1600 further include a control device 1603. The control device 1603 functions as a power source that supplies electric power to the photoelectric conversion apparatus 1602 and the display device. In addition, the control device 1603 controls the operations performed by the photoelectric conversion apparatus 1602 and the display device. The lens 1601 has an optical system formed therein to focus light onto the photoelectric conversion apparatus 1602.
The user's line of sight to the displayed image is detected from the captured infrared light images of the eyeball. Any known technique can be applied to line-of-sight detection using captured images of eyeballs. As an example, an eye gaze detection technique based on a Purkinje image obtained using reflection of irradiation light on the cornea.
More specifically, line-of-sight detection processing is performed on the basis of the pupillary-corneal reflection technique. The user's line of sight is detected by calculating a line of sight vector representing the orientation (the rotational angle) of the eyeball on the basis of the pupil image and the Purkinje image included in the captured eyeball image by using the pupillary-corneal reflection technique.
The display device according to the present embodiment may include a photoelectric conversion apparatus including a light receiving element and may control the display image of the display device on the basis of the user's line-of-sight information obtained from the photoelectric conversion apparatus.
More specifically, the display device determines a first field of view region that the user gazes at and a second field of view region other than the first field of view region on the basis of the line-of-sight information. The first field of view region and the second field of view region may be determined by a control unit of the display device. Alternatively, a first field of view region and a second field of view region determined by an external control device may be received. In the display area of the display device, the display resolution of the first field of view region may be controlled to be higher than the display resolution of the second field of view region. That is, the resolution of the second field of view region may be set to lower than that of the first field of view region.
Furthermore, the display area may have a first display area and a second display area different from the first display area, and a higher priority one of the first display area and the second display area may be determined on the basis of the line of sight information. The first field of view region and the second field of view region may be determined by the control unit of the display device or an external control device. Alternatively, a first field of view region and a second field of view region determined by an external control device may be received. The resolution of a high priority area may be set higher than the resolution of an area other than the high priority area. That is, the resolution of a relatively low priority area may be decreased.
Artificial intelligence (AI) may be used to determine the first field of view region and a high priority area. AI model may be a model configured to estimate the angle of the line of sight and the distance to an object in the line of sight from the eyeball image by using, as training data, eyeball images and the directions in which the eyeballs in the images are actually looking. The AI program may be stored in the display device, the photoelectric conversion apparatus, or an external device. When stored in the external device, the AI program is transmitted to the display device via communication.
In the case of display control based on visual recognition detection, the display control can be applied to smart glasses that further include a photoelectric conversion apparatus that captures the image of the outside. The smart glasses can display captured external information in real time.
The disclosure is not limited to the above embodiments, and various modifications can be made.
For example, an example in which part of the configuration of any one of the embodiments is added to another embodiment and an example in which part of the configuration of any one of the embodiments is replaced by part of another embodiment are also included in embodiments of the disclosure.
In addition, the photoelectric conversion systems according to the seventh embodiment and the eighth embodiment are examples of photoelectric conversion systems to which the photoelectric conversion apparatus can be applied, and a photoelectric conversion system to which the photoelectric conversion apparatus according to the disclosure can be applied is not limited to the configurations illustrated in
It should be noted that the above-described embodiments merely illustrate specific examples for carrying out the disclosure, and the technical scope of the disclosure should not be construed to be limited by the embodiments. That is, the disclosure can be carried out in various forms without departing from its technical concept or main features.
According to the disclosure, it is possible to increase the flexibility of placement of the transistors.
While the disclosure has been described with reference to exemplary embodiments, it is to be understood that the disclosure is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Application No. 2022-091510 filed Jun. 6, 2022, which is hereby incorporated by reference herein in its entirety.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2022-091510 | Jun 2022 | JP | national |
