Patent Grant
10404952
This application claims the benefit under 35 U.S.C. § 119 of a Korean patent application filed on Jul. 24, 2015 in the Korean Intellectual Property Office and assigned Serial number 10-2015-0105135, the entire disclosure of which are hereby incorporated by reference.
The present disclosure relates to an image sensor, and for example, to a configuration of an image sensor and a signal processing method thereof.
The image sensor is an element which converts an optical image into an electric signal. Thanks to the development of the computer industry and the communication industry, the demand for an image sensor having improved performance is increasing in various fields such as digital cameras, camcorders, personal communication systems (PCSs), game devices, security service cameras, medical micro cameras, robots, etc.
The image sensor may include a color filter. The image sensor may detect light corresponding to a specific wavelength through the color filter. For example, the image sensor may detect light corresponding to a red wavelength through a red filter. In addition, the image sensor may detect light corresponding to a blue wavelength through a blue filter.
A single pixel of the image sensor may include a micro lens, a single color filter, and a photodiode corresponding to the single color filter. Accordingly, light passing through the micro lens may be converted into light having a specific wavelength through the color filter, and then may be delivered to the photodiode. Through this process, light having wavelengths different from the specific wavelength may be blocked by the color filter and thus may not be delivered to the photodiode. Therefore, the sensitivity of light may deteriorate in the photodiode in part.
The above information is presented as background information only to assist with an understanding of the present disclosure. No determination has been made, and no assertion is made, as to whether any of the above might be applicable as prior art with regard to the present disclosure.
Aspects of the present disclosure are to address at least the above-mentioned problems and/or disadvantages and to provide at least the advantages described below. Accordingly, an aspect of the present disclosure is to provide a pixel structure of an image sensor which can receive light of various wavelengths.
Another aspect of the present disclosure is to provide a pixel structure of an image sensor which can provide a clear image when a background for photographing has low illumination.
In accordance with an aspect of the present disclosure, an image sensor is provided. The image sensor includes a plurality of pixels, wherein at least one pixel of the plurality of pixels includes a micro lens, a filter wherein the filter includes a first area for filtering a first color signal from a light signal transmitted through the micro lens, and a second area for filtering a second color signal from the transmitted signal, a first photodiode for converting the first color signal into an electric signal, and a second photodiode for converting the second color signal into an electric signal.
In accordance with another aspect of the present disclosure, an electronic device is provided. The electronic device includes an image sensor including a plurality of pixels, and a processor functionally connected with the image sensor, wherein at least one pixel of the plurality of pixels includes a micro lens, a filter wherein the filter includes a first area for filtering a first color signal from a light signal transmitted through the micro lens, and a second area for filtering a second color signal from the transmitted signal, and the electronic device further includes a first photodiode for converting the first color signal into an electric signal, and a second photodiode for converting the second color signal into an electric signal, and wherein the processor is configured to generate brightness information and color information based on at least the first color signal and the second color signal which are converted into the electric signals, and output an image corresponding to the brightness information and the color information.
In accordance with another aspect of the present disclosure, a signal processing method is provided. The signal processing method includes receiving a light signal through a single micro lens included in at least one pixel included in an image sensor, filtering a white signal and a color signal from the light signal through a filter including a first area and a second area; converting the white signal and the color signal into electric signals through a plurality of photodiodes, and outputting an image corresponding to brightness information and color information which are determined based on the electric signals through a display.
Other aspects, advantages, and salient features of the disclosure will become apparent to those skilled in the art from the following detailed description, which, taken in conjunction with the annexed drawings, discloses various embodiments of the present disclosure.
The above and other aspects, features, and advantages of certain embodiments of the present disclosure will be more apparent from the following description taken in conjunction with the accompanying drawings, in which:
Throughout the drawings, it should be noted that like reference numbers are used to depict the same or similar elements, features, and structures.
The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of various embodiments of the present disclosure as defined by the claims and their equivalents. It includes various specific details to assist in that understanding but these are to be regarded as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the various embodiments described herein can be made without departing from the scope and spirit of the present disclosure. In addition, descriptions of well-known functions and constructions may be omitted for clarity and conciseness.
The terms and words used in the following description and claims are not limited to the bibliographical meanings, but, are merely used by the inventor to enable a clear and consistent understanding of the present disclosure. Accordingly, it should be apparent to those skilled in the art that the following description of various embodiments of the present disclosure is provided for illustration purpose only and not for the purpose of limiting the present disclosure as defined by the appended claims and their equivalents.
It is to be understood that the singular forms “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to “a component surface” includes reference to one or more of such surfaces.
In the present disclosure, the expression “have”, “may have”, “include” or “may include” refers to existence of a corresponding feature (e.g., numerical value, function, operation, or components such as elements), and does not exclude existence of additional features.
In the present disclosure, the expression “A or B,” “at least one of A or/and B,” or “one or more of A or/and B” may include all possible combinations of the items listed. For example, the expression “A or B,” “at least one of A and B,” or “at least one of A or B” refers to all of (1) including at least one A, (2) including at least one B, or (3) including all of at least one A and at least one B.
The expression “a first,” “a second,” “the first,” or “the second” used in various embodiments of the present disclosure may modify various components regardless of the order and/or the importance but does not limit the corresponding components. For example, a first electronic device and a second electronic device may indicate different user devices, regardless of order or importance thereof. For example, a first element may be interchangeably referred to as a second element, and similarly, a second element may be interchangeably referred to as a first element without departing from the scope of the present disclosure.
It should be understood that when an element (e.g., a first element) is referred to as being (operatively or communicatively) “connected” or “coupled” to another element (e.g., second element), it may be directly connected or coupled directly to the other element. In such a situation, alternatively, any other element (e.g., third element) may be interposed between them. In certain embodiments of the present disclosure, it may be understood that when an element (e.g., first element) is referred to as being “directly connected,” or “directly coupled” to another element (second element), there are no element (e.g., third element) interposed between them (while there can be a connecting element, such as an adhesive or a connector between them).
The expression “configured to” used in the present disclosure may be interchangeably used with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of,” depending on the context. The term “configured to” may not necessarily imply “specifically designed to” in hardware. Alternatively, in some situations, the expression “device configured to” may mean that the device, together with other devices or components, “is able to.” For example, the phrase “processor adapted (or configured) to perform A, B, and C” may mean a dedicated processor (e.g. embedded processor) only for performing the corresponding operations or a generic-purpose processor (e.g., central processing unit (CPU) or application processor (AP)) that can perform the corresponding operations by executing one or more software programs stored in a memory device.
The terms used herein are merely for the purpose of describing particular embodiments and are not intended to limit the scope of other embodiments. As used herein, singular forms may include plural forms as well unless the context clearly indicates otherwise. Unless defined otherwise, all terms used herein, including technical and scientific terms, have the same meaning as those commonly understood by a person skilled in the art to which the present disclosure pertains. Such terms as those defined in a generally used dictionary may be interpreted to have the meanings equal to the contextual meanings in the relevant field of art, and are not to be interpreted to have ideal or excessively formal meanings unless clearly defined in the present disclosure. In some cases, even the term defined in the present disclosure should not be interpreted to exclude embodiments of the present disclosure.
An electronic device according to various embodiments of the present disclosure may include at least one of, for example, a smart phone, a tablet personal computer (PC), a mobile phone, a video phone, an electronic book reader (e-book reader), a desktop PC, a laptop PC, a netbook computer, a workstation, a server, a personal digital assistant (PDA), a portable multimedia player (PMP), a Moving Picture Experts Group phase 1 or phase 2 (MPEG-1 or MPEG-2) audio layer-3 (MP3) player, a mobile medical device, a camera, and a wearable device. According to various embodiments of the present disclosure, the wearable device may include at least one of an accessory type (e.g., a watch, a ring, a bracelet, an anklet, a necklace, a glasses, a contact lens, or a head-mounted device (HMD)), a fabric or clothing integrated type (e.g., an electronic clothing), a body-mounted type (e.g., a skin pad, or tattoo), and a bio-implantable type (e.g., an implantable circuit).
According to some embodiments of the present disclosure, the electronic device may be a home appliance. The home appliance may, for example, include at least one of a television, a digital versatile disc (DVD) player, an audio player, a refrigerator, an air conditioner, a cleaner, an oven, a microwave oven, a washing machine, an air purifier, a set-top box, a home automation control panel, a television (TV) box (e.g., HomeSync™ of Samsung, Apple TV™, or Google TV™), a game console (e.g., Xbox™, PlayStation™), an electronic dictionary, an electronic key, a camcorder, and an electronic frame.
According to another embodiment of the present disclosure, the electronic device may include at least one of various medical devices (e.g., various portable medical measuring devices (a blood glucose measuring device, a heart rate measuring device, a blood pressure measuring device, a body temperature measuring device, etc.), a magnetic resonance angiography (MRA), a magnetic resonance imaging (MRI), a computed tomography (CT) machine, and an ultrasonic machine), a navigation device, a global navigation satellite system (GNSS) receiver, an event data recorder (EDR), a flight data recorder (FDR), a vehicle infotainment devices, an electronic devices for a ship (e.g., a navigation device for a ship, and a gyro-compass), avionics, security devices, an automotive head unit, a robot for home or industry, an automatic teller's machine (ATM) in banks, point of sales (POS) in a shop, or internet device of things (e.g., a light bulb, various sensors, electric or gas meter, a sprinkler device, a fire alarm, a thermostat, a streetlamp, a toaster, a sporting goods, a hot water tank, a heater, a boiler, etc.).
According to some embodiments of the present disclosure, the electronic device may include at least one of a part of furniture or a building/structure, an electronic board, an electronic signature-receiving device, a projector, and various kinds of measuring instruments (e.g., a water meter, an electric meter, a gas meter, and a radio wave meter). The electronic device according to various embodiments of the present disclosure may be a combination of one or more of the aforementioned various devices. The electronic device according to some embodiments of the present disclosure may be a flexible device. Further, the electronic device according to an embodiment of the present disclosure is not limited to the aforementioned devices, and may include a new electronic device according to the development of technology.
Hereinafter, an electronic device according to various embodiments will be described with reference to the accompanying drawings. As used herein, the term “user” may indicate a person who uses an electronic device or a device (e.g., an artificial intelligence electronic device) that uses an electronic device.
Referring to
The processor 110 may run an operating system or an application program to control a plurality of hardware or software constituent elements connected to the processor 110, and may perform processing and operation of various data including multimedia data. The processor 110 may be, for example, implemented as a system on chip (SoC). According to an embodiment of the present disclosure, the processor 110 may further include a graphical processing unit (GPU) (not shown). The processor 110 may further includes at least one of other constitute elements (ex: the cellular module 121) drown in
According to an embodiment of the present disclosure, the processor 110 may receive color information from an image sensor (not shown) included in the camera module 191. For example, the color information may include brightness information and primary color information (or color information). For example, the brightness information may be information on light which is filtered through a transparent filter area of a light filter of the image sensor (not shown).
According to an embodiment of the present disclosure, thereafter, the processor 110 may convert the received color information into a YUV signal, store the YUV signal in the memory 130 or convert the YUV signal into a red, green, blue (RGB) signal, and display the RGB signal on the display 160.
According to another embodiment of the present disclosure, the processor 110 may receive the brightness information from the image sensor (not shown). For example, the processor 110 may detect a phase difference of light using the brightness information, and may move the lens of the camera module 191 to adjust the focus of the camera module 191 based on the detected phase difference.
According to an embodiment of the present disclosure, the communication module 120 may include a cellular module 121, a Wi-Fi module 123, a Bluetooth (BT) module 125, a GNSS module 127 (ex: a global positioning system (GPS)) module, a glonass module, a beidou module, or a Galileo module), a near field communication (NFC) module 128, and a radio frequency (RF) module 129.
The cellular module 121 may provide a voice telephony, a video telephony, a text service, an Internet service, and the like, through a telecommunication network (e.g., long term evolution (LTE), LTE-advance (LTE-A), code division multiple access (CDMA), wideband CDMA (WCDMA), universal mobile telecommunications system (UMTS), wireless broadband (WiBro), global system for mobile (GSM), and the like). In addition, the cellular module 121 may, for example, use a SIM (e.g., the SIM card 124) to perform electronic device distinction and authorization within the telecommunication network. According to an embodiment of the present disclosure, the cellular module 121 may perform at least some of functions that the AP 110 may provide. For example, the cellular module 121 may include a communication processor (CP).
The Wi-Fi module 123, the BT module 125, the GPS module 127 or the NFC module 128 each may include, for example, a processor for processing data transmitted/received through the corresponding module. According to an embodiment of the present disclosure, at least some (e.g., two or more) of the cellular module 121, the Wi-Fi module 123, the BT module 125, the GPS module 127 or the NFC module 128 may be included within one integrated circuit (IC) or IC package.
The RF module 129 may perform transmission/reception of data, for example, transmission/reception of an RF signal. Though not illustrated, the RF module 129 may include, for example, a transceiver, a power amplifier module (PAM), a frequency filter, a low noise amplifier (LNA), an antenna and the like. According to an embodiment of the present disclosure, at least one of the cellular module 121, the Wi-Fi module 123, the BT module 125, the GPS module 127 or the NFC module 128 may perform transmission/reception of a radio frequency (RF) signal through a separate RF module.
The SIM card 124 may be a card including a SIM, and may be inserted into a slot provided in a specific position of the electronic device 101. The SIM card 124 may include unique identification information (e.g., an integrated circuit card ID (ICCID)) or subscriber information (e.g., an international mobile subscriber identity (IMSI)).
The memory 130 may include an internal memory 132 or an external memory 134. The internal memory 132 may include, for example, at least one of a volatile memory (e.g., a dynamic random access memory (DRAM), a static RANI (SRAM) and a synchronous DRAM (SDRAM)) or a non-volatile memory (e.g., a one-time programmable read only memory (OTPROM), a programmable ROM (PROM), an erasable and programmable ROM (EPROM), an electrically erasable and programmable ROM (EEPROM), a mask ROM, a flash ROM, a not and (NAND) flash memory, and a not or (NOR) flash memory).
According to an embodiment of the present disclosure, the internal memory 132 may be a solid state drive (SSD). The external memory 134 may further include a flash drive, for example, compact flash (CF), secure digital (SD), micro-SD, mini-SD, extreme digital (xD), a memory stick, and the like. The external memory 134 may be operatively connected with the electronic device 101 through various interfaces.
The sensor module 140 may measure a physical quantity or detect an activation state of the electronic device 101, and convert measured or detected information into an electric signal. The sensor module 140 may include, for example, at least one of a gesture sensor 140A, a gyro sensor 140B, an air pressure sensor 140C, a magnetic sensor 140D, an acceleration sensor 140E, a grip sensor 140F, a proximity sensor 140G a color sensor 140H (e.g., a RGB sensor), a bio-physical sensor 140I, a temperature/humidity sensor 140J, an illumination sensor 140K, a ultraviolet (UV) sensor 140M, and the like. Additionally or alternatively, the sensor module 140 may include, for example, an E-nose sensor (not shown), an electromyography (EMG) sensor (not shown), an electroencephalogram (EEG) sensor (not shown), an electrocardiogram (ECG) sensor (not shown), an infrared (IR) sensor (not shown), an iris sensor (not shown), a fingerprint sensor (not shown), and the like. The sensor module 140 may further include a control circuit for controlling at least one or more sensors belonging therein. In various embodiments of the present disclosure, an electronic device 101 may further include a processor configured to control the sensor module 140 as a part of or separately from the processor 110, and may control the sensor module 140 while the processor 110 is in a sleep state.
The input device 150 may include a touch panel 152, a (digital) pen sensor 154, an input key 156, an ultrasonic input device 158, and the like. The touch panel 152 may, for example, detect a touch input in at least one of a capacitive overlay scheme, a pressure sensitive scheme, an infrared beam scheme, or an acoustic wave scheme. In addition, the touch panel 152 may further include a control circuit as well. In a case of the capacitive overlay scheme, physical contact or proximity detection is possible. The touch panel 152 may further include a tactile layer as well. In this case, the touch panel 152 may provide a tactile response to a user.
The (digital) pen sensor 154 may be implemented in the same or similar method to receiving a user's touch input or by using a separate sheet for detection. The input key 156 may include, for example, a physical button, an optical key, or a keypad. The ultrasonic input device 158 is a device capable of identifying data by detecting a sound wave in the electronic device 101 through an input tool generating an ultrasonic signal, and enables wireless detection.
The display 160 (e.g., the display 160) may include a panel 162, a hologram device 164, or a projector 166. The panel 162 may be, for example, a liquid crystal display (LCD), an active-matrix organic light emitting diode (AMOLED), and the like. The panel 162 may be, for example, implemented to be flexible, transparent, or wearable. The panel 162 may be constructed as one module along with the touch panel 152 as well. The hologram device 164 may use interference of light to show a three-dimensional image in the air. The projector 166 may project light to a screen to display an image. The screen may be, for example, located inside or outside the electronic device 101. According to an embodiment of the present disclosure, the display 160 may further include a control circuit for controlling the panel 162, the hologram device 164, or the projector 166.
The interface 170 may include, for example, a high definition multimedia interface (HDMI) 172, a universal serial bus (USB) 174, an optical interface 176, or a D-subminiature (D-sub) 178. Additionally or alternatively, the interface 170 may include, for example, a mobile high-definition link (MHL) interface, an SD card/multimedia card (MMC) interface or an infrared data association (IrDA) standard interface.
The audio module 180 may convert a voice and an electric signal interactively. The audio module 180 may, for example, process sound information which is inputted or outputted through a speaker 182, a receiver 184, an earphone 186, the microphone 188, and the like.
The camera module 191 is a device able to take a still picture and a moving picture. According to an embodiment of the present disclosure, the camera module 191 may include one or more image sensors (e.g., a front sensor or a rear sensor), a lens (not shown), an image signal processor (ISP) (not shown), or a flash (not shown) (e.g., an LED or a xenon lamp).
The power management module 195 may manage electric power of the electronic device 101. Though not illustrated, the power management module 195 may include, for example, a power management integrated circuit (PMIC), a charger IC, a battery, a fuel gauge, and the like.
The PMIC may be, for example, mounted within an IC or a SoC semiconductor. A charging scheme may be divided into a wired charging scheme and a wireless charging scheme. The charger IC may charge the battery 296, and may prevent the inflow of overvoltage or overcurrent from an electric charger. According to an embodiment of the present disclosure, the charger IC may include a charger IC for at least one of the wired charging scheme or the wireless charging scheme. The wireless charging scheme may, for example, be a magnetic resonance scheme, a magnetic induction scheme, an electromagnetic wave scheme, and the like. A supplementary circuit for wireless charging, for example, a circuit, such as a coil loop, a resonance circuit, a rectifier, and the like, may be added.
The indicator 197 may display a specific status of the electronic device 101 or one part (e.g., the processor 110) thereof, for example a booting state, a message state, a charging state, and the like. The motor 198 may convert an electric signal into a mechanical vibration. Though not illustrated, the electronic device 101 may include a processing device (e.g., a GPU) for mobile TV support. The processing device for mobile TV support may, for example, process media data according to the standards of digital multimedia broadcasting (DMB), digital video broadcasting (DVB), a media flow, and the like.
Each of the above-described elements of the electronic device according to various embodiments of the present disclosure may include one or more components, and the name of a corresponding element may vary according to the type of electronic device. The electronic device according to various embodiments of the present disclosure may include at least one of the above-described elements and may exclude some of the elements or further include other additional elements. Further, some of the elements of the electronic device according to various embodiments of the present disclosure may be coupled to form a single entity while performing the same functions as those of the corresponding elements before the coupling.
Referring to
According to an embodiment of the present disclosure, the front camera module 201 may be provided on the front surface of the electronic device 200, and may photograph a still image and a moving image regarding a subject located in front of the electronic device 200. The rear camera module 203 may be provided on the rear surface of the electronic device 200, and may photograph a still image or a moving image regarding a subject located behind the electronic device 200.
According to an embodiment of the present disclosure, the display 205 may output a still image or a moving image under the control of the processor 110. For example, the display 205 may output the moving image inputted from the front camera module 201 or the rear camera module 203 in the form of a preview screen.
Referring to
According to an embodiment of the present disclosure, the front camera module 230 may include one or more lenses 231 and an image sensor 233. According to an embodiment of the present disclosure, light entering through the opening 213 may be inputted to the image sensor 233 through the one or more lenses 231. The image sensor 233 may convert the incoming light (for example, a white signal or a color signal) into an electric signal.
Referring to
According to an embodiment of the present disclosure, a single pixel 305 included in the pixel structure 303 may include a single micro lens, a filter, and a plurality of photodiodes 307, 309. For example, the filter may include a white filter area and a color filter area. According to an embodiment of the present disclosure, the vertical scanning circuit 311 and the horizontal scanning circuit 313 may detect signals (for example, white color signals or color signals) of the plurality of photodiodes under the control of the processor 110 or an ISP. According to an embodiment of the present disclosure, the deciphering unit 315 may read out the detected signals.
Referring to
According to an embodiment of the present disclosure, the pre-processor 403 may be configured as a single independent IC chip or may be included in at least one of the image sensor 401 and the ISP 405. For example, the pre-processor 403 may perform a bad pixel correction (BPC) or reconstruction function. According to an embodiment of the present disclosure, the pre-processor 403 may receive a digital signal from the image sensor 401, convert the received digital signal into a YUV signal, or transmit the YUV signal to the ISP 405.
According to an embodiment of the present disclosure, the pre-processor 403 may receive the RGBW signal and convert the received RGBW signal into a YUV signal. For example, the pre-processor 403 may generate the YUV signal using the T or W components included in the RGBW signal.
For example, the YUV signal may be influenced by the sensitivity of a white filter area and a primary color filter area.
Referring to
For example, a light filter may include a transparent filter area (T or W), a red filter area (R), a green filter area (G), a blue filter area (B), or a color filter area which allows light of other colors to pass therethrough or blocks light of other colors. Since these color filter areas have different transmittance according to wavelengths of incident light, the sensitivity ratios of light (or exposure ratios) may be different.
Graphs 501 show the sensitivity ratios of the color filters. For example, according to graphs 501, the white filter area (T or W) may have high exposure ratios in comparison with the red filter area (R), the green filter area (G), and the blue filter area (B). In addition, the exposure ratios in the red filter area (R), the green filter area (G), and the blue filter area (B) may be similar to one another. In addition, light of a wider wavelength 503 may be transmitted through the white filter area (T or W) in comparison with the red filter area (R), the green filter area (G), and the blue filter area (B). In addition, light of wavelengths 505, 507, 509 having an almost similar range may be transmitted through the red filter area (R), the green filter area (G), and the blue filter area (B).
The amount of light exposed during the same exposure time may be higher in a photodiode located to correspond to the white filter area (T or W) than in a photodiode located to correspond to the primary color filter area. In addition, when light enters the pixel, the photodiode located to correspond to the white filter area (T or W) may receive light of more diverse wavelengths than the photodiode located to correspond to the primary color filter area. Accordingly, the photodiode located to correspond to the white filter area may react more sensitively to the brightness of light than the photodiode located to correspond to the primary color filter area. Therefore, when the YUV signal is generated using the T or W component included in the RGBW signal, the pre-processor 403 may generate an image of high sensitivity.
Referring to
In another example, the pre-processor 403 may calculate the value of the brightness signal (Y) using only the value of the W signal. In another example, the pre-processor 403 may calculate a value of a first brightness signal (Y1) using only the value of the W signal, calculate a value of a second brightness signal (Y2) using the value of at least one of the R, G and B signals, and calculate the brightness signal (Y) by adding Y1 and Y2.
In another example, the pre-processor 403 may calculate the value of the brightness signal (Y) by changing a mixing ratio between the value of at least one of the R, G, and B signals and the value of the W signal according to illumination of an external environment. For example, when the illumination of the external environment is lower than pre-designated reference illumination, the pre-processor 403 may determine that the external environment has low illumination, and calculate the value of the brightness signal (Y) using only the value of the W signal or may calculate the value of the brightness value (Y) by increasing the mixing ratio of the value of the W signal in comparison to the value of at least one of the R, G, and B signals. Herein, the reference illumination may indicate illumination which is experimentally measured when the external environment has low illumination. In addition, the pre-processor 403 may calculate the value of the color difference signal (U) based on the value of the B signal and the calculated value of the Y signal, and calculate the value of the color difference signal (V) based on the value of the R signal and the calculated value of the Y signal.
In another example, when the external environment has illumination higher than or equal to the reference illumination, the pre-processor 403 may determine that the external environment is bright enough, and may calculate the value of the brightness signal (Y) using only the value of at least one of the R, G, and B signals or may calculate the value of the brightness signal (Y) by increasing the mixing ratio of the value of at least one of the R, G, and B signals in comparison to the value of the W signal.
According to an embodiment of the present disclosure, the pre-processor 403 may generate phase difference information of light based on a difference between a plurality of W components outputted from the plurality of diodes included in one pixel, and transmit the phase difference information to a processor (for example, the processor 110) for controlling the camera module or the ISP 405 so as to adjust the focus by moving the lens of the camera module (for example, the camera module 191) based on the phase difference information. According to an embodiment of the present disclosure, the pre-processor 403 may generate 3-dimension (3D) information of an image included in a frame by processing the phase difference information on a frame basis, and transmit the generated image 3D information to the image signal processor 405.
According to an embodiment of the present disclosure, the ISP 405 may receive the YUV signal on the frame basis, stores the received YUV signal in a memory (for example, the memory 130) or convert the YUV signal into an RGB signal, and display the RGB signal on a display (for example, the display 160). According to an embodiment of the present disclosure, the ISP 405 may receive the phase difference information and adjust the focus of the camera module by moving the lens of the camera module based on the received phase difference information. According to an embodiment of the present disclosure, the ISP 405 may receive the image 3D information, generate a 3D image of a corresponding frame based on the received 3D information, and output the image on the display or store the image in the memory.
According to an embodiment of the present disclosure, the pixel structure may include a plurality of pixels. Each of the plurality of pixels may include a single micro lens, a light filter, and a plurality of photodiodes.
The light filter may be an optical element which controls characteristics of light by performing transmission, reflection, split, extraction, isolation, or separation with respect to light. The light filter may be an optical element which extracts (or isolates) a component of a specific wavelength from white light or light in which various wavelengths are mixed. According to an embodiment of the present disclosure, the light filter may include a color filter area. The color filter area may have a color for passing only light of a specific color in natural light (mixed light). The light filter may include a red (R) filter area, a green (G) filter area, or a blue (B) filter area. According to an embodiment of the present disclosure, the light filter may include a transparent filter area or a color filter area. The transparent filter area may be nearly colorless (transparent). In another example, the transparent filter area may have a color having transparency of 90% or higher. According to an embodiment of the present disclosure, the filter may include both the transparent filter (or white filter) area and the color filter area having at least one color of R, G, and B.
According to an embodiment of the present disclosure, the pixel structure may include a single micro lens per pixel, and the structure of the light filter and the number of photodiodes may vary according to the pixel structure. According to an embodiment of the present disclosure, as shown in
The transparent filter area and the color filter area in the single pixel do not overlap each other and may have various forms. According to an embodiment of the present disclosure, the transparent filter area may enclose an entirety or part of the color filter area. For example, as shown in
Referring to
According to an embodiment of the present disclosure, as shown in
According to an embodiment of the present disclosure, as shown in
According to an embodiment of the present disclosure, the color filter area 731 may occupy an area of a predetermined size with reference to the center of the single pixel 723, and the two transparent filter areas 727, 729 may occupy the upper/lower areas with reference to the color filter area 731. For example, the pixel 723 may include the light filter including a first transparent filter area 727 and a second transparent filter area 729. The first transparent filter area 727 may be located on the upper end of the pixel 723 and the second transparent filter area 729 may be located on the lower end of the pixel 723. The first transparent filter area 727 and the second transparent filter area 729 may enclose at least part (or the entire area) of the color filter area 731 included in the pixel 723.
According to an embodiment of the present disclosure, as shown in
According to an embodiment of the present disclosure, the color filter area 751 may occupy the middle area from among the three areas which are formed by vertically dividing the single pixel 743 into three equal parts, and the transparent filter areas 747, 749 may occupy the other left/right areas, such that the two transparent filter areas 747, 749 enclose a part of the color filter area 751. For example, the pixel 743 may include the light filter including a first transparent filter area 747 and a second transparent filter area 749. The first transparent filter area 747 may be located on the left of the pixel 743 and the second transparent filter area 749 may be located on the right of the pixel 743. The color filter area 751 included in the pixel 743 may be located in the middle between the first and second transparent filter areas 747, 749.
According to an embodiment of the present disclosure, the color filter area 771 may occupy the middle area from among the three areas which are formed by horizontally dividing the single pixel 763 into three equal parts, and the transparent filter areas 767, 769 may occupy the other upper/lower areas. For example, the pixel 763 may include the light filter including a first transparent filter area 767 and a second transparent filter area 769. The first transparent filter area 767 may be located on the upper end of the pixel 763 and the second transparent filter area 769 may be located on the lower end of the pixel 763. The color filter area 771 included in the pixel 763 may be located in the middle between the first and second transparent filter areas 767, 769.
According to an embodiment of the present disclosure, referring to a pixel structure 801 of
According to an embodiment of the present disclosure, referring to a pixel structure 821 of
According to an embodiment of the present disclosure, the color filter areas 831, 833 may occupy two of the four areas which are formed by dividing the single pixel 823 into four equal parts, and the transparent filter areas 827, 829 may occupy the other two areas. For example, the light filter included in the pixel 823 may include a first transparent filter area 827, a second transparent filter area 829, a first color filter area 831, and a second color filter area 833. The first transparent filter area 827 may be located on the left upper end of the pixel 823. The second transparent filter area 829 may be located on the right of the first transparent filter area 827 (for example, on the right upper end of the pixel 823). The first color filter area 831 may be located on the left lower end of the pixel 823. The second color filter area 833 may be located on the right lower end of the pixel 823.
According to an embodiment of the present disclosure, the first color filter area 831 and the second color filter area 833 may be the same color filter areas. For example, both the first color filter area 831 and the second color filter area 833 may be red filter areas.
According to an embodiment of the present disclosure, referring to a pixel structure 841 of
For example, the pixel 843 may include the light filter including a first color filter 847, a second color filter 853, a first transparent filter 849, and a second transparent filter 851. The first color filter 847 may be located on the left upper end of the pixel 843. The second color filter 853 may be located on the right lower end of the pixel 843. For example, the first color filter 847 may be located on the diagonal line from the second color filter 853. The first transparent filter 849 may be located on the right upper end of the pixel 843. The second transparent filter 851 may be located on the left lower end of the pixel 843. For example, the first transparent filter 849 may be located on the diagonal line from the second transparent filter 851.
According to an embodiment of the present disclosure, referring to a pixel structure 861 of
For example, the pixel 863 may include the light filter including a first color filter area 869, a second color filter area 873, a third color filter area 871, and the transparent filter area 867. The first color filter area 869 may be located on the right upper end of the pixel 863. The second color filter area 873 may be located on the lower end of the first color filter area 869. The third color filter area 871 may be located on the left of the second color filter area 873. The transparent filter area 867 may be located on the left upper end of the pixel 863.
According to an embodiment of the present disclosure, the first, second, and third color filter areas 869, 873, 871 may be the same color filter areas. For example, all of the first, second, and third color filter areas 869, 873, 871 may be red filter areas.
According to an embodiment of the present disclosure, as shown in
According to an embodiment of the present disclosure, the color filter area may be one of a green filter area, a red filter area, and a blue filter area. According to an embodiment of the present disclosure, the transparent filter areas may be white filter areas.
According to an embodiment of the present disclosure, the single color filter area and the single transparent filter area in the pixel may be the same or substantially the same in view of the size. Herein, being the same or substantially the same may refer to a case in which a difference in the size between the single color filter area and the single transparent filter area falls within a pre-designated range (about 90%).
When the single color filter area and the single transparent filter area are the same in view of the size as described above, a size ratio between the transparent filter area and the color filter area of the pixel structure may be 1:1 or 1:N, for example. For example, as shown in
In another example, as shown in
For example, as shown in
In another example, as shown in
According to a certain embodiment of the present disclosure, the color filter area and the transparent filter area may be different from each other in view of the size. For example, the size of the transparent filter area may be larger than the size of the color filter area. In another example, the size of the color filter area may be larger than the size of the transparent filter area. According to an embodiment of the present disclosure, the number of transparent filter areas may correspond to the number of color filter areas in the pixel structure.
The image sensor (for example, the image sensor 401) includes a plurality of pixels, and at least one pixel of the plurality of pixels may include photodiodes, a light filter which is stacked on the photodiodes and includes color filter and transparent filter areas, and a single micro lens.
According to an embodiment of the present disclosure, the image sensor may convert light which is received through the light filter including the color filter and transparent filter areas into a digital signal using the plurality of photodiodes on a pixel basis, and transmit the converted electric signal to a pre-processor (for example, the pre-processor 403). For example, when a single pixel area is divided into at least two areas, the image sensor may generate digital signals for the respective areas and transmit the generated digital signals.
In explaining the elements in detail, according to an embodiment of the present disclosure, light entering the micro lens 1003 may be collected through the micro lens 1003 and transmitted to the light filter 1013. For example, the light filter 1013 may include the transparent filter area and the color filter area. For example, the color filter area may be one of a green filter area, a red filter area, and a blue filter area. For example, the transparent filter area may be a white filter area. Referring to
According to an embodiment of the present disclosure, the light filter 1013 may be stacked on first and second photodiodes 1007, 1009 of the image sensor circuit unit 1001, and the color filter area and the transparent filter area may be stacked on the first and second photodiodes 1007, 1009, respectively. For example, as shown in
According to an embodiment of the present disclosure, the image sensor circuit unit 1001 may include the first and second photodiodes 1007, 1009. The first and second photodiodes 1007, 1009 may convert light (for example, a white signal or a color signal) entering through the light filter 1013 into an electric signal. For example, the first and second photodiodes 1007, 1009 may convert the energy of incident light into a voltage.
According to an embodiment of the present disclosure, the second photodiode 1009 having a predetermined size may be located on the center of the image sensor circuit unit 1001, and the first photodiode 1007 may be located on the other part. According to an embodiment of the present disclosure, a barrier 1005 may be located on the border of the pixel and between the first and second photodiodes 1007, 1009 to prevent at least part of the incident light from entering other pixels or other photodiode areas.
In addition, the sizes of the areas occupied by the first and second photodiodes 1007, 1009 may be substantially the same. Herein, being substantially the same may refer to a case in which a difference in the size between the areas of the first and second photodiodes 1007, 1009 falls within a pre-designated range (about 90%). In this case, the ratio between the areas of respective filter colors in the single pixel may be 1 (T or W area):1 (C area).
For example, reviewing the cross section view of the pixel with reference to the A direction by referring to
In explaining the elements in detail, according to an embodiment of the present disclosure, light entering the micro lens 1103 may be collected through the micro lens 1103 and transmitted to the light filter 1115. For example, the light filter 1115 may include the transparent filter area and the color filter area. For example, the color filter area may be one of a green filter area, a red filter area, and a blue filter area. For example, the transparent filter area may be a colorless or white filter area.
Referring to
According to an embodiment of the present disclosure, the light filter 1115 may be stacked on first to third photodiodes 1107, 1109, 1111 of the image sensor circuit unit 1101, and the color filter area and the transparent filter area may be stacked on the first, second and third photodiodes 1107, 1109, 1111, respectively. For example, as shown in
According to an embodiment of the present disclosure, the image sensor circuit unit 1101 may include the first to third photodiodes 1107, 1109, 1111. The first to third photodiodes 1107, 1109, 1111 may convert light entering through the light filter 1115 into an electric signal. For example, the first to third photodiodes 1107, 1109, 1111 may convert the energy of incident light into a voltage.
According to an embodiment of the present disclosure, the third photodiode 1111 having a predetermined size may be located on the center of the image sensor circuit unit 1101, and the first and second photodiodes 1107, 1109 may be located on the other part. For example, as shown in
In addition, the sizes of the areas occupied by the first to third photodiodes 1107, 1109, 1111 may be substantially the same. Herein, being substantially the same may refer to a case in which a difference in the size between the areas of the first to third photodiodes 1107, 1109, 1111 falls within a pre-designated range (about 90%). In this case, the ratio between the areas of respective filter colors in the single pixel may be 2 (T or W area):1 (C area).
For example, reviewing the cross section view of the pixel with reference to the A direction by referring to
In explaining the elements in detail, according to an embodiment of the present disclosure, light entering the micro lens 1203 may be collected through the micro lens 1203 and transmitted to the light filter 1215. For example, the light filter 1215 may include the transparent filter area and the color filter area. For example, the color filter area may be one of a green filter area, a red filter area, and a blue filter area. For example, the transparent filter area may be a colorless or white filter area.
Referring to
According to an embodiment of the present disclosure, the light filter 1215 may be stacked on first to third photodiodes 1207, 1209, 1211 of the image sensor circuit unit 1201, and the color filter area and the transparent filter area may be stacked on the first to third photodiodes 1207, 1209, 1211, respectively. For example, as shown in
According to an embodiment of the present disclosure, the image sensor circuit unit 1201 may include the first to third photodiodes 1207, 1209, 1211. The first to third photodiodes 1207, 1209, 1211 may convert light entering through the light filter 1215 into an electric signal. For example, the first to third photodiodes 1207, 1209, 1211 may convert the energy of incident light into a voltage.
According to an embodiment of the present disclosure, the single pixel area may be vertically or horizontally divided into three areas, and the first to third photodiodes 1207, 1209, 1211 may be located on the three areas, respectively. For example, as shown in
For example, the sizes of the areas occupied by the first to third photodiodes 1207, 1209, 1211 may be substantially the same. Herein, being substantially the same may refer to a case in which a difference in the size between the areas of the first to third photodiodes 1207, 1209, 1211 falls within a pre-designated range (about 90%). In this case, the ratio between the areas of respective filter colors in the single pixel may be 2 (T or W area):1 (C area).
For example, reviewing the cross section view of the pixel with reference to the A direction by referring to
Referring to
According to an embodiment of the present disclosure, the light filter 1317 may be stacked on the plurality of photodiodes of the image sensor circuit unit 1301, and the color filter area and the transparent filter area may be stacked on the plurality of photodiodes. For example, as shown in
According to an embodiment of the present disclosure, the image sensor circuit unit 1301 may include the first, second, third and fourth photodiodes 1307, 1309, 1311, 1313. The first to fourth photodiodes 1307, 1309, 1311, 1313 may convert light entering through the light filter 1317 into an electric signal. For example, the first to fourth photodiodes 1307, 1309, 1311, 1313 may convert the energy of incident light into a voltage.
According to an embodiment of the present disclosure, the single pixel area may be vertically divided into two parts and horizontally divided into two parts, thereby being divided into four areas, and the first to fourth photodiodes 1307, 1309, 1311, 1313 may be located on the four areas, respectively. For example, as shown in
For example, the sizes of the areas occupied by the first to fourth photodiodes 1307, 1309, 1311, 1313 may be substantially the same. Herein, being substantially the same may refer to a case in which a difference in the size between the areas of the first to fourth photodiodes 1307, 1309, 1311, 1313 falls within a pre-designated range (about 90%). In this case, the ratio between the areas of respective filter colors in the single pixel may be 3 (T or W area):1 (C area).
According to an embodiment of the present disclosure, a barrier 1305 may be located on the border of the pixel or between the first to fourth photodiodes 1307, 1309, 1311, 1313 to prevent the incident light from entering other pixels or other photodiode areas.
For example, reviewing the cross section view of the pixel with reference to the A direction by referring to
In explaining the elements in detail, according to an embodiment of the present disclosure, light entering the micro lens 1403 may be collected through the micro lens 1403 and transmitted to the light filter 1419. For example, the light filter 1419 may include the transparent filter area and the color filter area. For example, the color filter area may be one of a green filter area, a red filter area, and a blue filter area. For example, the transparent filter area may be a colorless or white filter area.
Referring to
According to an embodiment of the present disclosure, the light filter 1419 may be stacked on the plurality of photodiodes of the image sensor circuit unit 1401, and the color filter area and the transparent filter area may be stacked on the plurality of photodiodes. For example, as shown in
According to an embodiment of the present disclosure, the image sensor circuit unit 1401 may include the first, second, third, fourth and fifth photodiodes 1407, 1409, 1411, 1413, 1415. The first to fifth photodiodes 1407, 1409, 1411, 1413, 1415 may convert light entering through the light filter 1419 into an electric signal. For example, the first to fifth photodiodes 1407, 1409, 1411, 1413, 1415 may convert the energy of incident light into a voltage.
According to an embodiment of the present disclosure, the single pixel area may be divided into five areas. For example, as shown in
For example, the sizes of the areas occupied by the first to fifth photodiodes 1407, 1409, 1411, 1413, 1415 may be substantially the same. Herein, being substantially the same may refer to a case in which a difference in the size between the areas of the first to fifth photodiodes 1407, 1409, 1411, 1413, 1415 falls within a pre-designated range (about 90%). In this case, the ratio between the areas of respective filter colors in the single pixel may be 4 (T or W area):1 (C area).
According to an embodiment of the present disclosure, a barrier 1405 may be located on the border of the pixel or between the first to fifth photodiodes 1407, 1409, 1411, 1413, 1415 to prevent the incident light from entering other pixels or other photodiode areas.
For example, reviewing the cross section view of the pixel with reference to the A direction by referring to
Various pixel structures of the image sensor have been described up to now with reference to
In another example, at least one pixel structure may remove the transparent filter area of the light filter or may integrate the transparent filter area into the micro lens.
Referring to
In explaining the elements in detail, according to an embodiment of the present disclosure, the image sensor 1501 may be a device which converts light entering a lens of the camera module into an electric signal.
For example, a light filter may include a transparent filter area and a color filter area. For example, the color filter area may include at least one of a green filter area, a red filter area, and a blue filter area. For example, the transparent filter area may include at least one of a colorless filter area and a white filter area. For example, when the light filter includes the green filter area, the red filter area, the blue filter area, and the transparent filter area, the light filter may be referred to as an RGBW color filter.
For example, the image sensor 1501 may transmit an RGBW signal corresponding to RGBW colors of the light filter to the pre-processor 1503. For example, the image sensor 1501 may transmit color information including brightness information for each pixel of each frame and primary color information to the pre-processor 1503. For example, the brightness information may be information on the light which is filtered through the transparent filter area of the light filter.
For example, as shown in
According to an embodiment of the present disclosure, the data capacity of the color information may vary according to the number of photodiodes included in one pixel. For example, when two photodiodes are included in one pixel and the capacity of data outputted from one photodiode is 8 bits, the data capacity of the color information may be 16 bits.
According to an embodiment of the present disclosure, the pre-processor 1503 may receive the color information from the image sensor 1501, convert the color information into a YUV signal, and transmit the YUV signal to the ISP 1505. The ISP 1505 may receive the YUV signal on the frame basis, store the received YUV signal in a memory (for example, the memory 130) or convert the YUV signal into an RGB signal, and display the RGB signal on a display (for example, the display 160).
Referring to
In explaining the elements in detail, according to an embodiment of the present disclosure, the image sensor 1601 may be a device which converts light entering through a lens of the camera module into an electric signal. For example, the image sensor 1601 may transmit an RGBW signal corresponding to RGBW colors of a light filter to the pre-processor 1603. For example, the image sensor 1601 may transmit color information including brightness information for each pixel of each frame and primary color information to a color converter 1605 of the pre-processor 1603. For example, when three photodiodes are included in one pixel and a transparent filter area is stacked on two of the three photodiodes and a color filter area is stacked on the other diode, the image sensor 1601 may transmit the color information (for example, Pm,n(T1+T2+C(R or G or B))) including the brightness information corresponding to the plurality of photodiodes of each pixel, and the primary color information corresponding to one photodiode and related to one of red, green, and blue, as shown in
According to an embodiment of the present disclosure, the data capacity of the color information may vary according to the number of photodiodes included in one pixel. For example, when the pixel includes three photodiodes and the capacity of data outputted from one diode is 8 bits, the data capacity of the color information including the brightness information corresponding to two diodes and the primary color information corresponding to one diode may be 24 bits.
According to an embodiment of the present disclosure, the image sensor 1601 may transmit the brightness information of each pixel of each frame to a phase difference detector 1607 of the pre-processor 1603. For example, when the pixel includes three photodiodes and the transparent filter area is stacked on two of the three photodiodes, the image sensor 1601 may transmit the brightness information (for example, Pm,n(T1, T2)) corresponding to the plurality of photodiodes of each pixel as shown in
According to an embodiment of the present disclosure, the pre-processor 1603 may include the color converter 1605 and the phase difference detector 1607. For example, the color converter 1605 may receive the color information (for example, Pm,n(T1+T2+C(R or G or B))) from the image sensor 1601, convert the received color information into a YUV signal, and transmit the YUV signal to the ISP 1609.
According to an embodiment of the present disclosure, the phase difference detector 1607 may receive the brightness information (for example, Pm,n(T1, T2)) from the image sensor 1601, detect a phase difference of light based on the brightness information, and transmit the detected phase difference to the camera module controller 1611.
When at least a plurality of W components are included in the RGBW signal which is received on the pixel basis, the phase difference detector 1607 may generate phase difference information of light based on a difference between the plurality of W components, and transmit the phase difference information to the processor 110 for controlling the camera module 191 or the camera module controller 1611, such that the lens of the camera module 191 is moved based on the phase difference information to adjust a focus. For example, the phase difference detector 1607 may generate 3D information of an image included in the frame by processing the phase difference information on the frame basis, and transmit the generated image 3D information to the ISP 1609.
According to an embodiment of the present disclosure, the ISP 1609 may receive the YUV signal on the frame basis, store the received YUV signal in the memory 130 or convert the YUV signal into an RGB signal, and display the RGB signal on a display (for example, the display 160). For example, the ISP 1609 may receive the image 3D information from the phase difference detector 1607, generate a 3D image of a corresponding frame based on the received 3D information, and output the 3D image on the display or store the 3D image in the memory (for example, the memory 130).
According to an embodiment of the present disclosure, the camera module controller 1611 may receive the phase difference information from the phase difference detector 1607, and may adjust the focus of the camera module by moving the lens of the camera module based on the received phase difference information.
Referring to
In explaining the elements in detail, according to an embodiment of the present disclosure, the image sensor 1701 may convert light entering through a lens of the camera module 191 into an electric signal. For example, the image sensor 1701 may transmit an RGBW signal corresponding to RGBW colors of a light filter to the pre-processor 1703. For example, the image sensor 1701 may transmit color information including brightness information for each pixel of each frame and primary color information to a color converter 1705 of the pre-processor 1703. For example, when the pixel may include four photodiodes and a transparent filter area is stacked on three of the four photodiodes and a primary color filter is stacked on the other photodiode, the image sensor 1701 may transmit the color information (for example, Pm,n(T1+T2+T3+C(R or G or B))) including the brightness information corresponding to the three photodiodes of each pixel, and the primary color information corresponding to one photodiode and related to one of red, green, and blue, as shown in
According to an embodiment of the present disclosure, the data capacity of the color information may vary according to the number of photodiodes included in one pixel. For example, when the pixel includes four photodiodes and the capacity of data outputted from one diode is 8 bits, the data capacity of the color information including the brightness information corresponding to three diodes, and the primary color information corresponding to one diode may be 32 bits.
According to an embodiment of the present disclosure, the image sensor 1701 may transmit the brightness information of each pixel of each frame to a phase difference detector 1707 of the pre-processor 1703. For example, when the pixel includes four photodiodes and the transparent filter area is stacked on three of the four photodiodes, the image sensor 1701 may transmit the brightness information (for example, Pm,n(T1, T2, T3)) corresponding to the plurality of photodiodes of each pixel as shown in
According to an embodiment of the present disclosure, the pre-processor 1703 may include the color converter 1705 and the phase difference detector 1707. For example, the color converter 1705 may receive the color information (for example, Pm,n(T1+T2+T3+C(R or G or B))) from the image sensor 1701, convert the received color information into a YUV signal, and transmit the YUV signal to the ISP 1713.
According to an embodiment of the present disclosure, the phase difference detector 1707 may include a horizontal phase difference detector 1709 and a vertical phase difference detector 1711. For example, the phase difference detector 1707 may receive the brightness information (for example, Pm,n(T1, T2, T3)) from the image sensor 1701, detect horizontal and vertical phase differences of light based on the brightness information, and transmit the detected horizontal and vertical phase differences to the camera module controller 1715.
For example, the horizontal phase difference detector 1709 may generate horizontal phase difference information of light using brightness information (for example, T1, T2) on a pair of photodiodes (for example, 1307, 1309 of
According to an embodiment of the present disclosure, the horizontal phase difference detector 1709 may compare brightness information (for example, T1 and T2) detected from the photodiodes (for example, photodiodes 1307, 1309 of FIG. 13) located on the horizontal line, and generate horizontal phase difference information based on the result of the comparing. According to an embodiment of the present disclosure, the vertical phase difference detector 1711 may compare brightness information (for example, T1 and T3) detected from the photodiodes (for example, photodiodes 1307, 1311 of
According to an embodiment of the present disclosure, the ISP 1713 may generate a 3D image based on the image 3D information, and display the 3D image or store it in a memory (for example, the memory 160). The ISP 1713 performs almost similar operations to those of the ISP 1609 of
According to an embodiment of the present disclosure,
Referring to
According to an embodiment of the present disclosure, the photodiodes D1, D2, D3, D4 may receive light passing through the same micro lens. For example, three of the four photodiodes D1, D2, D3 may receive light passing through a transparent filter area, and generate an electric charge corresponding to the received light energy. The other photodiode D4 may receive light passing through a color filter area and generate an electric charge corresponding to the received light energy.
According to an embodiment of the present disclosure, the electric charges generated in the photodiodes D1, D2, D3, D4 may be transmitted to a floating diffusion area 1813 through the transmission switches 1805, 1807, 1809, 1811. According to an embodiment of the present disclosure, a selection switch (RS) 1801 may select a corresponding pixel from among at least a plurality of pixels included in the image sensor. According to an embodiment of the present disclosure, a reset switch (RST) 1803 may reset the floating diffusion area 1813. According to an embodiment of the present disclosure, when the RST 1803 resets the floating diffusion area 1813 and then one of the transmission switches 1805, 1807, 1809, 1811 transmits the electric charge to the floating diffusion area 1813, an amplification switch (SF) 1815 may output the electric potential of the floating diffusion area 1813 as an output voltage of the corresponding transmission switch 1805, 1807, 1809, 1811.
For example, when the transparent filter area of the light filter is stacked on the photodiodes D1 to D3 and the color filter area of the light filter is stacked on the photodiode D4, the output voltages of the photodiodes D1 to D4 may be components of the color information outputted from the image sensor 1701 of
Referring to
According to an embodiment of the present disclosure, when the selection switch 1801 generates the reset pulse, the transmission switches 1805, 1807, 1809, and 1811 may transmit the electric charges generated in the photodiodes D1, D2, D3, and D4 to the floating diffusion area 1813 in sequence by generating a pulse (hereinafter, referred to as a “transmission pulse”) having a high level during a pre-designated time in sequence according to a pre-designated order. In addition, when one of the transmission switches 1805, 1807, 1809, 1811) transmits the electric charge to the floating diffusion area 1813, the SF 1815 may output the electric potential of the floating diffusion area 1813 as an output voltage of the corresponding transmission switch.
Referring to
In the first graph 1901, the signals intensities 1903 of T1, T2 and T3 are the same or similar, and the signal intensity 1905 of C is different from the signal intensities 1903 of T1, T2 and T3. The same signal intensities 1903 of T1, T2 and T3 indicate that the horizontal and vertical phase difference of light is small (for example, there is no horizontal and vertical phase difference), and that the camera module 191 is in focus. In this case, a camera module controller (for example, the camera module controller 1715) may not move the lens of the camera module.
In the second graph 1907, the signal intensities 1909, 1911, 1913, 1915 of T1, T2, T3 and C are different from one another. The different signal intensities 1909, 1911, 1915 of T1, T2 and T3 may mean that there is a horizontal or vertical phase difference of light, and that the camera module 191 is out of focus. In this case, the camera module controller (for example, the camera module controller 1715) may adjust the focus of the camera module by moving the lens of the camera module based on the horizontal and vertical phase difference.
In the third graph 1917, the signal intensities 1919, 1921, 1923, 1925 of T1, T2, T3 and C are different from one another as in the second graph 1907. However, for example, the signal intensities 1919, 1923, 1925 of T1, T2, T3 may be higher than the signal intensities 1909, 1911, 1915 of T1, T2, T3 of the second graph 1907. The different signal intensities 1919, 1923, 1925 of T1, T2, T3 as in the third graph 1917 may mean that there is a horizontal or vertical phase difference of light, and that the camera module 191 is out of focus. In this case, the camera module controller (for example, the camera module controller 1715) may adjust the focus of the camera module by moving the lens of the camera module based on the horizontal and vertical phase difference.
Referring to
In operation 2003, the processor 110 may compare K1 and a pre-designated value (K0). For example, K0 may be close to 0. For example, when K1 has a range from 0 to 255, K0 may have a range from 1 to 10.
When K1 is less than K0 as a result of the comparing, the processor 110 proceeds to operation 2005, and otherwise, proceeds to operation 2007.
When the processor 110 proceeds to operation 2005, the processor 110 may determine that the camera module 191 is in focus. According to an embodiment of the present disclosure, thereafter, the processor 110 may display information indicating that the camera module 191 is in focus on the display 160 or may output the information through the speaker 182.
When the processor 110 proceeds to operation 2007, the processor 110 may determine that the camera module 191 is out of focus. Accordingly, according to an embodiment of the present disclosure, the processor 110 may move the lens of the camera module 191 based on K1. For example, the processor 110 may detect a moving distance of the lens corresponding to K1 using a table listing the relationship between the phase difference and the lens movement, and move the lens of the camera module 191 as long as the detected moving distance.
Referring to
In operation 2103, the processor 110 may compare K1 and a pre-designated value (K0). For example, K0 may be close to 0. For example, when K1 has a range from 0 to 255, K0 may have a range from 1 to 10.
When K1 is less than K0 as a result of the comparing, the processor 110 proceeds to operation 2105, and otherwise, proceeds to operation 2111.
When the processor 110 proceeds to operation 2111, the processor 110 may determine that the camera module 191 is out of focus. According to an embodiment of the present disclosure, thereafter, the processor 110 may move the lens of the camera module 191 based on K1. For example, the processor 110 may detect a moving distance of the lens corresponding to K1 using a table listing the relationship between the phase difference and the lens movement, and move the lens of the camera module 191 as long as the detected moving distance.
When the processor 110 proceeds to operation 2105, the processor 110 may calculate a second difference between the signal intensity of T1 and a signal intensity of T3. Herein, the second difference will be referred to as K2.
In operation 2107, the processor 110 may compare K2 and the pre-designated value (K0). When K2 is less than K0 as a result of the comparing, the processor 110 proceeds to operation 2109, and otherwise, the processor 110 proceeds to operation 2113.
When the processor 110 proceeds to operation 2113, the processor 110 may determine that the camera module 191 is out of focus. According to an embodiment of the present disclosure, thereafter, the processor 110 may move the lens of the camera module 191 based on K2. For example, the processor 110 may detect a moving distance of the lens corresponding to K2 using the table listing the relationship between the phase difference and the lens movement, and move the lens of the camera module 191 as long as the detected moving distance.
When the processor 110 proceeds to operation 2109, the processor 110 may determine that the camera module 191 is in focus. According to an embodiment of the present disclosure, thereafter, the processor 110 may display information indicating that the camera module 191 is in focus on the display 160 or may output the information through the speaker 182.
Referring to
In operation 2203, the processor 110 may generate focus data (for example, phase difference information) based on the color data and the white data.
In operation 2205, the processor 110 may generate YUV data based on the color data and the white data. In operation 2207, the processor 110 may output a preview image through the display 160 based on the YUV data.
In operation 2209, the processor 110 may determine whether the camera module 191 is in focus or not based on the phase difference information. As a result of the determining, when the camera module 191 is in focus, the processor 110 proceeds to operation 2211, and otherwise, proceeds to operation 2215.
When the processor 110 proceeds to operation 2215, the processor 110 may move the lens of the camera module 191 based on the phase difference information.
When the processor 110 proceeds to operation 2211, the processor 110 may determine whether there is a request for image capturing. For example, the image capturing refers to picking up an image of a subject or photographing the subject using a camera. As a result of the determining, when there is the request for image capturing, the processor 110 may proceed to operation 2213, and otherwise, proceed to operation 2207.
When the processor 110 proceeds to operation 2213, the processor 110 may generate an image through the camera module 191 and store the image in the memory 130.
The flowchart of
According to various embodiments of the present disclosure, an image sensor may include a plurality of pixels, and at least one pixel of the plurality of pixels may include: a micro lens; a filter which includes a first area for filtering a first color signal from a light signal transmitted through the micro lens, and a second area for filtering a second color signal from the transmitted signal; and a first photodiode for converting the first color signal into an electric signal, and a second photodiode for converting the second color signal into an electric signal.
According to various embodiments of the present disclosure, at least two sides of the first area may be connected with at least two sides of the second area.
According to various embodiments of the present disclosure, the first area may be nearly colorless.
According to various embodiments of the present disclosure, the second area may be at least one of a red (R) filter, a green (G) filter, or a blue (B) filter.
According to various embodiments of the present disclosure, the filter may consist of only the first area and the second area which includes one of red (R) filter, green (G) filter, or blue (B) filter.
According to various embodiments of the present disclosure, at least part of one of the first area and the second area (for example, the color filter area 609) may be enclosed by the other area of the first area and the second area (for example, the transparent filter area 607).
According to various embodiments of the present disclosure, the first area may include first and second transparent filter areas (for example, the transparent filter areas 747, 749), and the second area (for example, the color filter area 751) may be disposed between the first and the second transparent filter areas.
According to various embodiments of the present disclosure, one of the first area and the second area may be disposed on an upper end of the other area of the first area and the second area.
According to various embodiments of the present disclosure, the first area may include first and second transparent filter areas (for example, the transparent filter areas 827, 829), and the second area may include first and second color filter areas (for example, the color filter areas 831, 833), and the first transparent filter area and the first color filter area may be symmetrical to the second transparent filter area and the second color filter area with reference to a single axis (for example, a vertical (y) axis).
According to various embodiments of the present disclosure, the first photodiode and the second photodiode may have similar sizes.
An electronic device according to various embodiments may include: an image sensor including a plurality of pixels; and a processor functionally connected with the image sensor, and at least one pixel of the plurality of pixels may include: a micro lens; a filter including a first area for filtering a first color signal from a light signal transmitted through the micro lens, and a second area for filtering a second color signal from the transmitted signal; and a first photodiode for converting the first color signal into an electric signal, and a second photodiode for converting the second color signal into an electric signal, and the processor may generate brightness information and color information based on at least the first color signal and the second color signal which are converted into the electric signals, and output an image corresponding to the brightness information and the color information.
According to various embodiments of the present disclosure, one of the first area and the second area may be enclosed by the other area of the first area and the second area.
According to various embodiments of the present disclosure, the second area may include a first color filter area and a second color filter area, and the first area may be disposed between the first and second color filter areas.
According to various embodiments of the present disclosure, the first area may include a first transparent filter area and a second transparent filter area, and the second area may include a first color filter area and a second color filter area, and the first transparent filter area (for example, the transparent filter area 849) is located on a diagonal line from the second transparent filter area (for example, the transparent filter area 851), and the first color filter area (for example, the color filter area 847) is located on a diagonal line from the second color filter area (for example, the color filter area 853).
According to various embodiments of the present disclosure, the processor may control auto focusing using the brightness information.
According to various embodiments of the present disclosure, the processor may process the brightness information and the color information and convert the brightness information and the color information into YUV data.
According to various embodiments of the present disclosure, the processor may generate phase difference information based on at least the brightness information, and control a function of the electronic device using the phase difference information.
A signal processing method according to various embodiments includes: receiving a light signal through a single micro lens included in at least one pixel included in an image sensor; filtering a white signal and a color signal from the light signal through a filter including a first area and a second area; converting the white signal and the color signal into electric signals through a plurality of photodiodes; and outputting an image corresponding to brightness information and color information which are determined based on the electric signals through a display.
According to various embodiments of the present disclosure, the signal processing method may further include generating phase difference information based on at least the brightness information, and adjusting a lens location of a camera unit according to the phase difference information.
According to various embodiments of the present disclosure, the signal processing method may further include generating image 3D information based on at least the brightness information, and generating a 3D image based on at least the image 3D information.
The image sensor and the signal processing method thereof according to various embodiments of the present disclosure can reduce a loss in the sensitivity of light by receiving light of diverse wavelengths.
The image sensor and the signal processing method thereof according to various embodiments of the present disclosure can provide a clear image even when a background for photographing has low illumination, by receiving light of almost all wavelengths.
Further, the ‘module’ may be provided as a non-transitory computer-readable storage media storing one or more programs (or programming modules and applications). For instance, the software may be implemented by an instruction stored in a non-transitory computer-readable storage media in a form of the programming module. The one or more programs may include instructions for enabling an electronic device to execute methods according to an embodiment stated in the claims and/or specification of the present disclosure. If the instruction is executed by one or more processors (e.g., the processor 120), the one or more processors may perform a function corresponding to the instruction. The non-transitory computer-readable storage media may be, for example, the memory 230. At least a part of the programming module may be, for example, implemented (e.g., executed) by the processor 210. At least a part of the programming module may include, for example, a module, a program, a routine, sets of instructions, or a process, etc. for performing one or more functions.
The non-transitory computer-readable recording media includes a magnetic media such as a hard disk, a floppy disk, and a magnetic tape, an optical media such as a compact disc-read only memory (CD-ROM) and a DVD, a Magneto-Optical Media such as a floptical disk, and a hardware device specially configured to store and perform a program instruction (e.g., a programming module) such as a ROM, a RAM, a flash memory, etc., an EEPROM, a magnetic disc storage device or an optical storage device of other form, and a magnetic cassette. Or, the program instruction may be stored in a memory constructed by a combination of some or all of them. Also, each construction memory may be also included in plural. Also, the program instruction includes not only a mechanical language code such as a code made by a compiler but also a high-level language code executable by a computer using an interpreter, etc.
Further, the program instruction may be stored in an attachable storage device accessible to an electronic device through a communication network such as the Internet, an intranet, a local area network (LAN), a wireless LAN (WLAN) and a storage area network (SAN) or a communication network constructed by a combination of them. This storage device may connect to the electronic device through an external port. Also, a separate storage device on the communication network may connect to a portable electronic device as well. The aforementioned hardware device may be configured to operate as one or more software modules so as to perform operations according to various embodiments of the present disclosure, and vice versa.
The module or programming module according to various embodiments of the present disclosure may include at least one or more of the aforementioned constituent elements, or omit some of the aforementioned constituent elements, or further include additional other constituent elements. Operations carried out by the module, the programming module or other constituent elements according to various embodiments of the present disclosure may be executed in a sequential, parallel, repeated or heuristic method. Also, some operations may be executed in different order or may be omitted, or other operations may be added.
While the present disclosure has been shown and described with reference to various embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2015-0105135 | Jul 2015 | KR | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 7821553 | Ellis-Monaghan | Oct 2010 | B2 |
| 7990444 | Kudoh | Aug 2011 | B2 |
| 8547394 | Swic | Oct 2013 | B2 |
| 20060146067 | Schweng | Jul 2006 | A1 |
| 20070159542 | Luo | Jul 2007 | A1 |
| 20090090841 | Kusaka | Apr 2009 | A1 |
| 20100238330 | Hirota | Sep 2010 | A1 |
| 20120268566 | Kim | Oct 2012 | A1 |
| 20130070140 | Gove | Mar 2013 | A1 |
| 20130242149 | Terai et al. | Sep 2013 | A1 |
| 20140139716 | Sasaki et al. | May 2014 | A1 |
| 20140184863 | Tu | Jul 2014 | A1 |
| 20140232900 | Wernersson | Aug 2014 | A1 |
| 20140320735 | Ikedo | Oct 2014 | A1 |
| 20150054997 | Hynecek | Feb 2015 | A1 |
| 20170171470 | Sakioka | Jun 2017 | A1 |
| Number | Date | Country |
|---|---|---|
| 1331670 | Jul 2003 | EP |
| 2015087515 | Jun 2015 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20170026622 A1 | Jan 2017 | US |
