In the drawings, which illustrate what is currently considered to be the best mode for carrying out the invention:
There is a need for a defective pixel identifier that includes new ways to identify multiple pixel locations and reduces the number of nonvolatile programming elements required to identify all the bad pixels on an image sensor of acceptable quality. In addition, there is a need for a defective pixel identifier that includes more information than just the location for defective pixels and defective pixel clusters.
The present invention provides methods and apparatuses for identifying anomalous pixels in an image sensor. Anomaly registers may identify multiple pixel locations in the form of anomalous pixel rows, anomalous pixel columns, and anomalous pixel clusters. Furthermore, the anomaly registers may be configured to identify cluster sizes, cluster shapes, and types of anomalies.
Representative embodiments of the invention include an image sensor device including an array of pixels arranged on a semiconductor device wherein each pixel is configured for sensing light incident on the pixel and a plurality of anomaly registers comprising a plurality of nonvolatile elements. Each anomaly register is configured to identify an anomalous pixel cluster and includes a location indicator and a size indicator. The location indicator defines a row address and a column address of the anomalous pixel cluster and the size indicator defines a horizontal range and a vertical range of the anomalous pixel cluster. Some embodiments may include an anomaly type indicator in the anomaly register to indicate a type of anomaly for the anomalous pixel cluster and some embodiments may include a shape indicator in the anomaly register to define a shape of the anomalous pixel cluster.
Another representative embodiment of the invention includes an image sensor device including an array of pixels arranged on a semiconductor device wherein each pixel is configured for sensing light incident on the pixel and a plurality of anomaly registers comprising a plurality of nonvolatile elements. Each anomaly register in this embodiment includes a first address, a second address, a first direction flag associated with the first address, and a second direction flag associated with the second address. Each of the first direction flag and the second direction flag are configured to be in a first state to indicate a row address or a second state to indicate a column address. When the first direction flag and the second direction flag are in opposite states, a combination of the first address and the second address indicates a location of an anomalous pixel cluster determined by the row address and the column address. When the first direction flag and the second direction flag are both in the first state, the first address indicates a location of a first anomalous pixel row and the second address indicates a location of a second anomalous pixel row. Finally, when the first direction flag and the second direction flag are both in the second state, the first address indicates a location of a first anomalous pixel column and the second address indicates a location of a second anomalous pixel column.
Other representative embodiments of the invention comprise methods of manufacturing an image sensor. The methods include obtaining anomaly information on each pixel of an array of pixels during fabrication and testing of a semiconductor device bearing the array of pixels. The methods further include storing at least a portion of the anomaly information in an anomaly register comprising a plurality of nonvolatile elements on the semiconductor device, wherein the portion of the anomaly information identifies an anomalous pixel cluster by indicating a location, a horizontal range, and a vertical range. Some embodiments may include storing an anomaly type of the anomalous pixel cluster and some embodiments may include storing a shape of the anomalous pixel cluster.
Another representative embodiment of the invention comprises a method of operating an image sensor. The method includes obtaining a digital representation of an image captured with the image sensor and reading at least one anomaly register of a plurality of anomaly registers on the image sensor to retrieve anomaly information for an anomalous pixel cluster. The method further includes redefining pixel values in the anomalous pixel cluster with pixel data associated with at least one pixel substantially near the anomalous pixel cluster. In this method, each anomaly register includes a location, a horizontal range, a vertical range, and an anomaly type of the anomalous pixel cluster.
Another representative embodiment of the invention comprises an imaging system includes one of the previously defined image sensors, an image processor, a memory, and a communication element. The image processor reads a digital representation of the image from the image sensor, the memory stores the digital representation, and the communication element communicates the digital representation to an external device.
In this description, circuits and functions may be shown in block diagram form in order not to obscure the present invention in unnecessary detail. Conversely, specific circuit implementations shown and described are exemplary only and should not be construed as the only way to implement the present invention unless specified otherwise herein. Additionally, block definitions and partitioning of logic between various blocks is exemplary of a specific implementation. It will be readily apparent to one of ordinary skill in the art that the present invention may be practiced by numerous other partitioning solutions. For the most part, details concerning timing considerations and the like have been omitted where such details are not necessary to obtain a complete understanding of the present invention and are within the abilities of persons of ordinary skill in the relevant art.
In this description, some drawings may illustrate signals as a single signal for clarity of presentation and description. It will be understood by a person of ordinary skill in the art that the signal may represent a bus of signals, wherein the bus may have a variety of bit widths and the present invention may be implemented on any number of data signals including a single data signal.
The terms “assert” and “negate” are respectively used when referring to the rendering of a signal, status bit, or similar apparatus into its logically true or logically false state. If the logically true state is a logic level one, the logically false state will be a logic level zero. Conversely, if the logically true state is a logic level zero, the logically false state will be a logic level one.
Representative embodiments of the present invention use nonvolatile programming elements (may also be referred to as nonvolatile elements) to store information about pixels that may include anomalies. As used herein, a nonvolatile programming element is a device capable of being programmed to at least two different states and retaining that programmed state when no power is applied to the nonvolatile element. Different types of nonvolatile programming elements may be used to implement portions of the present invention. By way of example, and not limitation, fuses, anti-fuses, laser fuses, Flash memory cells, and EPROM cells may be used as nonvolatile elements. A number of nonvolatile programming elements may be arranged in a register fashion comprising a number of bits, such that each bit within the register is available when the register is addressed.
Additionally, the nonvolatile programming elements may be defined as a “0” or a “1” when in the un-programmed state. Furthermore, some nonvolatile elements may be multi-state elements capable of storing more than two different states. For ease of description, unless specified otherwise, the nonvolatile programming elements as described herein are assumed to produce a logic “1” as an asserted level when programmed and a logic “0” as a negated level when left un-programmed.
Some representative embodiments of the present invention may identify a pixel, or pixel cluster, as “good” or “bad,” whereas other representative embodiments of the present invention may include information for defining various anomalous states of pixels or pixel clusters. For example, a pixel may be partially functional or it may be useful to know how a pixel is defective. With information about an anomaly, more intelligent compensation mechanism may be used to modify a digital representation of the anomalous pixel or pixel cluster. For example, if a pixel is partially functional, compensating the pixel may include using the value for that pixel but also augmenting it with values from one or more neighbors.
By way of example, and not limitation, some anomalies that may be of interest are pixels that appear white, bright, dark, or black. The terms white and black, as used with reference to anomalies, generally refer to full intensity, and no intensity respectively, rather than reference to a given color. Therefore, a pixel configured for a specific color, for example green, may include an anomaly of white to refer to full intensity and black to refer to no intensity. Generally, white refers to a pixel that, regardless of the intensity of light that it is exposed to, is read at substantially near a maximum value. Similarly, black refers to a pixel that, regardless of the intensity of light that it is exposed to, is read at substantially near a minimum value.
Some pixels may be partially functional. For example, a pixel may include a relatively low gain with respect to light intensity when compared to normal pixels. Thus, the pixel may respond to light but may not be able to achieve a maximum output even when exposed to high intensity light. These pixels may be referred to as dark pixels. In other words, the slope of a gain line defining change in the signal from the pixel relative to change in the light intensity may be too low. Similarly, a pixel may include a relatively high gain with respect to light intensity when compared to normal pixels. Thus, the pixel may respond to light such that it achieves a maximum output even when exposed to relatively low intensity light. These pixels may be referred to as bright pixels. In other words, the slope of a gain line defining change in the signal from the pixel relative to change in the light intensity may be too high.
Other pixel anomalies are contemplated within the scope of the present invention. By way of example, and not limitation, pixels may have an acceptable gain, but may always exhibit an offset such that for any given light intensity a pixel may appear brighter or dimmer than it should.
“Anomaly” as referred to herein, may refer to a pixel as simply bad, or may refer to various degrees and types of anomalies.
The image processor may be, for example, application specific logic, a micro-controller 190, a microprocessor, a digital signal processor, or combinations thereof. The image processor 160 may perform a variety of control and signal processing functions. By way of example, and not limitation, some of these functions may be: sending and receiving control information between the image processor 160 and image sensor 100, receiving digital representations of images from the image sensor 100, storing digital representations in the memory 170, performing signal processing operations on the digital representations, and controlling other operations within the imaging system 90. It should be understood that the memory 170 might comprise a wide variety of devices including, for example, Static RAM (SRAM), dynamic RAM (DRAM), and Flash memory devices.
The communication element 180 may be used to transfer the digital representations, or other information, between the imaging system 90 and external devices (not shown). Any suitable communication element and communication protocol may be used, such as, for example, IEEE 1394, universal serial bus (USB), and wireless communications such as cellular phones, and 802.11 protocols.
The image sensor 100 includes a sensor array 110, a row decoder 112, a column decoder 114, and a controller 190. The sensor array 110 (may also be referred to as an array of pixels) includes photo-sensitive devices such as photodiodes or photo-transistors fabricated on, for example, a complementary metal oxide semiconductor (CMOS) device. Each photo-sensitive device is sensitive to light in such a way that it can create an electrical current that is proportional to the intensity of light striking the photo-sensitive device. The overall image captured by a sensor array 110 includes many pixels arranged in an array such that each pixel detects the light intensity at the location of that pixel.
As stated earlier, a single pixel may include a single photo-sensitive device configured for detecting a broad frequency range, which may be used for gray scale images. In addition, a pixel may be defined as a single photo-sensitive device configured for detecting a specific color (i.e. frequency). Finally a pixel may be a group of photo-sensitive devices arranged near each other wherein different devices within the group are configured for detecting different colors. Thus, a full color image may be detected with an appropriate combination of color sensing pixels. The term pixel as used herein may refer to a single photo-sensitive device for detecting a broad range of frequencies, a single photo-sensitive device for detecting a narrow frequency band, or a combination of photo-sensitive devices configured to capture a color image at the location of the pixel. The pixels of the sensor array 110 are arranged in individually addressable rows and columns such that the row decoder 112 can address each row of the sensor array 110 and the column decoder 114 can address each column of the sensor array 110.
While not illustrated with connections, it will be understood by those of ordinary skill in the art that the controller 190 may control functions of many or all of the other blocks within the image sensor 100. For example, the controller 190 may control the exposure of the sensor array 110 (i.e., capturing an image) and the sequencing of the row decoder 112 and column decoder 114 to read out the analog values at each pixel location within the sensor array 110.
As the pixels are each individually addressed, the resulting analog signal from each pixel may be sequentially directed from the column decoder 114 to a sample and hold unit 120 to sample the value of the analog signal and hold that value at a steady state during conversion to a digital signal. The output of the sample and hold unit 120 may be amplified by amplifier 125 for input to an analog to digital converter 130. The analog to digital converter 130 converts the analog signal for each pixel to a digital signal representing the intensity of light at that pixel.
The digital signal for each pixel may be directed through a pixel processor 140. At this point, the pixel processor 140 may perform compensation operations on any anomalous pixels. The anomaly register set 200 includes a plurality of anomaly registers 250 (details shown in
The pixel processor 140 may perform a number of functions on the pixel being processed. By way of example, and not limitation, if a pixel is identified by the anomaly register set 200 as including an anomaly, the value for the pixel may be replaced with a new value. For example, the value may be replaced by the value of a neighboring pixel or an average value from a number of neighboring pixels.
After processing, the current pixel may be transferred to the input/output (I/O) port 150 for transmission to the image processor 160. The I/O port 150 may include storage to save up values from a number of pixels such that pixel values may be transferred out of the image sensor 100 in a parallel or serial fashion.
In some embodiments, the image processor 160 may use anomaly information from the anomaly register set 200 to perform additional compensation for anomalous pixels. Thus, the image sensor 100 may include a path for the anomaly register set 200 to be read directly (not shown) or read through the register decoder 210.
In addition, some representative embodiments of the present invention, as illustrated in
As a result, the image processor 160 may perform a variety of functions such as, for example, image filtering and compression. In addition, the image processor 160 may use anomaly information from the anomaly register set 200 to perform intelligent compensation for anomalous pixels, anomalous pixel rows, anomalous pixel columns, and anomalous pixel clusters.
The location indicator 260 and the size indicator 270 may be defined in a variety of ways for pixel clusters 330. For example, the location indictor 260 may point to the upper right corner of the pixel cluster 330 while the horizontal range 272 indicates pixels to the right of the location indicator 260 and the vertical range 274 indicates pixels below the location indicator 260. As yet another example, the location indictor 260 may point to the center of the pixel cluster 330 while the horizontal range 272 indicates pixels to the right and left of the location indicator 260 and the vertical range 274 indicates pixels above and below the location indicator 260.
As a representative way of describing an anomalous pixel column 310, when the horizontal range 272 is programmed to a zero value, the entire column is defined as anomalous. Similarly, when the vertical range 274 is programmed to a zero value, the entire row is defined as anomalous. Thus, when describing an anomalous row, the vertical range 274 is set to zero, the row address 261 points to the anomalous row, and the column address 262 need not be programmed. Similarly when describing an anomalous column, the horizontal range 272 is set to zero, the column address 262 points to the anomalous column, and the row address 261 need not be programmed.
Of course, those of ordinary skill in the art will recognize that a value other than zero may be used for defining an anomalous row or column. In addition, an anomalous row may be defined in the horizontal range 272 and an anomalous column may be defined in the vertical range 274. Furthermore, combinations other than a simple binary coding may be used for defining the range values.
Embodiments of the present invention may include various combinations of the shape indicator 280 and anomaly type indicator 290. For example, some embodiments may only describe rectangular clusters and as a result have no need for a shape indicator 280. Or, a shape indicator 280 may be desirable, but anomaly type indicators 290 may not be needed. Furthermore, some embodiments may not need a size indicator 270. For example, it may be desirable to include an anomaly type indicator 290, but only describe individual pixel locations with the location indicator 260. If only individual pixels are described, the size indicator is not needed.
In addition, other configurations for the location indicator may be used. Recall that with the location indicator 260 of
By way of example,
In addition, the location indicator 260′ may be configured to indicate two different rows by using a “00” designation and a corresponding row address in the first address 265 portion and another row address in the second address 268 portion. Or, two different columns may be identified by using a “11” designation and a corresponding column address in the first address 265 portion and another column address in the second address 268 portion. In this way, the location indicator 260′ may be used to identify more anomalous pixels because two rows or two columns may be identified in each anomaly register 250.
With the location indicator 260′ of
In operation, and referring to
Testing may be performed as part of the manufacturing process prior to shipping or may be performed in situ while the image sensor 100 is in an imaging system 90. During manufacturing, the sensor array 110 may be exposed to varying intensities of light. At each intensity, the pixels may be read to determine if the intensity read is within an acceptable range. As a result of this exposure and reading, pixel quality and pixel anomalies can be determined. For example, black pixels may be detected by exposing the sensor array 110 to a full intensity light. Pixels that do not respond with a value corresponding to the full intensity may be defined as black pixels. In addition, pixels that respond, but not with a full intensity, may be defined as dark pixels. Similarly, if the senor array is exposed to darkness, pixels that do not respond with a value corresponding to the darkness may be defined as white pixels and pixels that respond, but not corresponding completely to the darkness may be defined as bright pixels. Varying degrees of light intensity may be used to define additional levels of anomalous pixel behavior. Consequently, a map of the anomalous pixels and their quality may be formed.
The resulting map may be used to group the pixels into anomalous pixel clusters 330, anomalous pixel rows 320, and anomalous pixel columns 310 as explained earlier. Then each of the anomaly registers 250 may be programmed appropriately.
In operation during use, after an image is exposed, the sensor array 110 is read out as explained earlier. As the pixels pass through the pixel processor 140, image processor 160, or combinations thereof, the register decoder 210, controller 190, or combinations thereof may read the anomalous register set 200 and decode them to determine if the current pixel in the pipeline includes an anomaly. If so, appropriate processing may be performed to compensate for the pixel anomaly.
After an image is stored, the image processor 160 may perform additional operations on the digital representation of the image to improve image quality or to perform other processing such as color correction or image compression. In this case, the image processor 160 may want information about anomalous pixels. Consequently, the anomaly registers 250 may be read directly, or via the register decoder 210, to provide anomalous pixel information to an internal or external image processor 160.
Although this invention has been described with reference to particular embodiments, the invention is not limited to these described embodiments. Rather, the invention is limited only by the appended claims, which include within their scope all equivalent devices or methods that operate according to the principles of the invention as described.
The present application claims the benefit of U.S. Provisional Application Ser. No. 60/814,628 entitled METHODS, DEVICES, AND SYSTEMS FOR SELECTABLE REPAIR OF IMAGING filed on Jun. 15, 2006, the disclosure of which is incorporated by reference herein in its entirety.
Number | Date | Country | |
---|---|---|---|
60814628 | Jun 2006 | US |