1. Field of the Invention
The present invention relates to image processing, and more particularly, to a de-noising method and related apparatus utilizing a specific noise threshold value for an image sensor.
2. Description of the Prior Art
In an imaging system, three components of color information must be captured simultaneously to precisely present an image. To create an analogous digital imaging system that simultaneously captures all three components of the color information requires three individual imaging detectors. This would be prohibitive due to the high cost and would also cause the packaging to be very complex. To keep the size and cost of a digital video imaging system to a minimum, an image sensor array of the system must also be kept to a small size. Therefore, the number of color samples must be kept low. An alternative approach is to have each detector of the imaging system gather data for a single color to create a sparse color image. To achieve this, the imaging systems typically use a mosaic filter generally called a color filter array (CFA), and acquire a scene image by sampling one of the three different color components to obtain an array that stores only one color component per pixel. The imaging system gets raw sensory data having less color samples per pixel because it ignores the other two color components for each pixel. Since each filter of the color filter array covers a single pixel and only allows a color in a specific spectral band to pass, before the scene image is further processed or displayed, the missing colors of each image pixel must be reconstructed so that each image pixel contains all three needed color components.
No images are absolutely perfect no matter how good the camera, since images are interfered with by the presence of noise. The principal sources of noise in digital images arise during image acquisition, digitization, and/or transmission. The performance of imaging sensors is affected by a variety of factors, such as environmental conditions during image acquisition, and by the quality of the sensing elements themselves. For instance, when acquiring images with a Charge-Coupled Device (CCD) sensor or a CMOS image sensor, luminosity and sensor temperature are major factors affecting the amount of noise in the generated images.
Filtering a digital image is one necessary stage in image processing and is used for reducing noise when protecting image details. For example, any noise in images will result in serious errors due to many applications being based on operands drawn out from applications for calculating images. Therefore, methods for reducing noise are desired to not only improve the visual quality, but also to improve the performance of subsequent processing tasks such as coding, analysis cutting, identification, or interpretation.
It is therefore one of the objectives of the present invention to provide a de-noising method and related apparatus utilizing a specific noise threshold value for an image sensor, to remove as much noise as possible while blurring the edges and other image details as little as possible.
According to one embodiment of the present invention, a de-noising method for an image sensor is disclosed, wherein the image sensor includes a specific color filter array and a pixel array. The pixel array includes a plurality of pixels. Each pixel corresponds to one color filter, and thus corresponds to one of a plurality of color components. The de-noising method comprises: comparing a pixel value of a target pixel with pixel values of a plurality of nearby pixels, wherein each of the target pixel and the nearby pixels corresponds to a specific color component; for each nearby pixel of the nearby pixels, utilizing a checking circuit to check if a difference between the pixel value of the target pixel and a pixel value of a nearby pixel of the nearby pixels is smaller than a specific noise threshold value, and when a difference between the pixel value of the target pixel and a pixel value of a nearby pixel of the nearby pixels is smaller than a specific noise threshold value, setting the nearby pixel as a similar nearby pixel; and updating the pixel value of the target pixel according to the pixel value of the target pixel and a pixel value of each similar nearby pixel.
According to another embodiment of the present invention, a de-noising apparatus for an image sensor is disclosed, wherein the image sensor includes a specific color filter array and a pixel array. The pixel array includes a plurality of pixels. Each pixel corresponds to one color filter, and thus corresponds to one of a plurality of color components. The de-noising apparatus includes a comparing circuit, a checking circuit and a pixel value updating circuit. The comparing circuit is for comparing a pixel value of a target pixel with pixel values of a plurality of nearby pixels, wherein each of the target pixel and the nearby pixels corresponds to a specific color component. For each nearby pixel of the nearby pixels, the checking circuit checks if a difference between the pixel value of the target pixel and a pixel value of a nearby pixel of the nearby pixels is smaller than a specific noise threshold value, and when a difference between the pixel value of the target pixel and a pixel value of a nearby pixel of the nearby pixels is smaller than a specific noise threshold value, the checking circuit sets the nearby pixel as a similar nearby pixel. The pixel value updating circuit is coupled to the checking circuit, and implemented for updating the pixel value of the target pixel according to the pixel value of the target pixel and a pixel value of each similar nearby pixel.
The exemplary embodiments of the present invention provide a de-noising method and related apparatus utilizing a specific noise threshold value for an image sensor while preserving edges and other image details.
These and other objectives of the present invention will no doubt become obvious to those of ordinary skill in the art after reading the following detailed description of the preferred embodiment that is illustrated in the various figures and drawings.
Certain terms are used throughout the description and following claims to refer to particular components. As one skilled in the art will appreciate, manufacturers may refer to a component by different names. This document does not intend to distinguish between components that differ in name but not function. In the following description and in the claims, the terms “include” and “comprise” are used in an open-ended fashion, and thus should be interpreted to mean “include, but not limited to . . . ”.
Please refer to
The de-noising apparatus 100 includes a comparing circuit 10, a checking circuit 20, a pixel value updating circuit 30 and a threshold setting circuit 40. The comparing circuit 10 is used for comparing a pixel value of a target pixel Pt with pixel values of a plurality of nearby pixels in the pixel array, wherein each of the target pixel Pt and the nearby pixels corresponds to a specific color component. That is, the target pixel Pt and the selected nearby pixels which are compared with the target pixel Pt have the same color component. The difference Diff between a pixel value of the target pixel Pt and a pixel value of each of the selected nearby pixel is calculated by the comparing circuit 10 and then passed to the following checking circuit 20 coupled to the comparing circuit 10.
For each nearby pixel Pn of the selected nearby pixels, the checking circuit 20 is implemented for checking if the difference Diff between the pixel value of the target pixel Pt and a pixel value of the nearby pixel Pn is smaller than the specific noise threshold value Vs. In this exemplary embodiment, when the difference Diff between the pixel value of the target pixel Pt and the pixel value of the nearby pixel Pn is smaller than the specific noise threshold value Vs, the checking circuit 20 sets the nearby pixel Pn as a similar nearby pixel.
The pixel value updating circuit 30 is coupled to the checking circuit 20, and implemented for updating the pixel value of the target pixel Pt according to the pixel value of the target pixel Pt and a pixel value of each similar nearby pixel identified by the checking circuit 20. The threshold setting circuit 40, coupled to the checking circuit 20, is for setting the specific noise threshold value Vs according to the target pixel Pt. The details directed to setting the specific noise threshold value Vs and updating a pixel value of the target pixel Pt will be illustrated later.
Please refer to
Taking the pixel G7 corresponding to the green color as the target pixel Pt for example, the comparing circuit 10 compares a pixel value of the target pixel G7 with pixel values of a plurality of nearby pixels G1, G2, G3, G4, G5, G6, G8, G9, G10, G11, G12, G13 in the image sensor 200. It should be noted that the above exemplary selection of the nearby pixels is for illustrative purposes only. In other words, the rule of selecting the nearby pixels of the target pixel can be adjusted, depending upon design requirements.
The checking circuit 20 checks if a first difference Diff_1 between the pixel value of the target pixel G7 and a pixel value of a nearby pixel G1 of the nearby pixels is smaller than the first color value C1, and if it is determined that the first difference Diff_1 is smaller than the first color value C1, the checking circuit 20 sets the nearby pixel G1 as a similar nearby pixel. The checking circuit 20 keeps checking if a second difference Diff_2 between the pixel value of the target pixel G7 and a pixel value of another nearby pixel G2 of the nearby pixels is smaller than the first color value C1, and if it is determined that the second difference Diff_2 is smaller than the first color value C1, the checking circuit 20 sets the nearby pixel G2 as a similar nearby pixel, and so on. After finishing the above checking operation performed by the checking circuit 20, if the nearby pixels G3, G5, G8, G10 are categorized as similar nearby pixels, the pixel value updating circuit 30 averages the pixel value of the target pixel G7 with pixel values of the similar nearby pixels G3, G5, G8, G10 to update the pixel value of the target pixel G7. More specifically, the updated pixel value of the target pixel G7 can be expressed as follows:
In the above equation (1), Pixel_G7, Pixel_G3, Pixel_G5, Pixel_G8, and Pixel_G10 represent pixel values of the target pixel G7 and the nearby pixels G3, G5, G8, G10, respectively.
Please refer to
In the above equation (2), Pixel_G7, Pixel_G3, Pixel_G5, Pixel_G8, and Pixel_G10 represent pixel values of the target pixel G7 and the nearby pixels G3, G5, G8, G10, respectively. However, this embodiment merely serves as an example for illustrating the present invention, and should not be taken as a limitation of the present invention. It should be appreciated by those skilled in the art that the present invention can adopt other averaging methods to derive an updated pixel value set to the target pixel.
Please refer to
It is worth noticing that, as light intensity decreases, the applied gain of the image sensor 200 will be increased which also enhances noise. Therefore, according to yet another embodiment of the present invention, the threshold setting circuit 40 sets the specific noise threshold value Vs according to an applied gain value corresponding to a light intensity. Taking the image sensor 200 for example, when the light intensity corresponds to a first luminance value, the specific noise threshold value Vs is set by a first light value V1, and when the light intensity corresponds to a second luminance value different from the first luminance value, the specific noise threshold value Vs is set by a second light value V2 different from the first light value V1. In one implementation, the first luminance value is greater than the second luminance value, and the first light value V1 is set smaller than the second light value V2. After setting the specific noise threshold value Vs by different luminance values, other operations of the de-noising apparatus 100 are similar to the afore-mentioned embodiments, and thus further description is omitted here for brevity.
Please note that any alternative designs combining features of the above-mentioned embodiments should also fall within the scope of the present invention. For example, the specific noise threshold value Vs can be first set according to the color component of the target pixel, and then set according to an applied gain value corresponding to a light intensity. Furthermore, please note that the de-noising operation in the above exemplary embodiments is applied to a target pixel with a green color (e.g., pixel G7); however, the de-noising operation can also be applied to a target pixel with a red color or blue color.
Please refer to
Step 301: Compare a pixel value of a target pixel with pixel values of a plurality of nearby pixels, wherein each of the target pixel and the nearby pixels corresponds to a specific color component;
Step 303: For each of the nearby pixels, utilize a checking circuit to check if a difference between the pixel value of the target pixel and a pixel value of the nearby pixel is smaller than a specific noise threshold value;
Step 305: When the difference between the pixel value of the target pixel and the pixel value of the nearby pixel is smaller than the specific noise threshold value, set the nearby pixel as a similar nearby pixel; and
Step 307: Update the pixel value of the target pixel according to the pixel value of the target pixel and a pixel value of each similar nearby pixel.
Please note that, as those skilled in this art can easily understand the operations of the steps 301-307 of the de-noising method after reading the disclosure of the above-mentioned embodiments directed to the de-noising apparatus, further description is omitted here for brevity.
In summary, the de-noising apparatus 100 can significantly remove noise of the pixels in an image sensor by setting a suitable noise threshold value. The present invention is designed for the image sensor such that the noise is removed at the earliest stage to avoid noise accumulation in the image processing pipeline. The specific noise threshold value can be adaptively adjusted based on noise level and noise distribution, thereby optimizing the de-noising performance.
Those skilled in the art will readily observe that numerous modifications and alterations of the device and method may be made while retaining the teachings of the invention.