Patent Grant
8606008
1. Field of the Invention
The present invention relates to image processing devices, image processing methods, image processing programs, and integrated circuits, and in particular relates to image processing devices, image processing methods, image processing programs, and integrated circuits that perform grayscale conversion without enhancing noise in the input signal.
2. Description of the Related Art
In general, image-capturing devices and display devices perform grayscale conversion for the purpose of correcting the brightness or the contrast of an input signal. Pixel unit processing and processing in which the surrounding region is referenced are known as examples of grayscale conversion.
Pixel unit processing is conversion processing that is performed based on only the pixel value of a target pixel, without referencing pixels other than the target pixel. A specific example is gamma correction, which is adopted for captured images in order to cancel out the photoelectric conversion characteristics of CRT display devices.
On the other hand, processing in which the surrounding region is referenced is conversion processing that is performed in reference to not only the pixel value of a target pixel but also to the pixel values of the region surrounding the target pixel or the pixel values of the entire image. A specific example is histogram equalization, in which the distribution of the frequency with which pixel values appear in an input signal is found, and grayscale conversion is performed on the input signal by assigning a wide range of grayscale to frequently appearing grayscale levels in the input image (with histogram equalization, if there is a narrow range of grayscale values that appear frequently in the input signal (for example, a case in which there are 5 to 20 grayscale values in the case of 8-bit data grayscale values), then grayscale conversion is performed to obtain grayscale values over a wide range (for example, 10 to 120 grayscale values) in the output signal). Another specific example is visual processing, in which conversion processing is performed based on the pixel value of the target pixel and the mean value of the pixel values of the surrounding region (mean pixel value).
By applying these grayscale conversions to an input signal, it is possible to obtain a converted signal in which the perception (feeling) of the brightness or the contrast is improved. However, as in the case of gamma correction, when a pixel with a small pixel value is processed with a high gain, the very tiny noise component in the input signal is amplified and the S/N ratio is significantly getting worse. One technology for remedying this issue that has been disclosed is the technology of performing noise reduction processing on the converted signals for pixels with a small pixel value (for example, JP 2001-309177A). With this technology, noise reduction is performed only on pixels in which the slope of the gamma curve, which expresses the input/output characteristics of the grayscale conversion, is greater than a predetermined threshold. As in gamma correction, in cases where the input/output characteristics of the grayscale conversion are determined in advance, those input/output characteristics have monotonically increasing properties, and the slope of the input/output characteristics curve expressing those input/output characteristics has the property of monotonically decreasing, the gain increases the lower the level of the input signal (the level of the input signal with a small grayscale value). Thus pixels in which the S/N ratio becomes poor, that is, pixels with a small pixel value that are processed with a high gain, are specified by detecting input signals that are below the threshold value. By performing noise reduction on the pixels that have been specified in this way, it is possible to improve the deterioration of the S/N ratio in dark areas.
However, with processing in which the surrounding region is referenced, such as histogram equalization and visual processing, the input/output characteristics of the grayscale conversion are changed for each image, or for each pixel, according to the frequency distribution of the pixel values or the mean pixel values around the target pixel of the input signal. When a conventional approach such as gamma correction is adopted in a processing method in which the surrounding region is referenced, it is not possible to pre-calculate the pixel values in which the slope of the input/output characteristics of the grayscale conversion is smaller than a predetermined threshold value, and thus if this is adopted for moving pictures, it is necessary to search the input/output characteristics of the grayscale conversion that are obtained each frame (or each field) or each pixel, and calculate a pixel level (pixel value) of the input signal that corresponds to the threshold value of the slope of the input/output characteristics curve for grayscale conversion that has been obtained. This processing, however, requires a large amount of computations.
In general grayscale conversion, there is no guarantee that the slope of the input/output characteristics curve for the input/output characteristics will decrease monotonically like with gamma correction, and thus with regard to the standard for determining a pixel value in which the slope is equal to or less than the threshold value, there is a possibility that it may not be possible to uniquely determine the range of the pixel levels (pixel values) of the input signal for which to adopt noise reduction.
Threshold processing simply is processing for switching whether or not to adopt noise reduction for a given pixel, and thus it is not possible to carry out noise reduction at a strength that is suited for each pixel.
It is an object of the invention to provide an image processing device, an image processing method, an image processing program, and an integrated circuit with which it is possible to execute noise reduction at a different strength for each target pixel, in accordance with the degree of the deterioration of the S/N ratio due to the amplification of noise components, not only in the case of grayscale conversion where only the target pixel is referenced, but also in the case of grayscale conversion in which the region around the target pixel is referenced also.
A first aspect of the invention is an image processing device that includes a grayscale conversion portion that performs grayscale conversion on an input signal that is made from pixel data that form an image to obtain a converted signal, a noise reduction degree determining portion that determines a noise reduction degree for the converted signal based on the input signal and the converted signal, and a noise reducing portion that executes noise reduction processing on the converted signal based on the noise reduction degree.
With this image processing device, from the input signal and the converted signal of the pixels it is possible to calculate a noise reduction degree that expresses a strength of the noise reduction processing to be adopted for each converted signal, and then adjust the strength of the noise reduction processing based on the value of the noise reduction degree that has been calculated. Thus, it is possible to perform noise reduction processing that corresponds to the degree to which the noise component is amplified in each pixel due to grayscale conversion.
A second aspect of the invention is the first aspect of the invention, in which the grayscale conversion portion performs grayscale conversion based on histogram information for the grayscale value of the pixel data of the input signal.
With this image processing device, it becomes possible to perform noise reduction processing that corresponds to the degree to which the noise component is amplified in each pixel due to grayscale conversion, even if histogram equalization is used as the method of grayscale conversion, for instance.
A third aspect of the invention is the first aspect of the invention, in which the grayscale conversion portion performs grayscale conversion based on target pixel data of the input signal and a signal that is obtained by performing predetermined processing on surrounding pixel data around the target pixel data.
With this image processing device, it is possible to effect a conversion while keeping noise from being enhanced, even in cases where the visual processing is executed on an input signal. Here, spatial-visual processing refers to processing in which grayscale correction is performed on a target pixel (region) using grayscale characteristics that change according to the brightness of the area around the target pixel (region). For instance, if the grayscale value of the target pixel in the input signal is small (such as an 8-bit grayscale value of 50) and the area surrounding the target pixel is dark (such as an 8-bit grayscale value of 20), then grayscale correction is performed in order to make the grayscale value of the target pixel a large grayscale value (such as an 8-bit grayscale value of 150). Conversely, if the grayscale value of the target pixel in the input signal is small (such as an 8-bit grayscale value of 50) and the area surrounding the target pixel is bright (such as an 8-bit grayscale value of 100), then grayscale correction is performed in order to make the grayscale value of the target pixel a small grayscale value (such as an 8-bit grayscale value of 30). This processing is one example of spatial-visual processing.
A fourth aspect of the invention is the first aspect of the invention, in which the noise reduction degree determining portion determines a noise reduction degree based on a ratio between the input signal and the converted signal.
Thus, it is possible to adjust the strength of the noise reduction processing based on the gain that has been adopted for the input signal.
A fifth aspect of the invention is the first aspect of the invention, in which the noise reduction degree determining portion determines a noise reduction degree based on a ratio between the input signal and the converted signal, and on the input signal.
Thus, it is possible to adjust the strength of the noise reduction processing based on the input signal and the gain that has been adopted for the input signal.
A sixth aspect of the invention is the fifth aspect of the invention, in which the noise reduction degree determining portion sets the noise reduction degree as a first noise reduction degree if a grayscale conversion gain, which is the ratio of the converted signal to the input signal (=(the converted signal)/(the input signal)), is larger than a first gain threshold, and the signal level of the input signal is smaller than a first signal level threshold. The noise reduction degree determining portion also sets the noise reduction degree as a second noise reduction degree, which is a smaller value than the first noise reduction degree, if the grayscale conversion gain is smaller than a second gain threshold, which is a smaller value than the first gain threshold, and the signal level of the input signal is smaller than the first signal level threshold. The noise reduction degree determining portion also sets the noise reduction degree as a third noise reduction degree, which is a smaller value than the first noise reduction degree, if the signal level of the input signal is larger than a second signal level threshold, which is a larger value than the first signal level threshold.
With this image processing device, strong noise reduction processing is executed for pixel data whose input signal level (grayscale value of the pixel data of the input signal) is small and whose grayscale conversion gain is large, and weak noise reduction processing is executed for pixel data whose input signal level is small and whose grayscale conversion gain is small. Further, weak noise reduction is performed for pixel data whose input signal level is large.
Thus, even in cases where pixel data making up a dark region of an image that is formed by the input signal is amplified by a large gain to create an output signal, and that output signal is displayed on a display device, it is possible to improve the brightness or contrast of the region corresponding to those pixel data while suppressing the noise component, on the display screen.
A seventh aspect of the invention is the fifth aspect of the invention, in which the noise reduction degree determining portion sets the gain-based noise reduction degree to a larger value the larger the grayscale conversion gain, which is the ratio of the converted signal to the input signal (=(the converted signal)/(the input signal)), and sets the signal level-based noise reduction degree to a smaller value the larger the signal level of the input signal. The noise reduction degree determining portion also determines a noise reduction degree based on the gain-based noise reduction degree and the signal level-based noise reduction degree.
Thus, it is possible to perform noise reduction processing that corresponds to the grayscale conversion gain and the signal level of the input signal.
It should be noted that here, “setting the gain-based noise reduction degree to a larger value the larger the grayscale conversion gain” refers to a relationship in which, for example, the gain-based noise reduction degree increases monotonically with respect to the change in the grayscale conversion gain. This includes not only a strict monotonic increase but also includes a substantially monotonic increase (it can also include ranges which in part are not monotonically increasing). For example, if the grayscale conversion gain is smaller than a predetermined value (this shall be called a “first gain value”), then the gain-based noise reduction degree is fixed at a predetermined value (this shall be called a “first gain-based noise reduction degree”), and if the grayscale conversion gain is equal to or greater than the first gain value but is equal to or less than a second gain value (which is greater than the first gain value), then the gain-based noise reduction degree is set to a value that monotonically increases with respect to the grayscale conversion gain. If the grayscale conversion gain is larger than the second gain value, then it is set to a value that is larger than the first gain-based noise reduction degree. Such a case also is included.
As the method for determining the noise reduction degree with the noise reduction degree determining portion, it is also possible to use a method in which the mean value (including the arithmetic mean and the geometrical mean) of the gain-based noise reduction degree and the signal level-based noise reduction degree is found and that mean value is established as the noise reduction degree, and a method in which a weighted mean of the gain-based noise reduction degree and the signal level-based noise reduction degree is found and that weighted mean is established as the noise reduction degree. It should be noted that here, the weighted mean is obtained by first finding the smaller of the gain-based noise reduction degree and the signal level-based noise reduction degree (this shall be called “value A” and the larger value shall be called “value B”) and assigning a large weight to the value A and a small weight to the value B, and then finding the average of those to obtain the weighted mean. For example, it is possible to find the weighted mean through (weighted mean)=((value A)×3+(value B))/4.
An eighth aspect of the invention is the seventh aspect of the invention, in which the noise reduction degree determining portion sets the smaller of the gain-based noise reduction degree and the signal level-based noise reduction degree as the noise reduction degree.
Thus, even in cases where pixel data making up a dark region of an image that is formed by the input signal is amplified by a large gain to create an output signal, and that output signal is displayed on a display device, it is possible to improve the brightness or contrast of the region corresponding to those pixel data while suppressing the noise component on the display screen.
A ninth aspect of the invention is the fifth aspect of the invention, in which the noise reduction degree determining portion includes a first noise reduction degree calculation portion, a second noise reduction degree calculation portion, and a noise reduction degree output portion. The first noise reduction degree calculation portion outputs a signal level-based noise reduction degree whose value is smaller than when the signal level of the input signal is equal to or less than a predetermined signal level threshold, if the signal level of the input signal is larger than the predetermined signal level threshold. The second noise reduction degree calculation portion outputs a gain-based noise reduction degree whose value is greater than when the grayscale conversion gain is equal to or less than a predetermined gain threshold, if the grayscale conversion gain, which is the ratio of the converted signal to the input signal (=(the converted signal)/(the input signal)), is larger than the predetermined gain threshold. The noise reduction degree output portion sets a value that has been calculated based on the signal level-based noise reduction degree and the gain-based noise reduction degree as the noise reduction degree.
Thus, it is possible to perform noise reduction processing that corresponds to the grayscale conversion gain and the signal level of the input signal.
A tenth aspect of the invention is the ninth aspect of the invention, in which the noise reduction degree output portion sets the smaller of the signal level-based noise reduction degree and the gain-based noise reduction degree as the noise reduction degree.
With this configuration, it is possible to execute strong noise reduction processing for pixel data whose input signal level (grayscale value of the pixel data of the input signal) is small and whose grayscale conversion gain is large, and execute weak noise reduction processing for pixel data whose input signal level is small and whose grayscale conversion gain is small. Further, weak noise reduction can be performed for pixel data whose input signal level is large.
Thus, even in cases where pixel data making up a dark region of an image that is formed by the input signal is amplified by a large gain to create an output signal, and that output signal is displayed on a display device, it is possible to improve the brightness or contrast of the region corresponding to those pixel data while suppressing the noise component on the display screen.
An eleventh aspect of the invention is an image processing device that is provided with a gain calculation portion that calculates a gain for conversion of an input signal that is made from pixel data that form an image, a multiplication portion that multiplies the gain with the input signal to obtain a converted signal, a noise reduction degree determining portion that determines a noise reduction degree for the converted signal based on the gain, and a noise reducing portion that executes noise reduction processing on the converted signal based on the noise reduction degree.
With this configuration, it is possible to calculate a noise reduction degree that expresses a strength of the noise reduction processing to be adopted for each converted signal from the input signal and the gain of the pixels, and then adjust the strength of the noise reduction processing based on the value of the noise reduction degree that has been calculated. Thus, it becomes possible to perform noise reduction processing that corresponds to the degree to which the noise component is amplified in each pixel due to grayscale conversion.
A twelfth aspect of the invention is the eleventh aspect of the invention, in which the noise reduction degree determining portion determines a noise reduction degree for the converted signal based on the input signal as well.
A 13th aspect of the invention is the eleventh aspect of the invention, in which the gain calculation portion calculates the gain based on histogram information for the grayscale values of the pixel data of the input signal.
With this image processing device, it becomes possible to perform noise reduction processing that corresponds to the degree to which the noise component is amplified in each pixel due to grayscale conversion, even if histogram equalization is to be executed, for instance.
A 14th aspect of the invention is the eleventh aspect of the invention, in which the gain calculation portion calculates the gain based on target pixel data of the input signal and a signal that is obtained by performing predetermined processing on surrounding pixel data around the target pixel data.
Thus, it becomes possible to perform noise reduction processing that corresponds to the degree to which the noise component of the pixels is amplified due to grayscale conversion, even in a case where visual processing is to be executed on the input signal.
A 15th aspect of the invention is the 12th aspect of the invention, in which the noise reduction degree determining portion sets the noise reduction degree as a first noise reduction degree if the gain of the converted signal is larger than a first gain threshold, and the signal level of the input signal is smaller than a first signal level threshold. The noise reduction degree determining portion also sets the noise reduction degree as a second noise reduction degree, which is a smaller value than the first noise reduction degree, if the gain is smaller than a second gain threshold, which is a smaller value than the first gain threshold, and the signal level of the input signal is smaller than the first signal level threshold. The noise reduction degree determining portion also sets the noise reduction degree as a third noise reduction degree, which is a smaller value than the first noise reduction degree, if the signal level of the input signal is larger than a second signal level threshold, which is a larger value than the first signal level threshold.
With this image processing device, strong noise reduction processing is executed for pixel data whose input signal level (grayscale value of the pixel data of the input signal) is small and whose signal gain for conversion is large, and weak noise reduction processing is executed for pixel data whose input signal level is small and whose signal gain for conversion is small. Weak noise reduction is also performed for pixel data whose input signal level is large.
Thus, even in cases where pixel data making up a dark region of an image that is formed by the input signal are amplified by a large gain to create an output signal, and that output signal is displayed on a display device, it is possible to improve the brightness or contrast of the region corresponding to those pixel data while suppressing the noise component on the display screen.
A 16th aspect of the invention is the 12th aspect of the invention, in which the noise reduction degree determining portion sets a gain-based noise reduction degree to a larger value the larger gain, sets a signal level-based noise reduction degree to a smaller value the larger the signal level of the input signal, and determines the noise reduction degree based on the gain-based noise reduction degree and the signal level-based noise reduction degree.
Thus, it is possible to perform noise reduction processing that corresponds to the grayscale conversion gain and the signal level of the input signal.
It should be noted that here, “setting the gain-based noise reduction degree to a larger value the larger the gain” refers to a relationship in which, for example, the gain-based noise reduction degree increases monotonically with respect to the change in the gain. This includes not only a monotonic increase in a strict sense but also includes a substantially monotonic increase (it can also include ranges which in part are not monotonically increasing). For example, if the gain is smaller than a predetermined value (this shall be called a “third gain value”), then the gain-based noise reduction degree is fixed at a predetermined value (this shall be called a “third gain-based noise reduction degree”), and if the gain is equal to or greater than the third gain value but is equal to or less than a fourth gain value (greater than the third gain value), then the gain-based noise reduction degree is set to a value that monotonically increases with respect to the gain. If the gain is larger than the fourth gain value, then it is set to a value that is larger than the third gain-based noise reduction degree. Such a case also is included.
As the method for determining the noise reduction degree with the noise reduction degree determining portion, it is also possible to use a method in which the mean value (including the arithmetic mean and the geometrical mean) of the gain-based noise reduction degree and the signal level-based noise reduction degree is found and that mean value is established as the noise reduction degree, and a method in which a weighted mean of the gain-based noise reduction degree and the signal level-based noise reduction degree is found and that weighted mean is established as the noise reduction degree.
A 17th aspect of the invention is the 16th aspect of the invention, in which the noise reduction degree determining portion sets the smaller of the gain-based noise reduction degree and the signal level-based noise reduction degree as the noise reduction degree.
Thus, even in cases where pixel data making up a dark region of an image that is formed by the input signal are amplified by a large gain to create an output signal, and that output signal is displayed on a display device, it is possible to improve the brightness or contrast of the region corresponding to those pixel data while suppressing the noise component on the display screen.
An 18th aspect of the invention is the 12th aspect of the invention, in which the noise reduction degree determining portion includes a first noise reduction degree calculation portion, a second noise reduction degree calculation portion, and a noise reduction degree output portion. The first noise reduction degree calculation portion outputs a signal level-based noise reduction degree whose value is smaller than when the signal level of the input signal is equal to or less than a predetermined signal level threshold, if the signal level of the input signal is larger than the predetermined signal level threshold. The second noise reduction degree calculation portion outputs a gain-based noise reduction degree whose value is greater than when the gain is equal to or less than a predetermined gain threshold, if the gain is larger than the predetermined gain threshold. The noise reduction degree output portion sets a value that has been calculated based on the signal level-based noise reduction degree and the gain-based noise reduction degree as the noise reduction degree.
Thus, it is possible to perform noise reduction processing that corresponds to the grayscale conversion gain and the signal level of the input signal.
A 19th aspect of the invention is the 18th aspect of the invention, in which the noise reduction degree output portion sets the smaller of the signal level-based noise reduction degree and the gain-based noise reduction degree as the noise reduction degree.
With this image processing device, strong noise reduction processing is executed for pixel data whose input signal level (grayscale value of the pixel data of the input signal) is small and whose signal gain for conversion is large, and weak noise reduction processing is executed for pixel data whose input signal level is small and whose signal gain for conversion is small. Weak noise reduction is also performed for pixel data whose input signal level is large.
Thus, even in a case where pixel data making up a dark region of an image that is formed by the input signal are amplified by a large gain to create an output signal, and that output signal is displayed on a display device, it is possible to improve the brightness or contrast of the region corresponding to those pixel data while suppressing the noise component on the display screen.
A 20th aspect of the invention is an image processing method that includes a grayscale conversion step of performing grayscale conversion on an input signal that is made from pixel data that form an image to obtain a converted signal, a noise reduction degree determining step of determining a noise reduction degree for the converted signal based on the input signal and the converted signal, and a noise reducing step of executing noise reduction processing on the converted signal based on the noise reduction degree.
With this configuration, from the input signal and the converted signal of each pixel it is possible to calculate a noise reduction degree that expresses a strength of the noise reduction processing to be adopted for each converted signal, and then adjust the strength of the noise reduction processing based on the value of the noise reduction degree that has been calculated. Thus, it becomes possible to perform noise reduction processing that corresponds to the degree to which the noise component is amplified in each pixel due to grayscale conversion.
A 21st aspect of the invention is an image processing method that includes a gain calculation step of calculating a gain for converting an input signal that is made from pixel data that form an image, a multiplication step of multiplying the gain with the input signal to obtain a converted signal, a noise reduction degree determining step of determining a noise reduction degree for the converted signal based on the gain, and a noise reducing step of executing noise reduction processing on the converted signal based on the noise reduction degree.
With this configuration, it is possible to calculate a noise reduction degree that expresses a strength of the noise reduction processing to be adopted for each converted signal from the input signal and the gain of the pixels, and then adjust the strength of the noise reduction processing based on this value. Thus, it becomes possible to perform noise reduction processing that corresponds to the degree to which the noise component is amplified in each pixel due to grayscale conversion.
A 22nd aspect of the invention is the 21st aspect of the invention, in which in the noise reduction degree determining step, a noise reduction degree for the converted signal is determined based on the input signal as well.
A 23rd aspect of the invention is an image processing program for causing a computer to execute a grayscale conversion step of performing grayscale conversion on an input signal that is made from pixel data that form an image to obtain a converted signal, a noise reduction degree determining step of determining a noise reduction degree for the converted signal based on the input signal and the converted signal, and a noise reducing step of executing noise reduction processing on the converted signal based on the noise reduction degree.
With this configuration, from the input signal and the converted signal of the pixels it is possible to calculate a noise reduction degree that expresses a strength of the noise reduction processing to be adopted for each converted signal, and then adjust the strength of the noise reduction processing based on that value. Thus, it becomes possible to perform noise reduction processing that corresponds to the degree to which the noise component is amplified in each pixel due to grayscale conversion.
A 24th aspect of the invention is an image processing program for causing a computer to execute a gain calculation step of calculating a gain for converting an input signal that is made from pixel data that form an image, a multiplication step of multiplying the gain with the input signal to obtain a converted signal, a noise reduction degree determining step of determining a noise reduction degree for the converted signal based on the gain, and a noise reducing step of executing noise reduction processing on the converted signal based on the noise reduction degree.
With this configuration, it is possible to calculate a noise reduction degree that expresses a strength of the noise reduction processing to be adopted for each converted signal from the input signal and the gain of the pixels, and then adjust the strength of the noise reduction processing based on the value of the noise reduction degree that has been calculated. Thus, it becomes possible to perform noise reduction processing that corresponds to the degree to which the noise component is amplified in each pixel due to grayscale conversion.
A 25th aspect of the invention is the 24th aspect of the invention, in which in the noise reduction degree determining step, a noise reduction degree for the converted signal is determined based on the input signal as well.
A 26th aspect of the invention is an integrated circuit that is provided with a grayscale conversion portion that performs grayscale conversion on an input signal that is made from pixel data that form an image to obtain a converted signal, a noise reduction degree determining portion that determines a noise reduction degree for the converted signal based on the input signal and the converted signal, and a noise reducing portion that executes noise reduction processing on the converted signal based on the noise reduction degree.
Thus, it is possible to achieve an integrated circuit that achieves the same effects as the first aspect of the invention.
A 27th aspect of the invention is an integrated circuit that includes a gain calculation portion that calculates a gain for conversion of an input signal that is made from pixel data that form an image, a multiplication portion that multiplies the gain with the input signal to obtain a converted signal, a noise reduction degree determining portion that determines a noise reduction degree for the converted signal based on the gain, and a noise reducing portion that executes noise reduction processing on the converted signal based on the noise reduction degree.
Thus, it is possible to achieve an integrated circuit that achieves the same effects as the eleventh aspect of the invention.
With this invention, it is possible to provide an image processing device, an image processing method, an image processing program, and an integrated circuit with which it is possible to execute noise reduction processing at a different strength for each target pixel, in accordance with the degree of the deterioration of the S/N ratio due to amplification of noise components, not only in the case of grayscale conversion in which only the target pixel is referenced, but also in the case of grayscale conversion in which the region surrounding the target pixel is also referenced.
Consequently, with the invention it is possible to convert the grayscale of an input signal without amplifying noise components.
Below, embodiments of the invention are described in detail with reference to the drawings.
An image processing device according to a first embodiment of the invention is described using
As shown in
With this configuration, the noise reduction degree determining portion 200 determines the noise reduction degree to be applied to each pixel based on the degree that the noise component is amplified by the grayscale conversion portion 100, and the noise reducing portion 300 can execute noise reduction based on the noise reduction degree NR that has been calculated by the noise reduction degree determining portion 200. Thus, an image that has preferable grayscale characteristics can be reproduced without amplifying the noise component when converting the grayscale of an input signal.
The various functional portions of
The grayscale conversion portion 100 inputs an input signal IS and performs grayscale conversion on the input signal IS to correct the brightness or contrast, creating a converted signal TS, and outputs the converted signal TS to the noise reduction degree determining portion 200 and the noise reducing portion 300. It is possible to adopt histogram equalization for the grayscale conversion by the grayscale conversion portion 100, in which grayscale conversion of the input signal IS is performed based on the histogram information of the entire image.
As shown in
Here, the grayscale processing portion 103 inputs the data making up the input/output characteristics HC and converts the input signal IS of a target pixel based on the input/output characteristics HC that have been calculated by the input/output characteristics determining portion 102, yielding a converted signal TS. The grayscale processing portion 103 outputs the converted signal TS that has been obtained to the noise reducing portion 300 and the noise reduction degree determining portion 200.
It should be noted that the method of the grayscale conversion is not limited to a method that is achieved by the grayscale conversion portion 100 shown in
A modified example using a grayscale conversion method that is separate from the grayscale conversion method discussed above is described as the grayscale conversion method.
The grayscale conversion method according to the modified example shown here is a visual processing method in which grayscale conversion is performed based on the target pixel of the input signal and a signal obtained by performing a predetermined processing on the pixels surrounding the target pixel.
Here, the spatial processing portion 151 performs filter processing on the target pixel and the surrounding pixels of the input signal IS. The spatial processing portion 151 for example calculates the surrounding image signal US by executing the following low-pass filter on the target pixel and the surrounding pixels of the input signal IS.
US=(Σ[Wij]×[Aij])÷(Σ[Wij])
Here, [Wij] is the weight coefficient of the pixel located in the i-th row j-th column of a matrix of the target pixel and surrounding pixels, and [Aij] is the pixel value of the pixel that is located in the i-th row j-th column of a matrix of the target pixel and surrounding pixels. The symbol Σ means to take the sum of the values for the target pixel and the surrounding pixels (calculate the sum of the series).
It should be noted that it is possible to assign a weight coefficient with a smaller value the larger the absolute value of the difference between the pixel values, and it is also possible to assign a smaller weight coefficient the greater the distance from the target pixel.
Next, the visual processing portion 152 inputs the input signal IS and the surrounding image signal US, which is the output from the spatial processing portion, and creates and outputs a converted signal TS by converting the grayscale of the input signal IS in accordance with the surrounding image signal US. The visual processing portion 152 for example can perform grayscale conversion based on the two-dimensional grayscale conversion characteristics shown in
As shown in
For example, if the surrounding image signal US is an 8-bit value, then in two-dimensional grayscale conversion, the grayscale conversion characteristics are separated into 256 levels and each of these is expressed as a grayscale conversion curve that has predetermined gamma conversion characteristics.
As shown in
As shown in
With the two-dimensional grayscale conversion characteristics shown in
As described above, the grayscale conversion portion 150 of the modified example makes it possible to execute grayscale conversion that corresponds to the surrounding image information for each pixel in the image.
Next, the noise reduction degree determining portion 200 inputs the input signal IS and the converted signal TS, and from the input signal IS and the converted signal TS calculates a noise reduction degree NR that expresses the strength of the noise reduction processing to be executed on the converted signal TS. The noise reduction degree determining portion 200 then outputs the noise reduction degree NR that it has calculated to the noise reducing portion 300. In general, noise components are amplified more significantly the smaller the signal level (grayscale value) of the input signal IS for the pixel and the larger the gain value that has been used to convert the grayscale of the pixel. Thus, it is preferable to execute stronger noise reduction processing the smaller the signal level (grayscale value) of the input signal IS and the larger the gain of the grayscale conversion, that is, the ratio between the converted signal TS and the input signal IS(=(grayscale value of converted signal TS)/(grayscale value of input signal IS)).
Hereinafter, the noise reduction degree NR is described as taking an (real) value from 0 to 1, in which the larger the value, the stronger the noise reduction processing that is performed.
As shown in
Here, it is desirable for the first noise reduction degree calculation portion 201 to execute stronger noise reduction processing the smaller the signal level (grayscale value) of the pixel, and thus it preferably outputs a larger noise reduction degree NRA the smaller the signal level (grayscale value) of the input signal IS.
On the other hand, because it is desirable for the second noise reduction degree calculation portion 203 to execute stronger noise reduction the higher the gain that has been used to convert the grayscale of the pixel, preferably it outputs a larger noise reduction degree NRB the larger the ratio DIV between the converted signal TS and the input signal IS.
Next, the minimum value output portion 204 outputs the smaller of the first noise reduction degree NRA that has been calculated by the first noise reduction degree calculation portion 201 and the second noise reduction degree NRB that has been calculated by the second noise reduction degree calculation portion 203, and thus it is possible to calculate a larger noise reduction degree NR the smaller the signal level (grayscale level) of the input signal IS for the pixel and the larger the gain that has been used to convert the grayscale of the pixel. The noise reduction degree determining portion 200 outputs the noise reduction degree NR that has been calculated to the noise reducing portion 300.
It should be noted that the processing by the minimum value output portion 204 is not limited to the processing described above, and for example it may also be processing in which the mean value (including the arithmetic mean and the geometrical mean) of the first noise reduction degree NRA and the second noise reduction degree NRB is found and that mean value is taken as the noise reduction degree NR, and processing in which the weighted mean of the first noise reduction degree NRA and the second noise reduction degree NRB is found and that weighted mean is taken as the noise reduction degree NR. Here, the weighted mean is obtained by first finding the smaller of the two values for the first noise reduction degree NRA and the second noise reduction degree NRB (this smaller value shall be called “value A” and the larger value shall be called “value B”) and assigning a large weight to the value A and a small weight to the value B, and then averaging these to obtain the weighted mean. For example, it is possible to find the weighted mean through the equation (weighted mean)=((value A)×3+(value B))/4.
Next, the noise reducing portion 300 performs noise reduction processing on the converted signal TS based on the noise reduction degree NR.
As shown in
Here, the smoothing portion 301 can be achieved by applying a low-pass filter, for example, to a plurality of pixels surrounding a target pixel.
The interpolation by the interpolation portion 302 can be achieved through the interpolation of the following equation, using the noise reduction degree NR that has been calculated by the noise reduction degree determining portion 200.
OS=LPS×NR+TS×(1−NR)
According to this equation, when the noise reduction degree NR is large, the resulting output is a value in which the smoothed signal LPS is weighted heavily, yielding a result (output) in which the noise has been strongly eliminated. Conversely, when the noise reduction degree NR is small, the resulting output is a value in which the converted signal TS is weighted heavily, yielding a result (output) in which the noise has been weakly eliminated.
Noise tends to stand out on the display screen in cases where an output signal OS that is obtained by applying a large gain to an input signal IS that corresponds to a target pixel with a small grayscale value (such a target pixel is displayed dark on the display screen) is displayed on a display device.
In other words, an input signal IS that corresponds to a target pixel with a small grayscale value has a small signal level (grayscale value) for the target pixel, and thus assuming an equal noise component, the S/N ratio is poor compared to a case in which the signal level (grayscale value) of the target pixel is large. Thus, when an output signal OS that has been obtained by amplifying this signal with a poor S/N ratio is displayed on a display device, the display has a poor S/N ratio and noise stands out easily.
Even if processing that causes blurring (such as processing in which LPF is applied) is executed on a target pixel with a small grayscale value, it is difficult to sense a deterioration in terms of the visual characteristics on the display screen.
Thus, it is preferable to perform strong noise reduction on regions that correspond to pixels with a small grayscale value, and on regions that have been amplified with a large gain.
On the other hand, an input signal IS that corresponds to a target pixel with a large grayscale value has a large signal level (grayscale value) for the target pixel, and thus assuming an equal noise component, the S/N ratio is good compared to a case in which the signal level (grayscale value) of the target pixel is small. Thus, if this is amplified by a large gain, noise does not stand out easily on the display screen.
Conversely, when processing that causes blurring (such as processing in which LPF is applied) is executed on a target pixel with a large grayscale value, it is easy to sense a deterioration in the visual characteristics on the display screen.
Thus, it is preferable that weak noise reduction is performed on regions that correspond to pixels with a large grayscale value.
The characteristics shown in
As described above, with the image processing device 1000 according to this embodiment, the degree to which the noise component of each pixel is amplified can be found from the signal level (grayscale value) of the input signal IS and the value of the gain if the input signal IS is grayscale converted to the converted signal TS, and thus by performing noise reduction at a suitable strength based on the degree to which the noise component will be amplified, it is possible to obtain an output signal OS whose noise component has been suppressed and which has preferable grayscale characteristics. With the image processing device 1000 according to the embodiment, it is possible to execute preferable grayscale conversion on an input signal IS without amplifying the noise component of the input signal IS, and output the result as the output signal OS, and thus by displaying the output signal OS that is output from the image processing device 1000 according to this embodiment on a display device (not shown) as an image (video picture), it is possible to reproduce an image (video picture) that has preferable grayscale characteristics.
It should be noted that it is also possible for the noise reduction degree NR to be calculated from only the ratio DIV of the converted signal TS and the input signal IS. Thus, it is possible to execute strong noise reduction on all pixels that have been processed with a high gain, regardless of the value of the input signal IS.
It is also possible for the noise reduction degree NR to be calculated based on the difference between the converted signal TS and the input signal IS. Thus, it is possible to execute strong noise reduction on pixels in which the converted signal TS is significantly higher than the input signal IS.
It should be noted that that the strength of the noise reduction processing by the noise reducing portion 300 can be adjusted by changing the filter coefficient of the low-pass filter that is applied to the converted signal TS. Thus, it is possible to adjust the degree to which high-frequency components are reduced by the low-pass filter, and the degree to which the noise is reduced can be adjusted.
An image processing device according to a second embodiment of the invention is described using
In the image processing device 1000 according to the first embodiment of the invention, once a converted signal TS has been calculated from the input signal IS by the grayscale conversion portion 100, the noise reduction degree determining portion 200 calculates the noise reduction degree NR from the input signal IS and the converted signal TS. With the image processing device 2000 according to the second embodiment of the invention, once a gain GN for grayscale conversion has been calculated from the input signal IS, the gain GN and the input signal IS are multiplied to calculate a converted signal TS, and the noise reduction degree NR is calculated from the input signal IS and the gain GN. This embodiment is described below using
In
When this configuration is used, the noise reduction degree determining portion 210 can specify pixels whose noise component has been amplified significantly when converted from the input signal IS to the converted signal TS, and the noise reducing portion 300 can execute suitable noise reduction processing based on the noise reduction degree NR that has been calculated by the noise reduction degree determining portion 210.
Thus, with the image processing device 2000 of this embodiment, it is possible to suitably execute noise reduction processing even on pixels whose noise component is amplified significantly when grayscale conversion is performed by multiplying the input signal and the gain, and thus by displaying the output signal OS that has been processed and output by the image processing device 2000 of this embodiment as an image (video picture) on a display device (not shown), it is possible to reproduce an image (video picture) that has preferable grayscale characteristics without amplifying noise components in the image.
Below, the functional portions in
The gain calculation portion 110 calculates the gain GN, which is the gain value for correcting the brightness or contrast of the input signal IS. The brightness or the contrast of the input signal IS is corrected by multiplying the gain GN and the input signal IS.
To achieve the gain calculation portion 110, it is for example possible to adopt an approach that is based on histogram equalization, in which the grayscale of the input signal IS is transformed based on the histogram information.
As shown in
In
Next, the grayscale conversion gain characteristics determining portion 112 determines the gain output characteristics GC, which have been set so that a wide range of grayscale is allocated the more frequently a grayscale level (grayscale value) appears in the input signal IS based on the histogram information HG, for each target pixel of the input signal IS, for each predetermined block (region) made of a plurality of pixels that includes the target pixel, or for the entire image.
The grayscale processing gain calculation portion 113 outputs a gain GN for multiplication with the input signal IS of the target pixel, to the multiplication portion 120 and the noise reduction degree determining portion 210, based on the gain output characteristics GC that have been calculated by the grayscale conversion gain characteristics determining portion 112.
It should be noted that the method for calculating the gain is not limited to the method that is achieved by the gain calculation portion 110 shown in
A modified example using a gain calculation method that is separate from the gain calculation method discussed above is described as a method for calculating the gain using grayscale conversion (gain calculation method).
The method of grayscale conversion (grayscale conversion using a gain) according to the modified example shown here is a method based on visual processing in which a gain is calculated based on the target pixel of the input signal IS and a signal obtained by performing predetermining processing on the pixels surrounding the target pixel.
The processing by the spatial processing portion 151 is the same as in the first embodiment, which was described in
Next, the visual processing gain calculation portion 162 calculates a gain GN to be multiplied with the input signal IS according to the input signal IS and the surrounding image signal US. Like in the modified example of the first embodiment, the processing in this case also is two-dimensional grayscale conversion, because the gain GN for converting the grayscale of the input signal IS is determined based on the two inputs of the surrounding image signal US and the input signal IS. Here, in order to achieve the same grayscale conversion characteristics as the two-dimensional grayscale conversion characteristics shown in
As shown in
In two-dimensional gain conversion, for example, when the surrounding image signal US is an 8-bit value, the gain conversion characteristics are separated into 256 levels and these can be expressed as conversion curves each having predetermined gain GN output characteristics (gain conversion characteristics).
As shown in
The gain conversion characteristics that are corresponded with the two-dimensional grayscale conversion characteristics shown in
In other words, the two-dimensional grayscale conversion characteristics that were described in the first embodiment are achieved by multiplying the input signal IS with the gain GN, which is determined by the gain conversion characteristics shown in
According to the conversion characteristics shown in
Next, with the multiplication portion 120, the gain GN that has been obtained is multiplied with the input signal IS to obtain a converted signal TS.
Next, the noise reduction degree determining portion 210 inputs the input signal IS and the gain GN, and from the input signal IS and the gain GN calculates the noise reduction degree NR to be effected on the converted signal TS. The noise reduction degree determining portion 210 then outputs the noise reduction degree NR that it has calculated to the noise reducing portion 300. In general, the noise component is more significantly amplified for pixels the smaller the signal level (grayscale value) of the input signal IS and the larger the gain that is used for grayscale conversion. For this reason, it is preferable to execute stronger noise reduction processing the smaller the signal level (grayscale value) of the input signal IS and the larger the gain GN of the grayscale conversion.
As shown in
Like in the case described in the first embodiment, as shown in
Next, the minimum value output portion 204 outputs the smaller of the first noise reduction degree NRA that has been calculated by the first noise reduction degree calculation portion 201 and the second noise reduction degree NRC that has been calculated by the second noise reduction degree calculation portion 213, and thus a larger value is calculated as the noise reduction degree NR for pixels the smaller the signal level (grayscale value) of the input signal IS and the larger the gain GN with which processing has been performed. The noise reduction degree determining portion 210 outputs the noise reduction degree NR that has been calculated to the noise reducing portion 300. It should be noted that the processing by the minimum value output portion 204 is not limited to the processing described above, and for example it may also be processing in which the mean value (including the arithmetic mean and the geometrical mean) of the first noise reduction degree NRA and the second noise reduction degree NRC is found and that mean value is taken as the noise reduction degree NR, and processing in which the weighted mean of the first noise reduction degree NRA and the second noise reduction degree NRC is found and that weighted mean is taken as the noise reduction degree NR. Here, the weighted mean is obtained by first finding the smaller of the two values for the first noise reduction degree NRA and the second noise reduction degree NRC (this smaller value shall be called “value A” and the larger value shall be called “value B”) and assigning a large weight to the value A and a small weight to the value B, and then averaging these to obtain the weighted mean. For example, it is possible to find the weighted mean through the equation (weighted mean)=((value A)×3+(value B))/4.
Next, the noise reducing portion 300 performs noise reduction processing on the converted signal TS based on the noise reduction degree NR. This processing is the same as that of the noise reducing portion 300 of the first embodiment, and thus will not be described in detail.
As described above, according to the image processing device 2000 of this embodiment, the degree with which the noise component of each pixel is amplified can be ascertained from the signal level (grayscale value) of the input signal IS and the value of the gain in a case where the input signal IS is grayscale converted by the gain into a converted signal TS (gain-type grayscale conversion), and thus by executing noise reduction at a strength that is appropriate for the degree to which the noise component is amplified, it is possible to obtain an output signal OS whose noise component has been suppressed and which has preferable grayscale characteristics. With the image processing device 2000 of this embodiment, it is possible to execute preferable gain-type grayscale conversion on an input signal IS without amplifying the noise component of the input signal IS, and output this as an output signal OS, and thus by displaying the output signal OS that is output from the image processing device 2000 according to this embodiment on a display device (not shown) as an image (video picture), it is possible to reproduce an image (video picture) that has preferable grayscale characteristics.
It should be noted that the noise reduction degree NR also can be calculated from the gain GN only. By doing this, it is possible to execute strong noise reduction processing on all pixels that have been processed with a high gain, regardless of the value of the input signal IS.
It is also possible to adjust the strength of the noise reduction processing by the noise reducing portion 300 by changing the filter coefficient of the low-pass filter that is applied to the converted signal TS. By doing this, it becomes possible to adjust the degree to which high-frequency components are reduced by a low-pass filter, and thus the degree to which noise is reduced can be adjusted.
It should be noted that the present invention has been described based on the above embodiments, but the invention is of course not limited to the embodiments discussed above. The invention is also inclusive of the following cases.
(1) The above devices specifically are computer systems made from a microprocessor, ROM, and RAM, for instance. The RAM stores a computer program. The microprocessor operates in accordance with the computer program, allowing each device to achieve its function. Here, in order to achieve a predetermined function, the computer program is arrived at by combining a plural number of command codes for indicating an order to the computer.
(2) Some or all of the structural elements making up the devices discussed above can be constituted by a single system LSI (Large Scale Integration). The system LSI is a multifunctional LSI that is produced by integrating a plurality of structural portions on a single chip, and specifically is a computer system that is constituted by a microprocessor, ROM, and RAM, for instance. The RAM stores a computer program. The microprocessor operates in accordance with the computer program, allowing the system LSI to achieve its function.
(3) Some or all of the structural elements making up the devices discussed above can be constituted by an IC card or a single module that can be attached to and detached from the devices. The IC card or the module is a computer system that is constituted by a microprocessor, ROM, and RAM, for instance. The IC card or the module may also include the multifunctional LSI discussed above. The microprocessor operates in accordance with a computer program, allowing the IC card or the module to achieve its function. It is also possible for the IC card or the module to be tamper-resistant.
(4) The invention also may be the methods indicated in the above discussion. It is possible for these methods to be a computer program that is achieved by a computer, or a digital signal that is made from the computer program. The invention also may be a computer-readable recording medium, such as a flexible disk, a hard disk, a CD-ROM, a MO, a DVD, a DVD-ROM, a DVD-RAM, a BD (Blu-ray Disc), or a semiconductor memory, on which the computer program or the digital signal is stored. The invention also may be the digital signal stored on these recording media.
The invention can be the computer program or the digital signal transferred via an electronic transmission line, a wireless or a wired communications line, a network such as the internet, or a data broadcast.
It is also possible for the invention to be a computer system provided with a microprocessor and a memory, in which the memory stores the computer program, and the microprocessor is operated according to the computer program.
By storing the program or the digital signal on a recording medium that is then transported, or by sending the program or the digital signal via a network, for example, it is possible to run the program or the digital signal on independent, separate computer systems.
(5) It is possible to combine the embodiments and the modified examples.
It is possible to conduct some or all of the processing of the embodiments with the pixel serving as the processing unit, and it is also possible to conduct some or all of the processing with blocks that are made from a plurality of pixels serving as the processing unit.
Each of the processing functions explained in the aforementioned embodiments may be carried out by hardware, or by software. Alternatively, it may be carried out by mixed processing using the hardware and software.
It should be noted that the specific configuration of the invention is not limited to the foregoing embodiments, and various changes and modifications are possible in a range that does not depart from the gist of the invention.
The image processing device, the image processing method, the image processing program, and the integrated circuit of the invention can reproduce an image that has preferable grayscale characteristics without amplifying noise components in the image, and thus they are useful in industrial fields related to image processing, and the image processing device, the image processing method, the image processing program, and the integrated circuit of the invention can be put to use in those fields.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2006-159432 | Jun 2006 | JP | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 4367490 | Riederer | Jan 1983 | A |
| 4984071 | Yonezawa | Jan 1991 | A |
| 5517333 | Tamura et al. | May 1996 | A |
| 6058248 | Atkins et al. | May 2000 | A |
| 6101271 | Yamashita et al. | Aug 2000 | A |
| 7453523 | Yamauchi | Nov 2008 | B2 |
| 7570390 | Mitsunaga | Aug 2009 | B2 |
| 7719551 | Aoki et al. | May 2010 | B2 |
| 7778478 | Kuniba | Aug 2010 | B2 |
| 7973860 | Hsieh et al. | Jul 2011 | B2 |
| 8023733 | Monobe et al. | Sep 2011 | B2 |
| 8035853 | Sambongi et al. | Oct 2011 | B2 |
| 8059904 | Hoshuyama | Nov 2011 | B2 |
| 8326072 | Someya et al. | Dec 2012 | B2 |
| 8340419 | Monobe et al. | Dec 2012 | B2 |
| 20040036703 | Aoki et al. | Feb 2004 | A1 |
| 20040234155 | Hoshuyama | Nov 2004 | A1 |
| 20050127846 | Kim | Jun 2005 | A1 |
| 20050226526 | Mitsunaga | Oct 2005 | A1 |
| 20050265627 | Yamauchi | Dec 2005 | A1 |
| 20060215925 | Kuniba | Sep 2006 | A1 |
| 20070165282 | Sambongi et al. | Jul 2007 | A1 |
| 20070286481 | Monobe et al. | Dec 2007 | A1 |
| 20090041348 | Someya et al. | Feb 2009 | A1 |
| 20110299773 | Monobe et al. | Dec 2011 | A1 |
| 20130018265 | Kim et al. | Jan 2013 | A1 |
| Number | Date | Country |
|---|---|---|
| 04-282689 | Oct 1992 | JP |
| 2001-186378 | Jul 2001 | JP |
| 2001-309177 | Nov 2001 | JP |
| 2004-007202 | Jan 2004 | JP |
| 2004-080299 | Mar 2004 | JP |
| 2004-200971 | Jul 2004 | JP |
| 2005-275581 | Oct 2005 | JP |
| 2005-318063 | Nov 2005 | JP |
| 3723043 | Dec 2005 | JP |
| 2005055588 | Jun 2005 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20130084021 A1 | Apr 2013 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 13210580 | Aug 2011 | US |
| Child | 13685824 | US | |
| Parent | 11806811 | Jun 2007 | US |
| Child | 13210580 | US |
