Noise reduction apparatus, method and program for controlling same, image sensor and digital camera

Abstract

Image data pixels indicative of the pixels in a noise-reduction target area having a size of 5×5 pixels is extracted from a plurality of types of CCD-RAW data having red, green and blue color components. A filter for reducing uncorrelated noise is calculated. Uncorrelated noise is removed by performing a filter operation using the calculated filter while correlativity of the CCD-RAW data is maintained. These processing steps are repeated for one frame of CCD-RAW data. After uncorrelated noise has been removed, spatial pixel processing such as an aperture correction is applied.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates part of the photoreceptor surface of a CCD;

FIG. 2 illustrates part of the photoreceptor surface of a CCD;

FIG. 3 illustrates part of the photoreceptor surface of a CCD;

FIG. 4 illustrates part of the photoreceptor surface of a CCD;

FIG. 5 illustrates part of the photoreceptor surface of a CCD;

FIG. 6 illustrates part of the photoreceptor surface of a CCD;

FIG. 7 illustrates the manner in which pixels are staggered;

FIG. 8 illustrates the relationship between color space and image data;

FIG. 9 illustrates the relationship between color space and image data;

FIG. 10 illustrates the relationship between color space and image data;

FIG. 11 illustrates the relationship between color space and image data;

FIG. 12 illustrates the structure of an image sensor;

FIG. 13 is a block diagram illustrating the electrical structure of a digital still camera;

FIG. 14 illustrates the relationship between amount of noise and pixel level;

FIG. 15 is a block diagram illustrating the electrical structure of a noise reduction circuit;

FIG. 16 is a block diagram illustrating the electrical structure of a computer system;

FIG. 17 is a flowchart illustrating noise reduction processing;

FIG. 18 illustrates an example of the image of a subject;

FIG. 19 illustrates the relationship between color space and image data;

FIG. 20 illustrates an example of an area to undergo noise reduction;

FIG. 21 illustrates an example of an area to undergo noise reduction;

FIG. 22 is a flowchart illustrating processing for area discrimination and the like;

FIG. 23 is a flowchart illustrating part of processing for area discrimination and the like;

FIG. 24 illustrates an example of an area to undergo noise reduction;

FIG. 25 illustrates an example of an area to undergo noise reduction;

FIG. 26 illustrates an example of pixels constituting a small block;

FIG. 27 illustrates the relationship between shooting sensitivity and luminance;

FIG. 29 illustrates an example of a pixel array;

FIG. 30 illustrates an example of a pixel array;

FIG. 31 illustrates part of the photoreceptor surface of a CCD;

FIG. 32 illustrates part of the photoreceptor surface of a CCD;

FIG. 33 illustrates part of the photoreceptor surface of a CCD; and

FIG. 34 illustrates pixel division processing.

Claims

1. A noise removal apparatus comprising: a noise removal device for removing uncorrelated noise contained in a plurality of types of color image data while maintaining correlativity of each of the items of color image data in the plurality of types of color image data, which represent substantially identical color images of a subject obtained by sensing the image of the same subject using a plurality of solid-state electronic image sensing devices; anda pixel processing device for executing spatial pixel processing with regard to the plurality of types of color image data from which uncorrelated noise has been removed by said noise removal device.

2. The apparatus according to claim 1, wherein the plurality of types of color image data have color components that are different from one another.

3. The apparatus according to claim 1, wherein the plurality of color images of the subject represented by the plurality of types of color image data are such that phase characteristics of the pixels that constitute respective ones of the plurality of color images of the subject are identical or exhibit a deviation of less than pixel pitch.

4. The apparatus according to claim 1, further comprising a noise removal processing suppression device for suppressing noise removal processing by said noise removal device in accordance with the level of color image data.

5. The apparatus according to claim 1, wherein uncorrelated noise removal processing in said noise removal device is executed with regard to at least one of a characteristic of the solid-state electronic image sensing devices and shooting information used when a subject is shot using the solid-state electronic image sensing devices.

6. The apparatus according to claim 1, wherein the plurality of types of color image data have color components that are different from one another; said noise removal device including:a color image data shifting device for shifting the levels of the plurality of types of color image data in such a manner that an average value of the levels of the plurality of types of color image data will become the position of the origin of color space of the plurality of color components;a filtering device for subjecting color image data, which has been shifted by said color image data shifting device, to processing for removing uncorrelated noise in accordance with the level of this color image data; anda color image data reverse shifting device for returning, in accordance with the amount of shift, the level of the color image data from which uncorrelated noise has been removed by said filtering device.

7. The apparatus according to claim 6, wherein noise removal processing in said filtering device is digital filtering processing that utilizes calculation conforming to the number of the plurality of color components.

8. The apparatus according to claim 6, wherein noise removal processing in said filtering device is digital filtering processing for handling the plurality of color components in such a manner that the number of the plurality of color components will become less than the original number of color components, with use being made of calculation conforming to the number of the plurality of color components.

9. The apparatus according to claim 8, further comprising a noise removal processing halting device for halting noise removal processing in a case where the operation in the digital filter processing has diverged.

10. The apparatus according to claim 6, further comprising an image data dividing device for dividing the plurality of types of color image data into a plurality of color image data groups including color image data exhibiting correlativity with one another; wherein said color image data shifting device shifts a color image data group, which contains color image data that is to undergo removal of uncorrelated noise, from among the plurality of color image data groups obtained by division in said image data dividing device.

11. The apparatus according to claim 10, wherein said image data dividing device includes a small-block dividing device for dividing the plurality of types of color image data into a plurality of small blocks; each of the plurality of small blocks obtained by division in said small-block dividing device being divided into a plurality of color image data groups exhibiting correlativity.

12. The apparatus according to claim 11, wherein said image data dividing device divides each of the plurality of small blocks into a plurality of color image data groups exhibiting correlativity, based upon representative pixels constituting the small blocks.

13. The apparatus according to claim 11, wherein said image data dividing device divides each of the plurality of small blocks into a plurality of color image data groups exhibiting correlativity, based upon the average pixel level of the plurality of small blocks.

14. The apparatus according to claim 11, wherein said image data dividing device divides each of the plurality of small blocks into a plurality of color image data groups exhibiting correlativity, based upon a pixel level obtained by application of a weighted mean to each of the pixels constituting the plurality of small blocks, the weighted mean using weighting coefficients such that the pixel level becomes the average pixel level as sensitivity of image sensing increases, and becomes a representative-pixel level of pixels constituting the small blocks as sensitivity of image sensing decreases.

15. The apparatus according to claim 10, further comprising a noise removal halting device for halting uncorrelated noise removal processing by said noise removal device in a case where the size of an image area represented by the color image data group obtained by division in said image data dividing device is less than a first predetermined value.

16. The apparatus according to claim 15, wherein in a case where the size of an image area represented by the color image data group is equal to or greater than a second predetermined value larger than the first predetermined value, a color image data group is shifted, this group being one which contains color image data that is to undergo removal of uncorrelated noise from among the plurality of color image data groups obtained by division in said image data dividing device.

17. The apparatus according to claim 16, wherein in a case where the size of an image area represented by the plurality of color image data groups is equal to or greater than a third predetermined value larger than the second predetermined value, uncorrelated noise removal processing by said noise removal device is halted and noise in each item of color image data is removed based upon the difference between a dispersion value of each item of color image data containing noise and a dispersion value of the noise.

18. The apparatus according to claim 10, further comprising a luminance data generating device for generating luminance data from the color image data; wherein said color image data dividing device divides the luminance data, which has been generated by said luminance data generating device, into a plurality of luminance data groups that exhibit correlativity.

19. The apparatus according to claim 6, further comprising: a block dividing device for dividing the color image data of respective ones of the plurality of items of color image data into a plurality of close-together blocks in color space; anda level difference calculation device for a calculating level difference with respect to a central pixel among pixels that constitute each of the blocks obtained by division by said block dividing device;wherein said color image data shifting device shifts image data representing a collection of pixels for which level difference calculated by said level difference calculation device is less than a predetermined value.

20. The apparatus according to claim 19, wherein said level difference calculation device includes: a detection device for detecting the direction in which the slope of a neighboring area, which contains a target pixel for which the level difference with respect to the center pixel is calculated, is small; andan average level calculation device for calculating an average level of pixels, inclusive of the target pixel, that are present along the direction in which slope is detected to be small by said detecting device;said level difference calculation device calculating a level difference between the average level calculated by said average level calculation device and the level of the target pixel being calculated.

21. The apparatus according to claim 19, wherein said detecting device includes a high-pass filter for extracting high-frequency components along the vertical direction, horizontal direction, northwest direction and northeast direction with respect to the target pixel and pixels around the target pixel; said detecting device detecting the direction in which slope is small based upon the high-frequency components extracted by the high-pass filter device.

22. A noise reduction apparatus comprising: a dividing device for dividing color image data representing one frame of a color image into a plurality of blocks in which pixels are spatially close together;a calculation device for calculating a difference between a dispersion value of block color image data representing an image in a block obtained by division by said dividing device and a dispersion value of noise in the block color image data;a noise removal device for removing noise in the color image data based upon the difference calculated by said calculation device; anda control device for exercising control in such a manner that calculation processing by said calculation device and noise removal processing by said noise removal device will be repeated with regard to the color image data representing one frame of the color image.

23. A noise reduction apparatus comprising: an extraction device for inputting image data representing one frame of an image constituted by a number of pixels and extracting image data representing a pixel within a zone in which the color image data is regarded as having correlativity;a noise removal device for removing uncorrelated noise while maintaining correlativity of the image data extracted by said extraction device; anda control device for exercising control in such a manner that extraction processing by said extraction device and noise removal processing by said noise removal device is repeated with regard to the image data representing one frame of the image.

24. A method of controlling a noise reduction apparatus, comprising the steps of: removing uncorrelated noise contained in a plurality of types of color image data while maintaining correlativity of each of the items of color image data in the plurality of types of color image data, which represent substantially identical color images of a subject obtained by sensing the image of the same subject using a plurality of solid-state image sensing devices; andexecuting spatial pixel processing with regard to the plurality of types of color image data from which uncorrelated noise has been removed.

25. A method of controlling a noise reduction apparatus, comprising the steps of: dividing color image data representing one frame of a color image into a plurality of blocks in which pixels are spatially close together;calculating a difference between a dispersion value of block color image data representing an image in a block obtained by division and a dispersion value of noise in the block color image data;removing noise in the color image data based upon the difference calculated; andrepeating difference calculation processing and noise removal processing with regard to the color image data representing one frame of the color image.

26. A method of controlling a noise reduction apparatus, comprising the steps of: inputting image data representing one frame of an image constituted by a number of pixels and extracting image data representing a pixel within a zone in which the color image data is regarded as having correlativity;removing uncorrelated noise while maintaining correlativity of the image data extracted; andexercising control in such a manner that extraction processing and noise removal processing is repeated with regard to the image data representing one frame of the image.

27. A program for controlling a noise reduction apparatus so as to cause said noise reduction apparatus to implement the following steps: removing uncorrelated noise contained in a plurality of types of color image data while maintaining correlativity of each of the items of color image data in the plurality of types of color image data, which represent substantially identical color images of a subject obtained by sensing the image of the same subject using a plurality of solid-state image sensing devices; andexecuting spatial pixel processing with regard to the plurality of types of color image data from which uncorrelated noise has been removed.

28. A program whereby a computer of a noise removal apparatus executes the steps of: dividing color image data representing one frame of a color image into a plurality of blocks in which pixels are spatially close together;calculating a difference between a dispersion value of block color image data representing an image in a block obtained by division and a dispersion value of noise in the block color image data;removing noise in the color image data based upon the difference calculated; andrepeating difference calculation processing and noise removal processing with regard to the color image data representing one frame of the color image.

29. A program whereby a computer of a noise removal apparatus executes the steps of: inputting image data representing one frame of an image constituted by a number of pixels and extracting image data representing a pixel within a zone in which the color image data is regarded as having correlativity;removing uncorrelated noise while maintaining correlativity of the image data extracted; andexercising control in such a manner that extraction processing and noise removal processing is repeated with regard to the image data representing one frame of the image.

30. An image sensor having the noise removal apparatus set forth in claim 1.

31. A digital still camera having the noise removal apparatus set forth in claim 1.